Tuesday, November 24, 2009

Test Out Our New Survey Tool

We just launched the first ever Silverlight-based Survey Design Tool. You can now design and publish email survey invites through the JangoMail interface and view your survey results for each individual participant.

Set Up Instructions

1. Click on the new Surveys tab. If you don't have Microsoft's Silverlight Platform installed, you will be prompted to install it at this time. Survey recipients see the survey on a regular webpage and will not need to download Silverlight.

2. Click the Create a New Survey button to get started. Then choose the Blank survey option.
3. To create a new question, click in the box under Create a new item.

Edit the question and answer points by deleting the text that is currently there and typing in your own.
To use a pre-defined set of answers, click the down arrow next to your answer set.
To add space for short text answers and comment sections, click on the down arrow in the box under Create a new item. Then click on the type of answer you would like.

4. When you are done creating your survey, click the Save button.
5. To make the survey available for people to respond, you must publish it. Click on the Publish tab and then click Publish Survey. Here you will get a link that you can send to people to fill out your survey.
6. Set your survey to end by navigating to the Close tab and entering in when the survey should close. You can choose to end it immediately, on a certain date, or after a given number of responses.

Send the Survey
1. When you publish your survey you will be provided with a link. Copy that link.
2. Create an email message in the Messages section as you normally would. Paste in the link that you copied where appropriate. After testing your email and your link, send the email out to your survey list.

View Results
View Recipient-Level Results by clicking on My Surveys in the Surveys section. Click on the Results button next to the survey that you would like to view results for.

We will soon add results in our Reporting and Analytics section as well.

Monday, November 23, 2009

Test our new no-frames interface

Our new no-frames user interface is ready to be tested. To switch to the no-frames interface, login to your account and click the "Switch to No-Frames" link. You can always switch back to the old frames interface if you want to later.

Below is the first screen of the new interface:

Along with the absence of frames, there are three key navigational differences:
  1. The "Account Info" section is no longer a main tab. It is a link in the top area.
  2. The "Help" section is no longer a main tab. It is a link in the top area.
  3. There is a new tab called "Surveys". You can now design and publish and email survey invites to your email lists. The Surveys feature is also in beta. The survey builder runs on Microsoft's Silverlight platform, so if you don't have Silverlight installed, the first time you click on the "Surveys" tab you'll be asked to install Silverlight.
The new no-frames interface will now be the basis for a whole host of user interface improvements, including improved text styling, contextual help, more videos, easier navigation, and clearer organization. Stay tuned for these improvements and future blog posts on how to use Surveys.

Error handling for the SMTP Relay Service


We've launched a new error handling system for the SMTP relay system. There are a number of reasons why a message sent through the SMTP relay may result in an error. For example:

1. If the recipient address is on your account's bounce list, then a "bounce" error will be thrown.
2. If the recipient address is on your account's unsubscribe list, then an "unsubscribe" error will be thrown.
3. If your account is over its sending limits, a "Over Account Limit" error will be thrown.
4. If a badly formed email message is passed through the relay, a "parsing" error will be thrown.

These errors are only detected after a message is received by the SMTP relay, so the error will NOT appear during the SMTP conversation with relay.jangosmtp.net.

When an error is detected, the error notification can be emailed to the Account Manager and/or the From Address on the email that resulted in the error. Additionally, the error is available for viewing under Reporting in the Transactional Emails section.

To set your error handling preferences, go to Settings --> SMTP Relay and set the two relevant checkboxes appropriately.


1. If you know that you regularly attempt to relay emails to addresses on your Unsubscribe and Bounce lists, you may wish to turn off the email notifications for errors entirely, as this could result in a large volume of emails to you.

2. To decide whether or not the error should also be sent to the From Address, consider what type of transactional emails you're sending through the SMTP relay. If the From Address is an address at your organization, then it's probably okay for the From Address to also receive errors. But, if you're sending social networking "You have a new message from John" type emails, where the From Address is that of one of your users, you probably don't want your users to receive error notifications from JangoMail. In this case, uncheck the box to send errors to the From Address.

Thursday, November 19, 2009

Eight Steps to Optimize Email Deliverability Using JangoMail

Email deliverability is a key concern for most email marketers, and at JangoMail, we think we do a pretty good job of ensuring the highest possible inbox placement of our clients' emails. JangoMail is a highly customizable platform, from sending speeds, to email headers, to SMTP protocol level customizations. Taking advantage of certain settings and features can help ensure high email deliverability.

Some of these steps involve separating yourself from other JangoMail clients. It's not that JangoMail accepts spammers, but JangoMail does accept different levels of opt-in email marketing. For example, some customers use single opt-in, while other customers use confirmed opt-in. Generally, customers using confirmed opt-in experience higher deliverability. Therefore, different JangoMail clients have different reputations with ISPs, so if two clients are both using a username@jangomail.com From Address for example, they both can impact the reputation of the domain jangomail.com.

The Eight Steps:

Following the below steps will solve most email delivery issues to major consumer ISPs, like Yahoo, Hotmail, Gmail, and AOL.

  1. Use your own domain name in the FROM address. Brand your messages using your own domain name in the FROM address. This requires using an SPF record (see #5 below) that includes jangomail.com. If you use your_username@jangomail.com, the delivery of your messages is mixed in with everyone else using jangomail.com as their FROM address. All JangoMail accounts come with the option to use username@jangomail.com as the FROM address on email campaigns, but that's meant to help smaller volume users to get started, and is provided as a convenience to our clients. If you have your own domain name, use it - it only helps.

    a. In order to separate your reputation from other clients using jangomail.com as their FROM address, use an actual company (or based on your domain name/website) email address, such as john.smith@company.com.

    b. You can also send using a subdomain within your company, using an address such as marketing@newsletters.company.com. And, if you want JangoMail to handle reply management for that subdomain (our branded subdomain feature), let us know via a support ticket. Replies to your messages can be forwarded to an address of your choosing (replies normally go to the FROM address you used in your message). For instructions, see the PDF Setting up a Branded Sub-Domain.

    Who should do this? The technical person who manages your domain's DNS settings.
  2. Use a custom tracking domain based on your domain name. The tracking domain is the domain referenced in the URLs for our open-tracking mechanism, click tracking, the unsubscribe link used in your CAN-SPAM footer, the forward-to-friend link, the view-as-a-web-page link, and other links that offer tracking in your email messages. By default, every JangoMail account is assigned a system tracking domain that is shared amongst multiple clients. Example names are x.jango5.com and x.jmxded92.net. By setting up your own tracking domain, you can isolate yourself from the activities of our other clients. Create a CNAME record in your DNS that points to jngo.net (that's "jango" without the "a"). Then, go to Settings --> Tracking --> Tracking Domain and enter your tracking domain name. If your domain is mycompany.com, then setting up x.mycompany.com makes for the perfect tracking domain. A URL that looks more like you than us helps with your branding. See our blog article for more information.

    Who should do this? The technical person who manages your domain's DNS settings.
  3. Use a Domain Key based on the domain name you use in your FROM address. Yahoo and other email providers look for DKIM signatures in the headers of email messages. The presence of a DKIM signature fosters a sense of trust that the email was sent by who was purported to send the email. To set up a domain key in JangoMail, see our PDF entitled Setting up DomainKeys/DKIM with JangoMail. Note: Yahoo requires that you have a "postmaster" address at your domain.

    Who should do this? The technical person who manages your domain's DNS settings.
  4. After creating a DomainKeys/DKIM record, click the link in the confirmation email sent from Yahoo to postmaster@yourdomain. Whe you enable a new domain key in your accoutnt, we upload it to Yahoo so that they can process complaints properly and report them back to JangoMail via a "Feedback Loop." Yahoo has a special process where they send a confirmation email to postmaster@yourdomain, and that email message contains a link that must be clicked by you in order to activate the Feedback Loop. If you do not do this, complaints will not be properly reported, and your delivery to Yahoo email address may be negatively impacted by recipients that complain over and over and aren't added to your unsubscribe list.

    Who should do this? The person who has access to the postmaster@domain account for your domain. Usually this is your organization's system administrator.
  5. Add or edit the SPF record for your domain. An SPF record authorizes someone else (JangoMail, in this case) to send on behalf of your domain. This marks the "smtp.mail" header information with your domain, not ours, and messages sent to a gmail address, as an example, won't display "your_domain_name via jangomail.com." Instructions can be found in the PDF How to Publish SPF Records in JangoMail.

    Who should do this? The technical person who manages your domain's DNS settings.
  6. If you're sending HTML email, make sure you include a corresponding plain text message. Spam filters tend to add a point or so to messages that do not include a plain text component. This is as easy as setting the Plain Text Message to "auto-generate" (see the screenshot below.)  This will cause a plain text message to be generated (based on your HTML message) at the time of email sending. If the auto-generated message isn't good enough, you can also edit the plain text or simply create your own.

    Who should do this? The person using JangoMail.
  7. Enroll in the Sender Score Certified program. In order to be accepted, you must have a complaint rate of 0 to 0.1%, plus meet some other requirements. Once enrolled, your emails are sent from IP addresses that are certified. Hotmail virtually guarantees inbox placement for emails sent from certified IPs, and Yahoo prioritizes emails sent from certified IPs.

    Who should do this? The person using JangoMail.
  8. If using click-tracking, anchor text should be phrases, not URLs. Some spam filters look closely at how you use links to determine whether the link is legitimate or fraudulent. They do this to prevent phishing scams--a type of scam where an email pretends to be a request from a legitimate company in order to get the login credentials of that company's user, or take you to some other completely unrelated site. For more information on phishing, see the Wikipedia article on phishing. The best way to explain good links versus bad links is with an example:

    Good Link: <a href="http://www.browniekitchen.com/">Visit our web site.</a>

    Bad Link - the visible link is to browniekitchen, but the actual link is to somewhere else: <a href="http://www.steal_your_money.com/">http://www.browniekitchen.com</a>

    What makes the bad link bad is that the anchor text is a URL, and that domain in that URL does not match the domain in the link destination. Phishing filters look for this domain mismatch. In the good example, however, the anchor text is not a URL to begin with, so the phishing filter will accept it as legitimate.

    Who should do this? The person designing your email campaigns.
Further Reading
For more information on general JangoMail deliverability practices, see our Deliverability page on our web site.

Additionally, there are simple steps that any email marketer can take with the content of messages to ensure high deliverability. The content-related issues are covered in the PDF document Optimizing Email Deliverability.

Friday, November 06, 2009

How we scaled up the JangoSMTP service to accommodate the ING New York City Marathon

Overview of the New York City Marathon and its Email Alerts

It was a sunny mid-September day when my head of sales informed me that the systems integrator for the 2009 New York City Marathon was looking at our SMTP service for the delivery of alert emails on runners during the marathon. Given that our SMTP relay service was relatively new at the time, we saw this as an opportunity to demonstrate the powerful tracking features and performance capabilities of our service.

Having tried other email vendors and their own internal systems in the past, the marathon was looking to avoid past issues, such as large delays in the delivery of the email alerts and email blocking issues with consumer ISPs like GMail and Hotmail.

The marathon had about 42,000 total runners. Friends and family members of each runner could "subscribe" to a runner, such that whenever the runner reached a checkpoint during the marathon, an email alert would be sent to the subscribers of that runner. This would be done electronically via a Chronotrack D-Tag. For more information on the electronic tracking of runners, see http://www.nycmarathon.org/race_scoring.htm.

My team was told to expect anywhere from 400,000 to 2,000,000 email alerts to be sent out during the race. That represents an average of 10 to 50 email alerts per runner, depending on how many individuals subscribe to a particular runner.

The Speed Problem

The JangoSMTP service has been a single-node service since its launch earlier in 2009. It was serviced by one single fault-tolerant, RAID-based, multiple-CPU, high memory Windows server located at relay.jangosmtp.net. The SMTP service works by receiving email at relay.jangosmtp.net, and then passing emails along to one of JangoMail's 40 outbound SMTP senders for delivery to the final recipient. The relay.jangosmtp.net server could receive an unlimited amount of emails and saturate our upstream Internet provider's bandwidth, but it could only process, add tracking, and transmit to the outbound SMTP senders over the Internet at a rate of 250 emails/minute. I therefore calculated:

250 emails/minute x 60 = 15,000 emails/hour

Based on past marathon results, I estimated that the fastest runners would complete the race in about 2 hours, and the slowest runners would complete the race in about 6 hours. However, the marathon would be initiated in 3 waves of 14,000 runners each, distributed over an hour the morning of the marathon. So 6 hours of running time, plus an added hour for the start of the last wave, meant 7 hours of sending email.

7 hours x 15,000 emails/hour = 105,000 emails over 7 hours

Uh oh. We weren't nearly fast enough. Even at the minimum expected volume, we weren't fast enough by a factor of 4. And at the maximum expected volume, we only had 5% of the needed capacity.

The two bottlenecks were 1) the processing of an email message, meaning the dis-assembly and re-assembly to determine what user it belonged to and add tracking mechanisms, and 2) transmitting the email to a SMTP sender. Point #2 warrants more explanation. While there is no bottleneck for relay.jangosmtp.net to receive emails from the outside world, there can be a bottleneck for relay.jangosmtp.net to transmit emails, since relay.jangosmtp.net must transmit the message to separate SMTP sender located on separate networks in separate data centers. This transmission happens over the Internet, and based on where the SMTP sender is located and the routing to it, speed can fluctuate.

Time to Scale Up

Load Balancing

We needed to scale up, and scale up fast. The initial plan was to order four more servers, and have them each serve as an additional SMTP receiver and processor for relay.jangosmtp.net. The JangoMail architecture does not employ the use of appliance-based load balancers, so I decided to handle the load balancing via our Domain Name System (DNS).

We created multiple DNS "A" records for relay.jangosmtp.net, each with a Time to Live (TTL) of 60 seconds. Five "A" records were created in total, each with a different IP address. The first of the five was the original IP for relay.jangosmtp.net, and the other four were for the four additional servers we commissioned.

By keeping the TTL at a short 60 seconds, I could make certain that each of the five servers would receive an equal load of email every minute. And since not all DNS servers on the net respect TTLs and sometimes do their own caching, I confirmed with our tech contact at the marathon that their systems would NOT cache the IPs for relay.jangosmtp.net beyond the designated 60 second TTLs.

SQL Query Caching

Every email that arrives at relay.jangosmtp.net is disassembled, tracked, re-assembled, and then passed to an outbound SMTP server for actual sending. In order to determine what email belongs to what user, and what tracking options each user has selected, the originating IP address of each email message is looked up against an IP Address table, and then once the user is determined from the IP address, the UserAccounts table in the core database is queried to determine what tracking/DomainKeys options the user has selected. These two queries combined took anywhere from 0.05 seconds to 0.2 seconds, depending on the load on the database at the time.

We shaved this time down to 0.001 seconds by caching the results of these queries and refreshing every five minutes.


Our custom SMTP architecture is a multi-threaded model, allowing for the simultaneous processing and delivery of emails across user accounts. We had been informed ahead of time that the marathon would trigger email alerts from two originating IP addresses. We therefore configured two dedicated processing threads on each of the 8 servers. This gave us 16 total processing threads, and also isolated the processing/delivery of the marathon's emails from our other clients' emails during the race.

Distributed Transactional Senders

The SMTP service is part of JangoMail's transactional email platform, which also includes the SendTransactionalEmail API method. All transactional emails are sent through the email sender that is assigned for a particular user. For fault-tolerance purposes, every user has a list of transactional email senders assigned to it, such that if the first email sender is unavailable or offline, the email is passed to the second, and to the third, and so on, until the email is successfully transmitted. This approach was great for fault-tolerance and redundancy, but not for scaleability. If the first server in the list was online and available, then it would receive all the transactional emails for that account.

I therefore decided to add an internally controlled user-level setting option to randomize the list of senders. Now, if a user's list of transactional senders included:

Sender1, Sender2, Sender3, Sender4

Now the relay.jangosmtp.net servers would farm out the emails for delivery to any sender in the user's list at random, ensuring that as many transactional senders as were assigned would receive an equal load of emails to deliver, rather than having the first available sender do all the delivery. This also aided in resolving the bandwidth bottleneck with the SMTP senders mentioned earlier.

Still not fast enough - need 3 more servers

Given our SQL query optimizations, multi-threading, additional servers, the timing now looked like:

400 emails/minute/server x 5 servers = 2,000 emails/minute x 60 minutes = 120,000 emails/hour.

120,000 emails/hour x 7 hours = 840,000 emails over 7 hours

However, if the load was over 1 million emails, there would still be a delay. I decided to annex 3 additional servers that are already a part of the JangoMail network but weren't active on port 25, and turn them into 3 additional SMTP receivers. There were now 8 total DNS "A" records for relay.jangosmtp.net.

With just 48 hours before the marathon, we were informed that the expected outbound volume, based on the number of subscribers so far, would be about 750,000 emails.

And just in case all else failed...

While we've always believed in the performance and reliability of our code and architecture, we decided that for a project of this caliber, a backup plan was necessary in case our custom SMTP architecture was unable to perform. The JangoSMTP custom architecture is what allows emails that pass through the SMTP relay to be open and click-tracked, and stored in a database, so that SMTP logs can be viewed and reports can be generated based on open and click timing, domains, and geo-tracking reports based on IP addresses. However, the primary issue for the marathon, was to ensure the emails were delivered, and delivered on time. Therefore, a backup plan was put into place that could guarantee the emails would be delivered, even if we had to eliminate the tracking and logging based on our own custom architecture. If we found that JangoSMTP could not handle the load, we would replace the instances of the JangoSMTP receiver with Microsoft's built-in SMTP service (part of Internet Information Services), such that emails would be received and delivered to the final recipient, without any processing needed in between.

In order to isolate the marathon from our other clients in case this backup plan had to be put in place, we setup the domain marathon.jangosmtp.net, which mimiced the 8 A records for relay.jangosmtp.net, and we asked the system integrators for the marathon to connect to marathon.jangosmtp.net instead of relay.jangosmtp.net. This would allow us to re-direct just their email on the day of the race if needed.

The Day of the Race - November 1, 2009

I awoke at 7:30 AM EST on that Sunday, after having been out the night prior for Halloween. The emails would begin trickling in at 8:30 AM EST for runner check-ins, and the first wave of the race was set to begin at 9:40 AM EST.

I was watching the race live on TV, while at the same time monitoring the traffic flows across our 8 instances of relay.jangosmtp.net. Emails were being received, processed, and sent quickly, and there was no backlog...until about 12:25 PM EST.

All three waves had been released, and runners from all three waves were triggering a massive volume of email alerts. From 12:25 PM EST to about 12:35 PM EST, there was a 4-5 minute delay with final delivery. Thankfully, the backlog period only lasted 10 minutes. It makes sense that approximately 3 hours after the release of the first wave, that the highest volume of email was passing through, since the greatest number of runners would still be running around this time.

Additionally, at about 1:00 PM EST, we discovered that comcast.net and att.net/bellsouth.net domains were blocking one of the IPs from which marathon email was sending. While we did have a mechanism by which domain-specific routes could be used, such that we could enable all comcast.net/att.net/bellsouth.net email to go through one specific non-blocked IP address, the complexity of the randomization system we had added to accomodate the bandwidth bottleneck rendered this mechanism non-functional. I called our lead developer, who was on call, asked him to make a change to the sender-determination algorithm, and re-deployed our code across all 8 SMTP server instances. We were now able to route all comcast.net/att.net/bellsouth.net email through a separate non-blocked SMTP sender. In the end, the marathon alert emails had less than a 0.3% blocking rate.

After 1:00 PM EST, no further email deliverability or backlog issues ensued.


We were thrilled that we were able to pull this off for the 2009 ING New York City Marathon. Even post-marathon, we continue to make performance and feature enhancements to our transactional email platform. If you have an important project for which sending email is critical, please get in touch with us, and we'll work as hard for you as we did for the marathon.

New API Method for Template-Driven Transactional Emails


Marketers can now create and edit Transactional Email Templates on the Send Email Page and without the help of their IT team.

To enable this, we launched a new API method: SendTransactionalEmailFromTemplate. Instead of passing the full details of the email message using the regular SendTransactionalEmail method, you can now simply pass the email Campaign ID and let your marketers use the JangoMail interface to dictate the email's creative content.

SendTransactionalEmailFromTemplate works like the SendTransactionalEmail method, but does not require you to enter FromEmail, FromName, Subject, MessagePlain, and MessageHTML. Instead you must enter CampaignID, PersonalizationValues, PersonalizationColDelimiter and PersonalizationFields.

Input Parameters

Your JangoMail account username

Your JangoMail account password

Marketers should create an Email Campaign as they normally would, starting on the Send Email Page (or just use the SendMassEmail API method). Once they create a campaign and send a test, you can find the Campaign ID in the Reporting and Analytics section of the JangoMail interface. Look in the ID column to find the Campaign ID.

The single email address to whom you want to send this email message.


JangoMail uses the following syntax for personalization and Marketers should continue to use this when creating transactional email templates. Mail-merge personalization tags can be used in the Subject, HTML Message, Plain Text Message, From Display Name, and From Email Address:

%%FieldName**Default Value%

The second version of the personalization syntax can be used in cases where the value of FieldName is blank, such that a default value is substituted instead of a blank. 


Specify a column delimiter as follows:
c for a comma
s for a space
n for a newline, or a hard break
t for a tab

A comma separated list of name/value pairs.  The names of the various options and their associated values are explained below.
You may specify:
TransactionalGroupID (a numeric ID of a previously created Transactional Group)
SkipUnsubCheck ("True" or "False")

You can also specify the following, but if not specified, values will pull from your original email campaign. User-specified values here will override values in your email campaign:

ReplyTo (any valid email address)
CC (any valid email address)
BCC (any valid email address)
CharacterSet (a character set like "ISO-8859-1" or "US-ASCII")
Encoding (either "7 bit" or "Quoted Printable" or "Base 64")
Priority (either "Low", "Medium", or "High") 
UseSystemMAILFROM ("True" or "False") 
Receipt ("True" or "False")
Wrapping (a numeric value from 0 to 100, with 0 meaning no wrapping takes place)
ClickTrack ("True" or "False") 
OpenTrack ("True" or "False")
NoClickTrackText ("True" or "False") 
Attachment1 (the name of a previously FTPd file, like "Invoice.pdf")
(the name of a previously FTPd file, like "Invoice.pdf")
Attachment3 (the name of a previously FTPd file, like "Invoice.pdf")
Attachment4 (the name of a previously FTPd file, like "Invoice.pdf")
Attachment5 (the name of a previously FTPd file, like "Invoice.pdf")


In the below example, the input parameters are:


This example assumes that the Subject and/or Body contains personalization placeholders for FirstName and OrderNumber.  In fact, for Campaign ID 265987456, the actual message is:

------BEGIN MESSAGE-------
Dear %%FirstName**Valued Customer%%,

Your order has shipped.  Your order number is %%OrderNumber%%.
------END MESSAGE-------

This transactional email will be open tracked.  To ensure that it will also be click tracked, we would add "ClickTrack=True" to the Options parameter.  If we wanted it to be a high priority message instead of the default medium priority, we could add "Priority=High", such that the full Options string would look like:


To test the operation using the HTTP POST protocol, visit the JangoMail API page on this method and click the Invoke button.

Frequently Asked Questions

Q. What's the difference between SendTransactionalEmailFromTemplate and SendTransactionalEmail?
A. When calling SendTransactionalEmail, you are passing in the full details of that email message, including the Subject, the HTML Message, the Plain Text Message, tracking options and much more.  However, if you've already created a template message in the Send Email section of the JangoMail web interface, you can call SendTransactionalEmailFromTemplate to send transactional emails and avoid having to pass in all the message details on every method call.  You simply reference the Campaign ID of the email message that you created in the Send Email section.

Q. I see there's a PersonalizationColDelimiter input paramter.  Where is the PersonalizationRowDelimiter input parameter?
A. Since this is a method to send a single transactional email, only one set of values can be provided for the single recipient of the email message.  There is no need for a row delimiter since the data should only be for one record.

Q. What is the TransactionalGroupID?
A. This is an optional setting that can be specified as part of the Options input parameter.  Transactional Groups are a way to categorize different types of transactional emails, like Order Confirmations, Shipping Notifications, and Thank You messages.  You can create different Transactional Groups using the web interface or by calling AddTransactionalGroup.  Every Group has a numeric ID assigned to it, and so to assign a particular transactional email to a Group, you can set the TransactionalGroupID.

Q. How do I send attachments with my transactional email?
A. You can specify the names of attachments in the Options input parameter.  However, the file names you specify must already be present in your account.  You should FTP your files to client.jangomail.com/Attachments prior to calling this method with attachments.