New Video Tutorial: Configuring Gmail with the JangoMail SMTP Relay

We have a new Video Tutorial on how to configure Gmail with the JangoMail SMTP Relay Service:

Video: How to Configure Gmail with the JangoMail SMTP Relay

The JangoMail SMTP Relay improves your sending capabilities by providing the following features for your person-to-person emails:

• Open Tracking
• Click Tracking
• DomainKeys/DKIM Signing
• Easy web-based access to SMTP Logs
• API methods to retrieve Reporting data

For more information, screenshots, and written directions, read our post on configuring your Gmail account with the JangoMail SMTP Relay Service.

If you are not a Gmail user, watch our other Video Tutorials on Configuring the JangoMail SMTP Relay with a Desktop Client or a Web Server.

Data Center Move Part 2: The Planning

There were several components to my planning of the data center move:

1. Connectivity / IP Addresses - We'd be getting a whole new set of IP addresses, so I had to map the old IPs to the new IPs. And, in order for the domains to map to the new IPs as soon as we came live at the new data center, I set all TTLs for important domains, like jangomail.com, to 60 seconds.
2. Reverse DNS - Because we operate several email servers, I needed to ensure that rDNS would work properly on all of them post-move. So prior to the move, I contacted the support team at the New Data Center to put in rDNS entries for 3 of our IPs, that would map to mail.silicomm.com, mx.jngo.net, and mail.jangomail.com.
   
3. Human labor – I’d need to find enough people to move our equipment over in the desired amount of time.
4. Rails – we were moving from non-Dell cabinets to Dell cabinets. We needed to make sure the rails purchased for our non-Dell cabinets would work in the New Data Center’s Dell cabinets.
5. Firewall – All of our server access rules are controlled by our firewall, and we’d need to make sure the firewall would be ready, with the rules around the new IP addresses.
6. Software/OS configuration changes – We would need to ensure that all web servers, email servers, database servers that had services bound to a designated IP would have those IPs changed.

IP Addresses

Prior to the day of the move, I took inventory of all our equipment and the IPs to which they were assigned. I then made an Excel spreadsheet detailing the device, the old IPs assigned to it, and the new IPs that would be assigned to it.

Reverse DNS

This was relatively simple. Since we don't own our IP addresses, and since we only control forward DNS in our own DNS system, we would rely upon the New Data Center's tech team to setup rDNS for us on 3 of our IPs, each of which would be assigned to a different email server.

Human Labor

I enlisted a team of 6, including myself: 5 doing the actual move, and one man on the outside to test things and remotely make configuration changes to our firewall and DNS system. The goal was to be offline for no more than 4 hours. The plan was simply this:

1. Within the first hour, I’d take the firewall from the Old Data Center to the New Data Center to test connectivity to the Internet. A team of 2 would begin dismantling the most important servers, the primary web server, the primary database server, and the API server, but not load them into a vehicle until I had called from the New Data Center to verify connectivity was working. If for some reason, connectivity wasn’t working, then we could easily fall back to the Old Data Center.
2. Within the next two hours, the primary 2, would meet me in the New Data Center with the 3 most critical servers.
3. During the last hour, the secondary 2 would arrive at the New Data Center with all other remaining equipment.

Rails

Several weeks prior to the move, we brought rails from the Old Data Centerto the New Data Center to ensure they were compatible. They were not, but with a lug, washer, and screw, we made them work. We tested mounting a 1U server two weeks prior to the actual move.

Firewall

We only had a single firewall appliance running in the Old Data Center, and I contemplated purchasing a second appliance of the same model prior to the move. That would enable me to have the second firewall configured with the New Data Center’s IPs, and would save time in having to manually re-configure the Old Data Center’s firewall mid-move. But the price tag of the appliance convinced me otherwise, and I determined it would only take 10 minutes of time to swap out the old IPs on the firewall with the new IPs once I was able to connect my laptop to its LAN port in the New Data Center.

A few days prior to the move, we became aware of a major difference between our connectivity setup at the Old Data Center and the setup at the New Data Center. At the Old Data Center, we were simply given a range of IPs, the gateway to use, and DNS servers. All servers and devices, including the firewall, used this configuration.

The connectivity setup at the New Data Center was slightly more complicated, requiring a Layer 3 routing device. An IP would have to be assigned to the WAN port on the firewall, and that WAN port would use the facility’s IP as its gateway. Then, all the devices connected to our DMZ would use the IP that we assigned to our firewall as their own gateways. I was wary of this new setup, and therefore wanted to test connectivity during the move (see step 1 above), prior to reaching a point of no-return with the move. And because the firewall would now serve as the gateway for all of our servers on the DMZ, the firewall now became an essential piece of equipment to our uptime. At the Old Data Center, the firewall could've been removed from the network flow, and the servers would remain online. But that wasn't the case anymore.

Software Configuration Changes

The software services that runs JangoMail consist of web servers, FTP servers, SMTP servers, SQL Servers, and some monitoring systems. We have many instances of IIS running, and prior to the move, I documented which applications would need configuration changes because of the new IPs. Many of our IIS web/FTP sites are bound not to a specific IP but to “All Unassigned”, which benefitted us because it prevented us from having to manually change the IP that a particular site was bound to. But some services, like our corporate email server (IceWarp), has to have DNS servers specifically set by IP address. It doesn’t inherit the DNS settings from the NIC’s configuration. It was given that I’d be changing the IPs and Gateway and DNS servers associated with all NICs on all servers, but I wanted to minimize the amount of additional IP changes that would have to be done on top of NIC changes.

With my planning finished a mere 3 hours before the move, I was ready to go. Stay tuned for Part III, where I’ll detail what went right and what went wrong during the move, and whether we were able to complete the move within our announced downtime window!

Data Center Move Part 1: Why I did it

This past Saturday night, we did what I hope we only have to do every 5 or more years - we switched data centers. We had to physically move all of JangoMail's servers out of one data center and into another, while minimizing the impact to our customers. JangoMail was offline during the entire move, which makes it a stressful experience. The worst case scenario would be for JangoMail to go offline and never come back online. A lot would have to go wrong to realize the worst case scenario, but still, taking your life’s work offline for multiple consecutive hours while trying to move and reconfigure equipment as fast as possible in the wee hours of a Sunday morning is a large undertaking. I decided to write this article 1. As an account for myself of what we did right and wrong and 2. To help other sysadmins navigate the messy waters of switching data centers.

Why I moved data centers

Because this article isn’t meant to be an endorsement or complaint against any organization, I won’t mention the names of the old data center and the new data center. I’ll simply refer to them as Old Data Center and New Data Center.

I moved because I was unhappy with Old Data Center. Old Data Center has colocation facilities all over the country and if the cabinet space you need is under a certain size, you can’t buy from them directly – you must go through one of their reseller partners. We rent two full cabinets to house our equipment.

My issues with Old Data Center were:

1. Every 6 months for the last 2 years, they increased their prices.
2. I was unimpressed with the onsite staff at Old Data Center. Whenever I needed to see them in person for a simple issue of getting a new access card made for a new employee, I’d show up at the assigned time and ALWAYS have to wait for the right person to be available. They weren’t dressed well, nor were they articulate.
3. 18 months ago, when we expanded from 1 to 2 cabinets, as part of the setup for the second cabinet, a data center employee physically cut the Ethernet cable connecting our main cabinet #1 to the WAN. We were down for 7 hours, and no explanation was provided by Old Data Center as to why that employee thought cutting that cable was a good idea. I requested a credit, and 12 days later Old Data Center responded that we would get a credit of $46.38. I protested, citing the damage that had been done to our business, and the credit was increased to $324.66. Our monthly fee for the one cabinet was approximately $1,200 and it was my opinion that for such a boneheaded mistake, one full month’s credit should’ve been issued.
4. Setup fees were, in my opinion, too high. When we needed to increase the Amps for our power circuit, along with paying for the increased Amps, a $500 setup fee was incurred. When we wanted to move from regular cabinet doors to mesh cabinet doors for better heat dispersion, we were quoted $750 per cabinet.
5. On the few times that there were connectivity issues with Old Data Center, we would call the support number of the reseller reporting the outage, and then they would contact Old Data Center on our behalf to get an explanation. I didn’t like the multiple layers involved.
6. After four years of having been a customer of reseller of Old Data Center, and never being late on an invoice payment, they suddenly informed us that we owe them a security deposit of $2,315. The reason? Apparently Old Data Center was suddenly charging reseller a security deposit. Again, I wasn’t pleased with this multi-vendor relationship.
7. I didn’t feel like a valued customer.

The reasons I picked New Data Center were:

1. It was a single, independent company that owned, managed, and supported the datacenter.
2. The monthly fee was less than that of Old Data Center, and already included mesh doors.
3. New Data Center also provided Internet-connected PDUs in each of our two cabinets.
4. When I toured New Data Center, all employees were dressed professionally, courteous, gave me their business cards, explained their positions, and offered help above and beyond what I expected.

Stay tuned for Part II, in which I’ll detail the planning of the data center move, and Part III, where I’ll detail the actual move operation.

Behind the scenes of our SMTP Service

The last couple of days have been a challenging and learning experience for the team in charge of our SMTP Relay service. As the volume of email that passes through the relay grows, so do the problems associated with performance and scaleability.

The first version of the relay service operated in a single-threaded linear fashion. Meaning, all emails sent to relay.jangosmtp.net, are processed and sent one at a time. Whenever an email message arrives at relay.jangosmtp.net, the following steps are executed:

1. The originating IP address and From Address are examined to determine what JangoMail user account the email message belongs to.

2. The tracking options for that particular user are loaded.

3. The email message is disassembled, the tracking mechanisms are added, and then the email message is re-assembled.

4. The email message is transmitted to the appropriate JangoMail sending server, based on whether the user account is enrolled in the Sender Score Certified program or based on the domain of the recipient email address.

5. The sending email server receives the message, signs the message with DomainKeys and DKIM, and then transmits the email message to the final destination email server based on the recipient domain.

A traditional SMTP service is much simpler and only need incorporate step 1 and a part of step 5. It can take our processes anywhere from 1/10th of a second to 3 seconds to process and deliver a single email message. The amount of time depends on the size of the email, the encoding of the email, and the tracking options selected.

Tuesday morning, a larger client submitted 50,000 transactional emails to relay.jangosmtp.net. The emails arrived at the relay.jangosmtp.net server over the course of 5 hours. Our linear process went to work, processing and transmitting each email message individually. As a result, email messages from other, smaller customers were delayed, some for serveral hours. We were alerted to the problem very soon after the backlog of emails began to build. Since all JangoMail staff members use the JangoMail SMTP relay with their desktop email clients, it was easy for them to tell something was wrong -- emails my staff was sending to customers, prospects, and themselves weren't being received. We realized that having a single process, processing each email one by one, just isn't going to cut it for scenarios where a customer transmits a large amount of email messages at once.

Within 8 hours, we re-architected the entire process so that multiple threads could operate on groups of email messages simultaneously. We now have the ability to assign a dedicated processing thread to any large customer. Since all threads operate simultaneously, a large amount of email from one customer will now not hold up a small amount of email from another customer.

After we deployed the new code last night, we thought our problems were solved. We awoke Wednesday morning to discover that the multi-threaded process was crashing, due to I/O concurrency issues. The constant crashing of the new code resulted in another backlog of undelivered email messages. Within 10 minutes, we discovered the issue was that all threads were attempting to log their activities to the same log file, and this was resulting in I/O disk errors. Within 60 minutes, we corrected the issue and deployed the new code, and since then (1:15 PM EST Wednesday), there have been no errors and no backlogs.

We appreciate our SMTP service customers' support over the last couple days, especially those that were adversely affected by the backlog of emails. One of the factors that I think makes JangoMail a different type of company is the awesome skill and dedication of our developers. When an architectural or functional issue is discovered, our programmers work until the problem is solved. We react swiftly, coming up with creative solutions to difficult problems within minutes -- because if we don't, the problems get worse: backlogs build, emails stop sending, the system could grind to a halt. The types of development problems we face are generally not the type that can be solved with a Google search. This is because much of JangoMail's technology is unique. We've done things that very few companies in our industry have done, including writing our own Email Rendering Tool, writing our own SMTP sending engine from scratch, and now, introducing the world's first SMTP service with tracking.

I hope this post sheds some light on the inner workings of the JangoMail operation, my staff's committment to delivering an amazing service, and a little bit of our secret sauce on how the magic happens. If anyone has any questions about the SMTP service, email me at ajay AT us dot jangomail dot com.

You Can Now Use JangoMail's SMTP Relay with Gmail

Gmail allows users to use a custom From Address to send emails, but some recipients would see both the From Address and the Gmail Sender Address. Messages would appear to recipients as "From username@gmail.com On Behalf Of username@customdomain.com". Gmail announced last week that users can now send with their custom domain through their company's email servers, and that will eliminate the "On Behalf of" message.

This means that you can now use the JangoMail SMTP Service with Gmail to eliminate the "On Behalf of" message and give you the added benefits of Open Tracking, Click Tracking, SMTP logging, and participation in the Sender Score Certified Program.

Configuring Your JangoMail Account

Before configuring Gmail, set up an Authentication method in your JangoMail account.

1. Go to My Options and Settings > SMTP Relay.
2. Choose Authenticate by From Address, click From Addresses and enter the From Address from which you’ll be sending email. You may enter multiple From Addresses.

Configuring Your Gmail Account

Set up Gmail to use the JangoMail SMTP Relay.

1. Sign into your Gmail account.
2. Go to Settings in the upper right corner.
3. Click on the Accounts tab.
4. In the Send mail as: section, click edit info next to the account that you want to change. You should already have your business email address set up to send through Gmail. If you don't, click the Add Another Email Address option and follow the steps provided.
5. Hit the Next Step button.
6. Choose the Send through [your domain] SMTP servers option.
7. For SMTP Server, erase the default address and type in relay.jangosmtp.net
8. For Port, choose 25.
9. Type in your JangoMail Username and Password.
8. Save Changes
Gmail will now send your emails via the JangoMail SMTP Relay instead of its own servers. Enjoy the Open Tracking, Click Tracking, and all the other benefits of our SMTP service.

To view Gmail's announcement of this new feature, visit:
http://gmailblog.blogspot.com/2009/07/send-mail-from-another-address-without.html