Archive for December, 2010

Story of the rock-solid wobbly email

Email is one of the important building block of almost every web application.  Given its ubiquity, it should be commodity and easy to do.  Truth, as we came to know in a rather tough way, is it is not so.

Till a few weeks back, we hosted on Amazon’s EC2.  Since, most of the IP addresses on EC2 are dynamic, they figure in many email blacklists. Users of Bandhan can opt to receive mails for matching profiles every week. We wanted to ensure our users didn’t miss out on our mails. So, we signed up with a third-party SMTP service from DNS Made Easy.  We sent (and continue to send) mails only to the opt-in list.   The volume of mail is less than 4 mails per user per month.  We provide a single click unsubscribe and that link is present in every email that we send. If you think what’s so great about it, check out the Oatmeal on email etiquette (scroll down to “unsubscribe”).  Things went quite smoothly, till the point when they didn’t.

I renewed subscription of DNS Made Easy at the end of September 2010 and we were set for another year.

It was morning of the  Diwali festival in early November, when I saw the mail from DNS Made Easy with subject “SMTP account terminated.”  The mail said our account has been marked as spam.  We were sending mails to non-existent email addresses. That caused Yahoo to block mail server of DNS Made Easy. They in-turn blocked our account.

That robbed fun out of my Diwali celebration, to put it mildly.

When we send mails via DNS Made Easy, we don’t receive any reports about result.  So, we had no idea we were sending mails to non-existent addresses.  We sent 2 messages to Yahoo via a form meant for bulk senders.  There was one reply to our requests, but sadly, the response was unrelated to the issue raised. We tried to reason with DNS Made Easy. They agreed that we are not spamming, but they politely refused to reinstate the service.

So, we setup postfix on our own. We setup SPF to give a signal that we are not spammers.  We monitored postfix log to get list of bouncing mail addresses. We now ensure that we never send a mail to that address again.

The problem is users give their email address incorrectly.  Here are the invalid domain names that people enter.

Yahoo  :  yahoo.con  yahoo.coin  yahoo.mail  yahoo.inr  yahoo.i  yahoo.gom  yahoo.comh  yahoo.cin  yahoo.cfom  yahoo.ami   YAHOO.GO.COM  YAHOO.COM.IN

Gmail:  gmail.con  gmail.ocm  gmail.cpm  gmail.corr  gmail.comp  gmail.comm  gmail.cim  gmail.cdm

Rediffmail:  rediffmail.vcom  rediffmail.cpm  RIDIFFMAIL.COM

There are many more but Yahoo, Gmail and Rediffmail are the most popular ones. Plus there are cases where users have entered incorrect username (this probably includes deactivated accounts.)

In a country like India, where English is not the first language (in many cases, it is not even second),  email is quite brittle.  While it is extremely simple and powerful, it is not as simple as a phone number.  Sir Tim Berners-Lee would be surprised to know that some users think, like websites, email addresses need to start with “www.”

Some unsolicited advice (and request) to Yahoo. Yahoo carried a re-branding exercise few months back where they ran TV commercials and bought the entire front page of largest English daily.  Next time, they should allocate a small part of the budget to create ads about email awareness.  Ideally, these educational ads should be run in regional language news papers.  It will make a far-reaching impact on non-digital natives.

Well, a long post which is part reflective, part rant and part learning, but mostly desire to put this on record so that I can come back later to amuse myself.

Comments (4)