Story of the rock-solid wobbly email

Email is one of the important building block of almost every web application.  Given its ubiquity, it should be commodity and easy to do.  Truth, as we came to know in a rather tough way, is it is not so.

Till a few weeks back, we hosted  Bandhan.com on Amazon’s EC2.  Since, most of the IP addresses on EC2 are dynamic, they figure in many email blacklists. Users of Bandhan can opt to receive mails for matching profiles every week. We wanted to ensure our users didn’t miss out on our mails. So, we signed up with a third-party SMTP service from DNS Made Easy.  We sent (and continue to send) mails only to the opt-in list.   The volume of mail is less than 4 mails per user per month.  We provide a single click unsubscribe and that link is present in every email that we send. If you think what’s so great about it, check out the Oatmeal on email etiquette (scroll down to “unsubscribe”).  Things went quite smoothly, till the point when they didn’t.

I renewed subscription of DNS Made Easy at the end of September 2010 and we were set for another year.

It was morning of the  Diwali festival in early November, when I saw the mail from DNS Made Easy with subject “SMTP account terminated.”  The mail said our account has been marked as spam.  We were sending mails to non-existent email addresses. That caused Yahoo to block mail server of DNS Made Easy. They in-turn blocked our account.

That robbed fun out of my Diwali celebration, to put it mildly.

When we send mails via DNS Made Easy, we don’t receive any reports about result.  So, we had no idea we were sending mails to non-existent addresses.  We sent 2 messages to Yahoo via a form meant for bulk senders.  There was one reply to our requests, but sadly, the response was unrelated to the issue raised. We tried to reason with DNS Made Easy. They agreed that we are not spamming, but they politely refused to reinstate the service.

So, we setup postfix on our own. We setup SPF to give a signal that we are not spammers.  We monitored postfix log to get list of bouncing mail addresses. We now ensure that we never send a mail to that address again.

The problem is users give their email address incorrectly.  Here are the invalid domain names that people enter.

Yahoo  : yahoomail.com  ymail.com  yahoomail.co.in  yhoo.com  yahoo.co  yahoo.co.uk  yaho.com  yhaoo.com  yahoomail.in  tahoo.co.in  yohoo.com  yahoo.con  yahooco.in  yahoo.coin  yahoo.ca  ahoo.co.in  yyahoo.com  yqahoo.co.in  ymail.co  ylahoo.com  yhoo.om  yhhoo.co.in  yaqhoo.com  yaoo.com  yaoo.co.uk  yalhoo.in  yahooo.com  yahooo.co.in  yahoomail.co  yahoo.om  yahoo.o.in  yahoo.mail  yahoo.inr  yahoo.im  yahoo.i  yahoo.gom  yahoo.fr  yahoo.comh  yahoo.com.in  yahoo.com.au  yahoo.coi.in  yahoo.co.ln  yahoo.co.jn  yahoo.co.inn  yahoo.co.inm  yahoo.co.ic  yahoo.cin  yahoo.ci.in  yahoo.cfom  yahoo.c0.in  yahoo.ami  yahoio.co.in  yahhoo.in  yahh.com  yahaoo.in  yah00.com  yah00.co.in   YAHOO.GO.COM  YAHOO.COM.IN

Gmail: gamil.com  gmil.com  gmai.com  google.com  gmal.com  gmail.co.in  gmail.om  gmail.con  googal.com  gnail.com  gml.com  gmaj.com  gmailc.om  gmail.ocm  gmail.cpm  gmail.corr  gmail.comp  gmail.comm  gmail.co.m  gmail.cim  gmail.cdm  gimel.com  ggmail.com  gane.com  ganail.com  gamial.com

Rediffmail: ridiffmail.com  rediff.com  radiffmail.com  riddifmail.com  reidfmail.com  reediff.com  rediffmail.vcom  rediffmail.in  rediffmail.cpm  rediffmail.com.in  rediff.cm  redif.com  redffmail.com  redeiffmail.com  redeffmaill.com  delhirediffmail.com  Redimail.com  RIDIFFMAIL.COM

There are many more but Yahoo, Gmail and Rediffmail are the most popular ones. Plus there are cases where users have entered incorrect username (this probably includes deactivated accounts.)

In a country like India, where English is not the first language (in many cases, it is not even second),  email is quite brittle.  While it is extremely simple and powerful, it is not as simple as a phone number.  Sir Tim Berners-Lee would be surprised to know that some users think, like websites, email addresses need to start with “www.”

Some unsolicited advice (and request) to Yahoo. Yahoo carried a re-branding exercise few months back where they ran TV commercials and bought the entire front page of largest English daily.  Next time, they should allocate a small part of the budget to create ads about email awareness.  Ideally, these educational ads should be run in regional language news papers.  It will make a far-reaching impact on non-digital natives.

Well, a long post which is part reflective, part rant and part learning, but mostly desire to put this on record so that I can come back later to amuse myself.

About these ads

4 Comments »

  1. Arun said

    >>Till a few weeks back, we hosted Bandhan.com on Amazon’s EC2. Since, most of the IP addresses on EC2 are dynamic, they figure in many email blacklists.

    Just wondering, how Amazon EC2 IP address are blacklisted. Are you using AWS Route 53 service for DNS service ?

  2. Bandhan Team said

    The dynamic IP addresses means you could very well be given an IP address that was previously used to spam. Given that anyone can signup with Amazon, start a instance and potentially spam, it is not unusual to see the block of IP addresses on EC2 figuring in the black lists.

    In recent times, Amazon has taken significant steps to curb such abuse. Now there is unspecified limit on number of emails you send from a instance and this restriction can be requested to be taken off if you inform them about a valid use case.

    AWS’ DNS service play no role in this.

  3. was going through the blog posts, ymail.com is a valid e-mail address account name. It’s provided by Yahoo, and I myself have an mail id that is of the form anubhavagarwal@ymail.com

  4. Bandhan Team said

    Anubhav,

    Thank you for pointing this out.

RSS feed for comments on this post · TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: