Avoiding the Junk Mail Folder

We’ve all experienced the inconvenience of finding an important message sent to us has ended up in the Junk Mail folder in our mailbox and the risk that this might have been removed before we were able to read it. Similarly, for those who run websites, particularly those who send out newsletters and accept orders from visitors, the threat of mail being rejected or junked is something we need to be constantly aware of.

The reputation of an IP address or domain needs to be protected, particularly in mission-critical or e-commerce scenarios, and the threat of being listed by one of the major blacklisting firms or a provider such as Hotmail or Google is a huge risk. There are a multitude of reasons your reputation can take a nose dive, such as a compromised script or a security breach resulting in spam being sent out. It is also necessary with a new IP address to establish a reputation as a responsible owner. In my own case, I’d found that given my IP address was a new one (due to me migrating from my old server a few months ago) some of the messages I was sending via contact forms, or to Manchester Film Co-op newsletter subscribers were unintentionally ending up in Junk Mail folders for Google mailboxes.

There are however a number of things you can do to avoid this: some technical, some non-technical. In the case of bulk mail, you can firstly ensure that whatever service you run complies with the laws on bulk mail. In my case, the newsletter system integrated into Manchester Film Co-op has a confirmation routine and includes an unsubscribe link on every message.

So what then, are the technical measures? I’ll look here at two of these in detail, reverse DNS and SPF. I can’t claim to be an expert in either, as these tips are mostly what I’ve learnt through my own experiences, so do feel free to post any corrections or further suggestions in the comments below.

rDNS

When you type a domain into your web browser, your machine looks up an “A” record for the domain, to discover the IP address of the website. The main purpose of DNS is to translate domain names into IP addresses. When you type “www.bbc.co.uk” your ISP’s DNS server should return an IP of 212.58.244.70. This “forward” A record allows you to reach the website. Reverse DNS works the way you might expect, it sets a host name for your IP address. But why is this important for email delivery?

A large number of mail servers will perform a reverse DNS check when receiving mail from an IP address to see whether the reverse DNS entry for the server matches the forward A record of the domain it is supposedly from. If these do not match, or if no rDNS record exists, senders are liable to have their mail rejected by some recipients.

So for example, if a mail server at 11.22.33.44 identifies itself as mail.yourdomain.com, the recipient mail server will then ask 11.22.33.44 for its rDNS record, sometimes called a PTR record. If the rDNS record of the IP is blank, is not set to mail.yourdomain.com (or is a default PTR such as ‘no-dns-11-22-33-44.myisp.uk.’) then mail is more likely to be rejected. rDNS is usually controlled by your hosting or broadband provider, depending on where you host your domain.

If your IP address is used to host a single website or mail server, you should set your rDNS to match the domain name of this site. In the case of this website, which runs on Bytemark’s BigV platform, we have a rDNS set of “bigv.jonathandavis.me.uk”.

A record: bigv.jonathandavis.me.uk = 213.138.101.210

rDNS record: 213.138.101.210 = bigv.jonathandavis.me.uk

As this server hosts several different sites, and you can only have one rDNS for an IP address, setting a valid domain with a matching forward A record is the best strategy to improve the server’s mail reputation. Lack of an rDNS record is a far bigger issue than the record not matching the sender domain.

Now we come to the next important measure I want to discuss: SPF.

SPF Records

SPF – or Sender Policy Framework – leverages the DNS system to provide additional verification that the server is question is allowed to send mail for your domain name. As is the case for many websites, you might be sending email from a variety of servers. For the most part, these servers will also be the ones on which you receive mail, and you would have MX records set-up (eg: “mail.yourdomain.com”) and this server would have an rDNS record to match, which would verify that this server is related to your domain and that your mail should be accepted as legitimate. However, this isn’t always the case, as most websites also have the potential to generate emails, such as from a contact form, or a forum sign-up page. As most mail and web servers are separated, this poses a problem, as we need to let providers such as Google and Hotmail know that they should be accepting mail from our domain name which is being generated from perhaps several different IP addresses.

SPF makes use of the DNS record type – TXT – to provide a list of server names or IP addresses from which we should expect our mail to originate from. TXT records can be used for a variety of purposes, but SPF is now one of the most common. Creating an SPF record – through your domain registrar – allows a mail provider such as Google to grab this information and compare the IP address the email has been sent from, to the addresses we have specified will be sending mail for our domain.

To use an real-life example, some of the messages I sent out to subscribers of the Manchester Film Co-op, using a database-driven mailing list script were being rejected, as there was nothing to explicitly state that the web server’s IP address will be sending mail from an address at manchesterfilmcoop.uk. So here I added the following SPF record:

manchesterfilmcoop.uk       TXT       “v=spf1 mx a ~all”

This configuration instructs an SPF-compliant server that any IP address listed in the A or MX records is also designated as being allowed to send mail for manchesterfilmcoop.uk. In this case, 213.138.101.210 (the IP my bulk mails are arriving from) is listed as an A record, so Google then has no qualms about accepting this as legitimate.

The syntax of SPF records can sometimes be quite tricky to assemble, so I can highly recommend the services of spfwizard.net, a free online SPF record builder. This is particularly important where adding multiple IP addresses, or a range of addresses from where recipients should expect your domain to send.

Summary

So to round up: whilst there are a huge number of things which can affect the reputation of your IP address and the likelihood of mail being accepted (so much so you could easily write several books on it) we can see that through a few simple measures you can avoid many of the potential reasons that senders find that their mails are never read by their intended recipient.  Since implementing SPF I have found that almost all of my mail is now arriving in inboxes and is dodging the dreaded Junk Mail folder entirely, which although is a useful improvement for me, it could be decisive for an e-commerce company struggling to reach their customers.