Our Blog

Improving Mail Deliverability

March 12th, 2014

closeThis post was published 5 years 4 months 7 days ago which means the content may no longer be applicable or relevant to the service we offer today. If in doubt, please contact us.

When looking for and choosing web hosting, one of the most important services clients often rarely discuss or ask about is e-mail. These days it’s become a package deal – get web hosting and your e-mail service is thrown in as part of the purchase. Usually we don’t think any more of it than that, and in most cases e-mail is almost considered second place to the website hosting itself. As a customer, we just expect it to work and deal with any annoyances as they may surface, such as things incorrectly getting flagged as spam or e-mails being returned. At ThisWebHost we believe we can make our e-mail better.

In a shared hosting environment and when e-mail is vital to your business, the key concern is making sure your e-mails get to their intended recipient(s). The problem with this in a shared hosting environment is that there could be hundreds or even thousands of users all sending out e-mail every day from a single server. Most of the time these e-mails are legitimate e-mails, but sometimes an attacker may have compromised an e-mail password or a hosting account and then begins to send out UBE (Unsolicited Bulk E-mail), otherwise known as “spam”. Once this happens, there is a possibility that your own legitimate e-mails are affected because of the reputation of the server you are all sharing. Some recipients servers may determine that they have received spam from the server you are on, and therefore are going to block any further e-mails from being received until the situation is resolved. This is typically known as being blacklisted. Other servers may opt for a different approach and decrease the reputation of the server you are on, instead flagging your legitimate e-mails as spam and moving them to an appropriate spam or junk folder. Whilst not as serious as being blacklisted, this can naturally have significant repercussions for your business.

When a server is blacklisted, typically it is possible to contact the list provider and request that the server be de-listed once the source of spam has been identified and the issue resolved. Unfortunately in cases where very small amounts of spam have been sent across multiple accounts, it can be very difficult to find the source of spam and subsequently becomes a mammoth task become de-listed. To make matters worse, some blacklist providers do not even provide the ability to request removal. Instead of they operate on a reputation basis where they will blacklist or de-list a server based on the number of spam messages they detect. The problem with this system is of course it could take days or even weeks for their reputation systems to reflect any changes.

At ThisWebHost we monitor outbound e-mail activity from our servers very carefully. We have limitations in place that prevent the number of e-mails that can be sent from an account on an hourly basis as well as systems that will notify and alert us if specific outbound activity is detected. This is fairly common-place for hosting providers, and usually it works very well. Unfortunately, attackers are becoming aware of such restrictions and are implementing their own workarounds in order to get around these and continue to send spam. Instead of sending out large amounts of mail very quickly in order to try and reach a significant number of people as quickly as possible, they are now opting to send out very low amounts of e-mail over time but over many accounts so as to not raise suspicion. This makes it very difficult to find the spam e-mails amongst normal activity. Sending a few spam e-mails now and then may not seem very significant, but over time this can decrease the reputation of the server and cause deliverability issues for legitimate clients. At this point we begin to see the limitations of the systems we have in place and need to look at different systems – systems dedicated to the task(s) of delivering e-mail and ensuring reputation. Welcome to the world of SmartHosts.

A Smart Host is, very simply, an intermediary network we forward all of our e-mail through. Traditionally an e-mail that is generated on one of our servers would go straight to the recipients server and would either be accepted or rejected. With a Smart Host, the e-mail goes from our servers to the Smart Host and then out to the recipients server for delivery. This may seem like an unnecessary step but by using a Smart Host, and specifically a Smart Host that can scan, learn and subsequently “filter” e-mails, we can reduce the number of spam messages that are being sent from our network because they can be stopped and rejected at the Smart Host before making it to the recipient. Why can’t we do this directly on our servers? Well, we actually do and indeed try to block as many outgoing spam messages as possible – the problem is that any learning mechanisms we implement have a much lower source of e-mail to learn from than say, a Smart Host that is being used by thousands of other people worldwide. This makes the implementations on a Smart Host far more effective than ours could ever be.¬†More importantly, the main advantage and benefit of using a Smart Host is that Smart Hosts typically have a network dedicated to delivering e-mail. Unlike a shared hosting account on a single server, which is easy to blacklist and obtain a negative reputation, having a network of multiple machines sending out e-mail will (or at least as is advertised) guarantee you will never become blacklisted again. If a machine of theirs obtains a negative reputation it can simply shut it down and bring up a new one with a new address that is not blacklisted, whilst they resolve the blacklisting (this is how we speculate it happens anyway). The result? E-mail that should always be deliverable and never rejected and bounced back to you.

What does this all mean for you as a ThisWebHost customer? We are now actively using and trialing a Smart Host system on our shared hosting servers. This should improve deliverability (less e-mails being incorrectly flagged as spam or junk) as well as completely prevent e-mails from being rejected due to blacklisting. Whilst this is an added cost for us, this service is free of charge to all existing and new clients because we feel this is a significant improvement to e-mail and thus a great benefit to being a ThisWebHost client.

This system has already been globally implemented at a server level, and customers should not need to perform any configuration or changes to anything in order to take advantage of this. However, because of the way Smart Hosts operate there is a small risk that some servers that implement SPF record checking may reject e-mail coming from us because of an invalid SPF record. We have taken steps to enable and update the SPF records of all accounts that we host to prevent this issue, however, those hosting their own DNS externally (and not using our nameservers) will need to create and update their own SPF records with their DNS provider to prevent these issues. We have created a KB article for clients to use to check their SPF records:

http://www.thiswebhost.com/clients/knowledgebase.php?action=displayarticle&id=553

We hope that you see the benefits to this change we have made, however if you are experiencing any issues or problems sending out e-mail we would like you to get in touch as soon as possible. To do this, please submit a support ticket from within our client area.

One Comment

  1. Doug

    March 13, 2014

    Some questions from Twitter. If you have any more questions please post a comment on this article and we can follow up:

    Q1. I can see how this benefits you – your servers don’t get blacklisted – but what’s their record on false positives on addresses?

    A: It has no real benefit for us. We’re not trying to avoid getting our servers blacklisted, but instead we are trying to increase mail deliverability for our clients. Being blacklisted is to be expected given the nature of shared hosting. We’re happy we’ve been able to keep this to an extreme minimum so far, however with the implementation of Smart Hosts this is something we can not only avoid completely in the future, but we can do so whilst filtering out more spam from our network – reducing the amount of spam in the world.

    We have not seen any false positives to date. As with any filter system, should a false positive occur and be found, we will be able to take immediate action and correct this.

    Q2. Yes, but if you’re blacklisted, I’ll know immediately (bounce message) – not sure this would happen via a third party?

    A: The implementation of the Smart Host effectively removes the possibility of blacklisting, so this should be a non-issue. Bouncebacks can still occur when using a Smart Host, so if there is a widespread issue you should be notified of this.

What do you think?