Permissions & Deliverability

Email Deliverability July 2017 Is Your Marketing Getting Lost in the Mail?

Josep GarciaRobert Brandl

By Josep & Robert

Email Deliverability

EmailTooltester is supported by readers like yourself. We may earn a commission when you purchase through our links. Of course, this won't increase the cost for you.

Update: Please find our latest deliverability test results from Jan 2024 here. 

If you use Google Mail, you’ll notice it’s constantly getting better at filtering your emails. Firstly, it will send unsolicited messages to the spam folder. If they’re from a newsletter, they will land in your Promotions tab.

This is all part of a larger trend designed to completely remove spam from our digital lives. And it’s great for the end user. But as an email marketer, it also means some of your messages are more likely to be ignored.

To ensure you get the best results from your campaigns, you need to factor in something called email deliverability. Quite simply, it’s a percentage that represents how many of your emails will reach their audience.

And according to our tests, some of the results were pretty surprising indeed.

How we tested deliverability

We launched 5 test rounds with 10 newsletter tools to 28 ISPs worldwide, using the testing tool of Glockapps.com, which they provided as a courtesy. This was more than three thousand emails over two months!

Next to major players such as Gmail, Outlook, AOL and Yahoo there are also more exotic ISP’s among them. For example, Yandex from Russia, Locaweb from Brazil, and Laposte from France. They weigh the results according to the international popularity of the email service. That’s why a spam report for Gmail has more weight than, say, Godaddy Email.

From our side, we optimized everything we could. Every email marketing service got its own sender subdomain, which we equipped with both SPF and DKIM records (where available). Overall, we sent two different newsletter types (all text-based), some with links, others without.

AWeber Email Deliverability Glock Apps Report

AWeber sample report

Let’s now look at the results.

Email Deliverability: The Results According to Our Tests

Provider Deliverability result (%)
ConvertKit 92.10
Constant Contact 89.74
Drip 89.14
Mailchimp 87.52
GetResponse 87.50
Active Campaign 85.79
Brevo 83.86
AWeber 80.25
MailerLite 77.64
Benchmark 63.88

Here are the main things to note:

Our tests results varied quite a lot: for example, the worst Brevo deliverability test was 58.2%, (our last test), and the best one was 81.96% (3rd test). We also saw big changes for other providers such as GetResponse or Mailchimp. Therefore, the deliverability seems to vary a lot, which is why these results should be taken as guidelines only.

There is quite a wide range of results: The average seems to be that around 15% of your emails will only reach the spam folder of your email box. However, depending on your provider, this could be anything between 8 to 36% of all emails!

ConvertKit use external mail servers. While all the newsletter tools have their own servers, ConvertKit use Mailgun to deliver messages. Could it be why they performed so well?

Benchmark’s results were lower than we expected: as one of our top 5 solutions, we were a bit disappointed to see their deliverability performed so poorly.

All providers have a small percentage of blacklisted servers. Between 0.2 and 2.8% for each provider. It sounds bad, but we did not see any correlation between these numbers and deliverability.

Active Campaign had quite a large percentage (13.5%) of missing messages. We expected better results from them. We were also really shocked to see that Benchmark had almost 1/3 of their emails labeled as spam, which explains their bad deliverability results. MailerLite, AWeber and Brevo also had over 15% of their messages considered spam.

The results weighted for different ISPs

Mileage will vary depending on the Internet Service Provider (ISP). Because we tested our emails with 65 different ISPs, the results might not necessarily represent the reality in the field. This is because ISPs have different number of users and market shares around the world.

email providers popularity us

Source: Statista

In order to reflect this imbalance, we recalculated our results for the 4 main US providers, namely Gmail, Yahoo, Microsoft & AOL. So for each 1000 emails sent, 590 went to a Gmail address, 180 to Yahoo, 150 to Microsoft (Outlook and Hotmail) and 80 to AOL.

Tool Gmail Yahoo Microsoft AOL Deliverability Avg. (%)
ConvertKit 100 100 64.44 100 94.4
Brevo 100 100 37.7 94.28 90.14
Active Campaign 100 100 80 11.43 89.37
Mailchimp 100 100 20 100 87.95
Drip 100 100 20 100 87.95
Constant Contact 80 100 88.9 97.14 86.68
GetResponse 80 100 80 94.28 85.09
AWeber 100 100 0 100 84.94
MailerLite 100 100 0 97.14 84.7
Benchmark 80 60 20 0 60.2

These results could make a big change if you have a quick look at subscribers’ providers. Let’s say you use MailChimp and you are only sending to Gmail, Yahoo and AOL addresses. This could be a perfect 100% success rate (but see the promotion tabs section below)!

However, if you use AWeber or MailerLite and all your users are with Microsoft, this could be disastrous. Also note that Gmail, Yahoo and AOL seem to be the easiest inboxes to get into. Except for poor Benchmark.

The Gmail Promotion Tabs Effect

Gmail is one of the most used providers worldwide and as you may know, they categorize your emails in different folders. If you land in the promotions folder you, your open rates might suffer. On the other hand, landing into the primary inbox is a big plus for you.

Tool Main Inbox Promotions Tab
ConvertKit 100 0
Active Campaign 100 0
AWeber 100 0
MailerLite 100 0
Drip 100 0
GetResponse 80 0
Constant Contact 80 11.4
Benchmark 80 20
Brevo 77.1 22.9
Mailchimp 0 100

The main thing to note here is that Gmail seems to be onto Mailchimp. All their emails went directly to the Promotions Tab.

The good reputation of the email server is what counts

As the emails in our tests were pretty much identical, we can pinpoint the differences in delivery to the individual newsletter service.

For bulk emails, there is also a complex chain of trust in place. The newsletter tool has to trust your email address and domain name. Then, the Internet Service provider (ISP) has to trust the newsletter tool’s server.

If at any point recipients start complaining about the emails they receive, an ISP such as gmail.com might say: we’re considering this server to be spam-heavy. Bounce rate and user behaviours also affect these results (i.e. recipients placing the messages in the spam box themselves).

This is why newsletter tools have every interest to maintain their server’s reputation, and why you should make sure you choose one with solid email deliverability results. But you can also improve your chances by following these steps:

What can I do to improve deliverability?

While your newsletter service is working hard to keep their server reputation pristine, you can also make sure the odds are stacked in your favour by implementing the following practices:

  • Avoid spammy content: there are no precise guidelines here, but we all recognise spam when we see it. For example, an email that repeatedly offers “free” or “100% success” products in its subject line and message body does certainly feel a bit spammy.
  • Authenticate your domain: this is done through SPF (Sender Policy Framework) and DKIM (Domain Key Identified Standard) records. Your newsletter software should have information on how to do it, and essentially it proves your emails are sent from a legitimate source.
  • Keep the links in your email tidy: if you are linking to your website  (www.mywebsite.com), keep in mind that your newsletter software may change the link behind this URL to make it trackable (e.g. http://mywebsite.randomispdomain.com). As phishing attacks look very similar, email providers might flag your messages accordingly. A better approach: don’t put URLs in your email but put the link behind a word instead.
  • Create engaging content: your sender reputation improves when people open the emails, and even more so when they reply. Make sure the messages you send are relevant and that users actually want to read them.
  • Enable sign up opt-ins: this is just to make sure your user list is actually made of people who want to receive your messages. And of course, never purchase subscriber lists from shady sources or add users yourself without asking their permission first.
  • Avoid code mistakes: it’s a small detail, but for example, a line of HTML with a missing tag could impact you negatively.

Conclusion

So as you can see, email deliverability can have a huge impact on your email marketing campaign. Choose the wrong provider (sorry Benchmark) and you could expect about 40% of your messages never to reach the right inbox. But the results are looking really good for ConvertKit, Constant Contact, Drip, GetResponse and MailChimp. Interestingly, GetResponse was also heading for a top spot before the last (pretty poor) test round moved them down to position 5.

We know that five rounds of tests is not enough to make general statements about the deliverability of certain email marketing tools. The results seem to fluctuate quite a bit. That’s why we will repeat this test in a few months time. We are looking forward to this! Because then we will also add these test results to the individual reviews.

Having said that, you should also take these results with a pinch of salt, and look at the bigger picture. Price vs features, for instance could also be just as important for you. Or ease of use vs design options. Does it really matter if some emails go missing if you have to pay twice as much for the service? Of course we try and take all these parameters into account when we help you compare all the different newsletter tools.

 

Feedback, questions? Stay in touch via the comment section below.

The authors

Learn more about us

Josep Garcia

Content Manager

Josep believes a good digital marketing strategy isn’t complete without emails, which is why he's glad to help businesses and individuals through EmailToolTester. If you can’t find him testing newsletter tools, he's probably munching popcorn in front of Netflix.

Robert Brandl

Founder and CEO

Hello! I'm Robert, an email marketing expert with over 15 years of experience in the industry. After honing my skills by serving major corporations working for an agency, I founded EmailTooltester as a passion project to help small and medium-sized businesses. My goal is to assist you in maximizing email marketing's potential.

Learn more about us

Our Methodology

This article has been written and researched following our EmailTooltester methodology.

Our Methodology