Email Deliverability March 2018: The Top-Performing Newsletter Tools

A lot of email experts argue that email deliverability is the responsibility of the sender. Which is partly true – there are plenty of things you can be doing to reduce the likelihood of your email getting marked as spam.

But tell that to the marketer who has spent months cleaning email lists, building immaculately-coded templates, crafting spam-free copy, and running countless tests – only to find that a good chunk of their emails still aren’t getting through!

When it comes to email deliverability, it’s clear that there are some things that are out of our control. At EmailToolTester, this became all the more evident when we started trying out different newsletter services a while back, and getting different results using the same emails and lists.

So after several months of testing ten different email services (and a few thousand emails sent!), we’re excited to announce that we’ve now identified some clear winners in the email deliverability stakes. And, of course, those who aren’t performing so well. This time around we again used GlockApps to carry out the tests.

What’s more, we’ve now added ‘Email deliverability’ as a criterion in our reviews, so each of our reviewed email tools has a star rating based on how well they performed in our tests.

So, without any further ado, let’s take a look at the results of our latest round of testing, where we calculated average deliverability rate based on five separate tests.

Email Deliverability: The Results of Our Latest Tests

PROVIDER	DELIVERABILITY RESULT %	OVERALL RATING*
ActiveCampaign	96
Constant Contact	93
Drip	89.8
MailerLite	89
ConvertKit	88
MailChimp	82.6
Aweber	79.8
GetResponse	78
Benchmark	75.8
SendinBlue	75.6

*Overall rating based on combined performance of this round + previous round of testing

 

Here are our main takeaways:

• There is a significant difference in deliverability rates between our top and lowest performers: We saw this in our previous round of testing last August, so it wasn’t a huge surprise. Still, considering we treated all the service providers equally in each test (setting up sender subdomains, authenticating records, and sending the same email to the same seed list at the same time), it’s clear that some providers are a lot stronger in deliverability than others.
• ActiveCampaign now have the highest deliverability rate: We were happy to see that ActiveCampaign – our #1 rated email service – have improved their overall deliverability. In our previous round of testing, their deliverability rate was 85.79%, placing them in the middle of the pack. But 96% seems more fitting for a service we rate so highly.
• Constant Contact and Drip continue to have good deliverability: Being able to count on your email service is key, and it’s great to see that both Constant Contact and Drip have consistently performed well, ending up in the top three in both rounds of testing.
• MailerLite’s deliverability has improved considerably: Last round, MailerLite averaged a disappointing 77.64% deliverability rate, placing them second-last in the rankings. Things are looking a lot better for them this time around, with deliverability going up to 89%. 
• ConvertKit, MailChimp, GetResponse and SendinBlue’s deliverability rates have dropped: For these services, we’ve seen a drop in performance of between 4 – 9% from our previous round. As it’s difficult to know whether this reflects a general trend or is simply the result of normal fluctuations, we’ve calculated our deliverability ratings (the last column in the table) based on the combined performance of the two rounds – and we’ll continue to keep doing so with future testing rounds.
• Benchmark continues to lag behind in terms of deliverability: Although there’s been an improvement since the last round (where average deliverability was a very poor 64%), Benchmark’s deliverability rate is still at the lower end of the scale, earning them our lowest rating of 2 stars.
• Average spam rates ranged between 2 – 24%: SendinBlue, Benchmark and Aweber had the highest percentage of emails marked as spam, and – surprise, surprise – were also in the bottom four in the deliverability rankings. The top three (ActiveCampaign, Constant Contact and Drip) had less than 6% of emails marked as spam.
• We couldn’t see any direct correlation between blacklists, sender scores and deliverability: All the tools, except for ActiveCampaign and Benchmark, had a small number of blacklisted servers. We didn’t see this have any effect on deliverability (in fact, Benchmark suffered poor deliverability despite having no blacklisted servers). Which isn’t to say a relationship doesn’t exist – just that our results couldn’t demonstrate one. The same applied to sender scores – even high sender scores resulted in low deliverability sometimes, and vice versa.

Now let’s take a closer look at the results to see how each tool performed in getting through to the main ISPs.

Deliverability by Internet Service Provider (ISP)

Although our tests were sent to 28 ISPs worldwide, the following table shows deliverability to the four main Internet Service Providers – Gmail, Yahoo, Microsoft (Hotmail and Outlook), and AOL. For clarity, we’ve broken up emails to Gmail’s Primary inbox and their Promotions tab into separate columns.

PROVIDER	GMAIL PRIMARY %	GMAIL PROMO %	YAHOO %	MICROSOFT %	AOL %
ActiveCampaign	100	0	100	100	100
GetResponse	100	0	97.5	20	100
MailerLite	100	0	100	100	100
MailChimp	0	100	100	24	100
SendinBlue	100	0	100	7	100
Benchmark	100	0	100	0	80
Aweber	100	0	100	18	100
Constant Contact	100	0	100	100	100
Drip	100	0	100	100	100
ConvertKit	100	0	100	69	100

*Microsoft rates recalculated in May 2019 to include weighting

Reinforcing the results from our previous testing round, we can see that:

• MailChimp emails are more likely to go to the Gmail Promotions tab: While all the other providers were able to get through fine to Gmail’s main inbox, MailChimp’s emails all ended up in the Promotions folder. If you’re sending through MailChimp, this could have a big impact on your open rates as this tab is checked less frequently.
• Benchmark have trouble getting delivered to Microsoft: This explains their poor deliverability overall. All their Hotmail and Outlook-bound emails were flagged as spam. SendinBlue and GetResponse also struggled to get through to Microsoft inboxes, and were either marked as spam or simply not delivered at all.

Conclusion

After two extensive rounds of testing that have yielded some fairly consistent results, we’re pretty confident that we can begin to draw some general conclusions around deliverability. It’s clear that some newsletter services are better at delivering emails than others, and specifically, at getting through to certain ISPs like Outlook.

There are also some tools (namely, MailChimp) that really struggle to be identified as senders of non-promotional email, which means that the performance of campaigns may take a bit of a hit.

We really do believe that deliverability should be an important factor to consider when evaluating different newsletter services, which is why we’re making this information available. That being said, results in the real world will inevitably vary. The best way to get your deliverability rates as high as possible is to follow best practice guidelines, and to trial different newsletter services to see which one gives you the best results.

Update: check out our latest deliverability test results here.