Update: Please find our latest deliverability test results from Jan 2023 here.
If you use Google Mail, you’ll notice it’s constantly getting better at filtering your emails. Firstly, it will send unsolicited messages to the spam folder. If they’re from a newsletter, they will land in your Promotions tab.
This is all part of a larger trend designed to completely remove spam from our digital lives. And it’s great for the end user. But as an email marketer, it also means some of your messages are more likely to be ignored.
To ensure you get the best results from your campaigns, you need to factor in something called email deliverability. Quite simply, it’s a percentage that represents how many of your emails will reach their audience.
And according to our tests, some of the results were pretty surprising indeed.
We launched 5 test rounds with 10 newsletter tools to 28 ISPs worldwide, using the testing tool of Glockapps.com, which they provided as a courtesy. This was more than three thousand emails over two months!
Next to major players such as Gmail, Outlook, AOL and Yahoo there are also more exotic ISP’s among them. For example, Yandex from Russia, Locaweb from Brazil, and Laposte from France. They weigh the results according to the international popularity of the email service. That’s why a spam report for Gmail has more weight than, say, Godaddy Email.
From our side, we optimized everything we could. Every email marketing service got its own sender subdomain, which we equipped with both SPF and DKIM records (where available). Overall, we sent two different newsletter types (all text-based), some with links, others without.
Let’s now look at the results.
|Provider||Deliverability result (%)|
Here are the main things to note:
Our tests results varied quite a lot: for example, the worst Sendinblue deliverability test was 58.2%, (our last test), and the best one was 81.96% (3rd test). We also saw big changes for other providers such as GetResponse or Mailchimp. Therefore, the deliverability seems to vary a lot, which is why these results should be taken as guidelines only.
There is quite a wide range of results: The average seems to be that around 15% of your emails will only reach the spam folder of your email box. However, depending on your provider, this could be anything between 8 to 36% of all emails!
ConvertKit use external mail servers. While all the newsletter tools have their own servers, ConvertKit use Mailgun to deliver messages. Could it be why they performed so well?
Benchmark’s results were lower than we expected: as one of our top 5 solutions, we were a bit disappointed to see their deliverability performed so poorly.
All providers have a small percentage of blacklisted servers. Between 0.2 and 2.8% for each provider. It sounds bad, but we did not see any correlation between these numbers and deliverability.
Active Campaign had quite a large percentage (13.5%) of missing messages. We expected better results from them. We were also really shocked to see that Benchmark had almost 1/3 of their emails labeled as spam, which explains their bad deliverability results. MailerLite, AWeber and Sendinblue also had over 15% of their messages considered spam.
Mileage will vary depending on the Internet Service Provider (ISP). Because we tested our emails with 65 different ISPs, the results might not necessarily represent the reality in the field. This is because ISPs have different number of users and market shares around the world.
In order to reflect this imbalance, we recalculated our results for the 4 main US providers, namely Gmail, Yahoo, Microsoft & AOL. So for each 1000 emails sent, 590 went to a Gmail address, 180 to Yahoo, 150 to Microsoft (Outlook and Hotmail) and 80 to AOL.
|Tool||Gmail||Yahoo||Microsoft||AOL||Deliverability Avg. (%)|
These results could make a big change if you have a quick look at subscribers’ providers. Let’s say you use MailChimp and you are only sending to Gmail, Yahoo and AOL addresses. This could be a perfect 100% success rate (but see the promotion tabs section below)!
However, if you use AWeber or MailerLite and all your users are with Microsoft, this could be disastrous. Also note that Gmail, Yahoo and AOL seem to be the easiest inboxes to get into. Except for poor Benchmark.
Gmail is one of the most used providers worldwide and as you may know, they categorize your emails in different folders. If you land in the promotions folder you, your open rates might suffer. On the other hand, landing into the primary inbox is a big plus for you.
|Tool||Main Inbox||Promotions Tab|
The main thing to note here is that Gmail seems to be onto Mailchimp. All their emails went directly to the Promotions Tab.
As the emails in our tests were pretty much identical, we can pinpoint the differences in delivery to the individual newsletter service.
For bulk emails, there is also a complex chain of trust in place. The newsletter tool has to trust your email address and domain name. Then, the Internet Service provider (ISP) has to trust the newsletter tool’s server.
If at any point recipients start complaining about the emails they receive, an ISP such as gmail.com might say: we’re considering this server to be spam-heavy. Bounce rate and user behaviours also affect these results (i.e. recipients placing the messages in the spam box themselves).
This is why newsletter tools have every interest to maintain their server’s reputation, and why you should make sure you choose one with solid email deliverability results. But you can also improve your chances by following these steps:
While your newsletter service is working hard to keep their server reputation pristine, you can also make sure the odds are stacked in your favour by implementing the following practices:
So as you can see, email deliverability can have a huge impact on your email marketing campaign. Choose the wrong provider (sorry Benchmark) and you could expect about 40% of your messages never to reach the right inbox. But the results are looking really good for ConvertKit, Constant Contact, Drip, GetResponse and MailChimp. Interestingly, GetResponse was also heading for a top spot before the last (pretty poor) test round moved them down to position 5.
We know that five rounds of tests is not enough to make general statements about the deliverability of certain email marketing tools. The results seem to fluctuate quite a bit. That’s why we will repeat this test in a few months time. We are looking forward to this! Because then we will also add these test results to the individual reviews.
Having said that, you should also take these results with a pinch of salt, and look at the bigger picture. Price vs features, for instance could also be just as important for you. Or ease of use vs design options. Does it really matter if some emails go missing if you have to pay twice as much for the service? Of course we try and take all these parameters into account when we help you compare all the different newsletter tools.
Update: for the latest deliverability test results, check out our infographic.
Feedback, questions? Stay in touch via the comment section below.