Some friends at the OPEN network asked me about my experience using ReturnPath email monitoring when I was managing the tech for a 45-million member international email list. I decided to publish my stream-of-consciousness reply here, in case it helps someone else. –MH
ReturnPath does a couple of things to track email delivery that would be pretty difficult to do yourself, and a couple other things which you can’t do yourself, because they depend on business deals you don’t have.
The basic technology is straightforward; ReturnPath signs up email accounts on many, many ISP’s around the world. They give you this list of “seed” accounts to include in your mass email blast, and then they check their inboxes to find out if your message arrived to every seed address, or whether some of the ISP’s blocked it as spam and didn’t deliver it. Their business advantage is that it’s a logistical pain to keep all those email addresses going, and they’ve already got a stockpile.
They also track a bunch of spam blacklists and alert you if you end up on one. Almost as importantly, they also give you some pretty good inline help about what to do to get off of a particular blacklist (the rules are all different), and how much to worry about it (some blacklists are widely used in spam filters, but others are mostly ignored).
You could do all those things yourself, if you wanted to put in the time. But ReturnPath also has some business deals in place which mean they can compile data you can’t get for yourself.
For one, they seem to have arrangements with a couple of the major ISP’s that somehow report email opens from real users to them. So once the number of real Gmail users who have opened your email reaches a certain threshold, the stats dashboard in ReturnPath will show you the actual delivery rate for real people at Gmail, instead of the estimation based on ReturnPath’s collection of Gmail accounts. This can be important sometimes. I’ve had cases where it *looks* like my deliverability at Gmail is terrible, because 3 of 10 Gmail seed accounts didn’t get delivered, but once data from real users comes it we see that the email is actually being delivered just fine, it was just bad luck that on this particular day a high proportion of that small number of seed accounts was filtered out. This feature only works with a couple of big ISP’s, and sorry I don’t remember which ones off the top of my head.
They also have a program called “Certified Sender”, which some people think is essentially a way to buy your way into more inboxes, at least at Hotmail and maybe Yahoo and some others. The way it works is that you have to pay a hefty fee, then you have to go through a long checklist of both setup steps and also sending practices, and they decide if particular sending IP’s that you use are sending enough high-quality messages to be “certified” as non-spamming.
It’s been argued that if you are following all those stringent rules, you’re already going to have good deliverability. Another drawback is that, as mentioned, it’s expensive. The contract they offered me was for $80,000 USD to get certified. And if you start getting a lot of spam reports on one of your IP’s, they’ll drop your “certification”, with no refund. I have been told by at least one other large activist list that using the certification *did* lead to a noticeable improvement in delivery at Hotmail. Given how hard Hotmail is to work with (ask me sometime about a year-long lobbying effort I participated in with them, including flying to Redmond with gifts in hand…), that alone might make it worth it. It’s also been said that this feels a lot like extortion — pay us a bunch of money and we’ll get your mail delivered.
In case this is sounding like an advertisement for ReturnPath, here are some downsides:
Cost is first. They charge per email-sending IP address which you want to monitor, and also for the use of those “seed” accounts — the more email blasts you send, the more it’s going to cost. The most recent contract I have from them is from a couple years ago, and they were giving me a negotiated deal for about $27,000 USD/year, which was for monitoring 30 IP addresses and up to 150 uses of the seed list per month. (That may sound like a lot of blasts, but we did a ton of test blasts, and often changed the content or the targeting strategy based on the deliverability results.) And there’s no “non-profit” rate, or at least there wasn’t when I was trying to get one.
Their advice on how to improve your deliverability is uneven. Sometimes it’s pretty useful to hear, like for instance “don’t worry about this particular blacklist, nobody pays attention to them”. But often they recommend things like “Spread out your email sending over time, don’t blast the whole list at once”, which is of course useless advice for an activist organization. They’ll also tell you not to bother with personal contact or lobbying with ISP’s, but my experience is that can be massively helpful in certain cases, with certain ISP’s. So, take their advice with a grain of salt.
Also, like any service, sometimes it breaks. Those seed accounts get shut down by ISP’s who notice that no real person is reading them, or they expire for other reasons; data calculations in the stats have been wrong; data has been very delayed in being updated, etc. But those problems, while super annoying, were pretty rare in my experience.
There are a few competitors out there, with which I don’t have direct experience; this seems like a good list of alternatives.
A number of people also recommended the new kid on the block, 250ok . At a glance they seem to have very comparable tools to ReturnPath, at a lower rate, so I’d give them a serious look.
If you have questions I didn’t address, or your own experience with ReturnPath, 250ok, or any other service like this, please comment!