Falling into an Email Blacklist with DreamHost
A blacklist can be a powerful tool for identifying spam email senders, but if you find yourself unfairly blacklisted, it’s maddening. Since sometime last September, roughly 30% of all my outbound customer-related emails have been rejected by the destination email server. Most of these are order confirmations or shipment notifications, and when they go missing, I get lots of frustrated inquiries from customers wondering why they never heard anything after placing an order. The rejections from the destination email server typically look like this:
550-“JunkMail rejected – pdx1-shared-relay1.dreamhost.com
[66.33.200.130]:40663 550-is in an RBL on rbl.unified-contact.com, see
Blocked – see 550 http://psbl.surriel.com/listing?ip=66.33.200.130” (in
reply to RCPT TO command)
Reporting-MTA: dns; pdx1-shared-relay1.dreamhost.com
X-Postfix-Queue-ID: D4C0A30000327
X-Postfix-Sender: rfc822; steve@bigmessowires.com
Arrival-Date: Mon, 9 Jan 2017 14:07:03 -0800 (PST)
The exact message varies, but it usually mentions being on a realtime blacklist, or simply says my email was suspended, blocked, or refused. Other mail hosts such as Yahoo and Outlook.com take a passive-aggressive approach, and just drop the connection when I try to send email to one of their customers:
mta6.am0.yahoodns.net[66.196.118.34] while sending RCPT TO
Reporting-MTA: dns; pdx1-shared-relay2.dreamhost.com
X-Postfix-Queue-ID: B795D38088EC2
X-Postfix-Sender: rfc822; steve@bigmessowires.com
Arrival-Date: Wed, 4 Jan 2017 15:51:07 -0800 (PST)
I haven’t tested it thoroughly enough to be certain, but I believe the problem only occurs for auto-generated emails from the BMOW store, and not for customer support emails that I compose manually – even though both are sent through mail.bigmessowires.com to the same destination email server.
Identifying a Spammer
So how did I get on these blacklists? It turns out it has nothing to do with the content of my own emails, but is entirely due to my web and email hosting provider, DreamHost. They offer cheap and convenient hosting, which doubtless attracts a few people using their servers for evil purposes, sending spam. This causes the DreamHost email relay server to be placed on multiple blacklists, affecting all the other DreamHost customers who share that relay. While I only started to notice the problem last fall, this forum discussion reveals it’s been happening since at least 2013.
I’ve contacted DreamHost customer support several times about this issue. At first, they said the problem was resolved, and they had confirmed with all major blacklist providers that the block on the affected relay had been removed. And the situation did seem to improve temporarily, though it was never completely resolved. When the blocks grew more frequent again, I contacted DreamHost a second time on December 8 and received this reply:
sending mail, and it is used by hundreds of individual users. …
Over the last week, we have experienced a surge of compromised customer
SMTP users that were being used to send out malicious emails. Although we
monitor outgoing mail traffic closely and were able to stop these
compromised domains quickly, enough email managed to get through to cause
several blocklist providers to block a percentage of our email servers.
Many providers have already delisted the IP, but some holdouts do remain,
with whom we are actively working to fully resolve the block. If these
rejection notices continue for more than about 48 hours, please don’t
hesitate to let us know.
Sorry, we’re working on it, everything will be back to normal soon. But unfortunately it didn’t go back to normal, and a few weeks later I contacted them a third time. I received a detailed technical reply that focused primarily on a specific provider named 1&1. Apparently 1&1 doesn’t like the way DreamHost mail servers identify themselves when communicating – an issue related to reverse lookups involving a load balancer – so the DreamHost servers get blacklisted regardless of the content of the email. It wasn’t clear if a solution to this identification problem was imminent, or even possible. Customer support also mentioned that it can take up to a month to be removed from a blacklist:
Backscatter, and LashBack), provide a paid “express” delisting, while
imposing an unreasonable long wait for manual or automated delisting (In
the case of LashBack, they autodelist after a month). As this amounts to
extortion, it is Dreamhost policy not to utilize paid delisting services
(they provide no added benefit to customers, encourage “bad behavior”,
and are generally a sign of an overzealous mail system administrator).
It seems unlikely that 1&1 is the only remaining problem, since my emails to domains like Yahoo and Outlook.com are also being rejected. As far as I’m aware, these are unaffiliated with 1&1.
Getting Past the Block
DreamHost’s responses have all been apologetic, giving the impression that service should be back to normal soon. Maybe I should just be patient and wait, but it’s been three more weeks since that last customer support response, and the situation hasn’t improved. The 2013 forum discussion complaining of this same problem proves it’s not a one-time occurrence. And I received no reply to my most recent CS inquiry asking for a status update or work-around suggestions.
Maybe I should move bigmessowires.com to a Virtual Private Server with a unique IP, instead of relying on shared hosting. I’d consider that if I were confident it would fix the problem, but that’s exactly what the 2013 forum poster tried and complained didn’t work. It’s unclear to me whether that was his fault or DreamHost’s. Even if I knew it would solve the email problem, I’m a little reluctant to jump to a VPS due to the extra server admin hassles it would involve. I really like the convenience of shared hosting, where I focus entirely on the content and leave the server administration to someone else.
Perhaps it’s time to migrate the whole site to another hosting provider, but I don’t think so. I expect most other shared hosting providers will have similar issues, and possibly worse service. During the 13 years I’ve been with DreamHost, their customer support has been excellent. This email blacklist problem is the first time I’ve felt let down by their service.
The best option I’ve come up with is to move BMOW’s email functions to a more “trusted” provider, while leaving the web site and store with DreamHost. That would mean monkeying with DNS entries to relocate mail.bigmessowires.com and a few others, or else simply using a different domain like bmowmail.com for all email. Zoho looks like it might fit my needs, and it would be free for my level of usage. I need to dig into the technical details to confirm it would do what I think it does, and would actually solve the blacklist problem.
If you’ve ever dealt with an email blacklist dilemma, or have any other suggestions on how I might resolve this one, please leave your feedback in the comments. Thanks!
Read 12 comments and join the conversation12 Comments so far
Leave a reply. For customer support issues, please use the Customer Support link instead of writing comments.
I use FastMail with a custom domain for my business and personal email and I\’ve been very happy with their service. I migrated from Gmail a few years ago and I\’m never going back. As a plus, they also provide native push email to the iOS Mail app.
They walk you though the records that need to be added to your DNS configuration. For controlling spam, you need to add DKIM and SPF records, but they tell you what to use so it\’s just copy and paste.
https://www.fastmail.com/help/receive/domains-setup-mxonly.html
There’s an interesting comment in that 2013 discussion that explains why (in the writer’s opinion) all email hosts are susceptible to this problem, so switching away from DreamHost for email may not be any better. https://discussion.dreamhost.com/thread-138445-post-170099.html#pid170099
You probably shouldn’t be sending email from your own server, whether it is a shared host, a VPS, or a dedicated server. Use a service that specializes in transactional mail. Their entire focus as a business is making sure your emails get delivered. I prefer Postmark (https://postmarkapp.com/). There’s also Sendgrid, Mailgun, and Amazon SES, among others.
Hi,
You say that the non-automated messages are fine so I assume you are using a different outgoing mail server to sent those?
Can you specify an outgoing SMTP server for the automated messages or are you tied into using the one they provide?
If you can change it and you don’t have one you can change it to then I can offer use of mine. I’m a longtime bmow reader and happy to help out if I can.
I hope it all works out.
I was wrong about non-automated messages, although the behavior is inconsistent. In one recent instance, a customer emailed me a question, and my reply was rejected by his server as spam. But in another instance, when I directly emailed a customer whose automated ship notification had been rejected, my email got through fine. Maybe there’s more at work here than the IP blacklist, or the list changed in the days between the two emails.
When I compose email manually, I use a desktop email client, which connects to mail.bigmessowires.com with a username and password. But when an automated email is sent from the store or blog software, I think it uses non-authenticated sendmail on the local host (it uses the function wp_mail(), which in turn uses PHP mail(), which uses sendmail). In both cases I think it ultimately uses the same server with the same IP address to send the mail, because mail.bigemssowires.com is actually the same host as my web server.
I don’t know if email sent with an authenticated username/password gets something extra added to its headers that decreases the chances of it being rejected by the recipient, but I think not.
The blog post mentioned “passive-aggressive” rejections from some receiving servers, where the server simply drops the connection rather than sending a refusal response. I received a note from DreamHost CS yesterday, saying these aren’t mail blocks, but intermittent problems caused by high load on the receiving server. I could believe that if the problem were intermittent with those servers, but it’s not. Yahoo, outlook.com, hotmail.com, and live.com.au appear to reject ALL my emails in this way, for multiple emails and delivery attempts.
You use your server for sending your hand-typed mail too? You should stop doing this. 20 years ago you could get away with it. But running a mail server is a full time job. You’ve now seen what happens when you don’t use a professional mail service. Use fastmail or zoho or even gmail for your inbox, and postmark, sendgrid, or mailgun for automated mail from your server. Both can use your domain name at the same time. It’s easy and your provider will have instructions for setting it up.
Thanks, I’m undeniably ignorant of normal email administration practices in 2017. Stupid question: why would you use one service for hand-typed email, and a different one for automated email? As long as I have a username and password on an SMTP server somewhere, couldn’t I use the same service for both? I don’t send a large volume of automated email – just a couple dozen per day at most. And for manual inbox email, I don’t really care about having a web client, but prefer using a stand-alone desktop email client with IMAP.
You’re right, I think you would be fine using the same service for your inbox and the automated mail. (I guess it’s possible it might run afoul of some specific provider’s ToS.) Some reasons to use a transactional mail service are faster delivery, api integration, reporting and analytics. I tend to just automatically throw in postmark for everything automated because the level of effort is the same, postmark is designed for that application and has the corresponding tools and level of service.
I had the same issues with my host, suddenly last year at one point a good % of my automated email (shipping conf etc) started getting rejected by hotmail, yahoo and others.
I was mentally preparing to move to a dedicated email service (huge PITA right?) but a user contacted me and helped me setup DKIM and SPIF values in cpanel. That seems to have solved all those problems, never had an issue after that, glad I didn’t have to move to another service.
I was able to set that up from researching the web about what those things mean and how to set them, I suggest you try this first since it might solve your problem.
Felix, thank you! You were right, my outgoing emails did not have DKIM and SPF set up correctly. That’s now fixed. I’ve also changed the store software to send email using an authenticated SMTP connection to the mail server, rather than PHP’s mail function, which I believe was using sendmail directly from the web server. So there should be no difference between store email and hand-typed email. None of this will get my IP address off any blacklists, but I suspect I was also losing emails to spam filters due to the missing DKIM and SPF info. I’ll watch what happens with outgoing mail in the next week. It still may be best to switch to professional email provider.
A follow-up: since configuring DKIM and SPF correctly in my outbound emails, I haven’t had any more email blacklisted by the receiving mail server. I still have some emails landing in the recipient’s spam folder in their email client, but at least this is progress.
i have a problem with my Macintosh 512K ED with the floppy emu I have the Macintosh folder in the Macintosh folder on the emu
blanks 800k does not work anymore it worked before but now its messed up bc wanted to try to put software on it and I made a backUP
of the emu but the backUp does not work for the blank