zombie anti-spammers? fabulous

[rone]

Dear dr_strych9: please supplement my arguments against real-time blackhole lists with this delightful story about a blackhole list that came back to life fifteen months after its death and caused all its subscribers' incoming mail to bounce.

Comments

[zonereyrie.livejournal.com]

It depends - were the queries done by DNS name or IP? They're often setup using IP to avoid a DNS-spoofing attack.

I don't think it is feasible for everyone to maintain their own list of 'bad' networks. My company certainly can't afford the resources, and before we started using them the spam volume was also killing us. We turn off the RBL (it's been done by accident) and spam fills up the inboxes, even though the Bayesian filters, etc, catch a lot of it. The vast majority of email that hits our mail server is spam - and that's after passing through a relay that only allows email to whitelisted addresses. 80-90% spam.

Using the RBLs as part of the filters are the only thing that really keeps it manageable. And frankly I'd rather take the risk of using a respected RBL as part of the anti-spam system than take the hit from all the garbage that gets through the rest of the filters without it.

I'm not going to use some random RBL, but there are lists that are well established and respected, and I consider the risk of using one of those acceptable given the benefit.

The tradeoff is the, IMHO, minimal risk from using a respected RBL vs. lost resources reinventing the wheel to develop the filters 'in house'. It just isn't worth rolling your own.

[ronebofh.livejournal.com]

So re-IP the affected machines. Or just turn them off.

If there's no arrangement between my company and the RBL administrator that includes liability if the RBL fucks up, there's no way i'm running my mail with it. Period.

[zonereyrie.livejournal.com]

They shouldn't have to retire an IP because some other admins are lazy. Not everyone has IPs to spare.

Does your company also only use commercial software where the vendor has liability? Or do you use an free or open-source software? If the latter, why is that different? Do you audit the source of all the free software to make sure there are no errors there too? Especially if you're running a non-commercial mail server.

Is there a risk with RBLs? Sure. But there is a risk with everything. I think the risk is low (if you select a stable, respected RBL) and the reward is high. There is a small potential of a problem with the RBL blocking email until caught - which would likely happen fairly quickly. That's weighed against the cost of not blocking the tons of spam daily and having to spend resources to deal with it. Constant, known, large pain or the small risk of some short term pain cleaning up an RBL gaff.

I think it would be a massive waste of resources for every company to invest in developing their own filters. That's reinventing the wheel thousands, or millions, of times over. And that's a terrible waste of resources better spent doing other things and not wasted on building a list someone else has already built.

There are a number of things that can disrupt email delivery - a problem with the RBL isn't high on the probability list, IMHO. We had a recent outage which blocked incoming mail because our commercial A/V system wedged and plugged up delivery and had to be manually recovered. And the vendor isn't liable for anything.

[zonereyrie.livejournal.com]

There's another aspect - RBLs harness the power of the group. Instead of waiting until *I* get spam from a site, if anyone if the group gets spam then the RBL protects the rest of the group from *future* spam. Instead of it always being a response to a problem, you can prevent the problem. So instead of everyone having to individually experience the pain before applying the cure, some small number get the pain, then an immunization is developed, and the rest are spared. That's the same system that makes Gmail's spam filtering so good - it only takes some subset of users to report something as spam for their system to learn and protect everyone.

As I've said, RBLs are part of a system. With SpamAssassin you can use them as a weight, and not necessarily a real black hole. With the commercial system we use at work it isn't as flexible as SA, but it is one of many filters the mail goes through and one of the first is a learning whitelist. When we first installed it we had to do some training, and I fed it a lot of old mail to help it learn (good and bad), but after a couple of months it settled down and has been running well for a couple of years.

I'd rather my friends warn me of a scam before I get burned, than have to be scammed to learn the lesson myself. And I'd rather have the power of a large group working for me than be on my own.

[ronebofh.livejournal.com]

Maybe i'm just not with it these days, but unlike most popular open-source software, the history of RBLs has not been one of stability or respect. I might change my tune in 5 years.

Mail failure due to a false positive hit is a pernicious problem, because how do you tell the other party that you're not a spammer if you can't send them mail? It's also a PR problem: "HOW DARE YOU CALL ME A SPAMMER"

[zonereyrie.livejournal.com]

I've run into that personally, just recently. Someone used the comment form on my GizmoLovers.com site to ask me a question. So I replied to them at the email address they used. Only they used their work email address. I got a bounce with a standard error - 550 or something like that, saying it was spam. I figured it was because I mentioned some pricing and such, so I send a plain message - it bounced the same way. Eventually I contacted their postmaster - turns out they have a 'no non-work email' filter rule, which they lumped in with their anti-spam rules. I was a little annoyed at being called a spammer by their ruleset, but mainly I was annoyed at the bad configuration they used that caused that. If they want to block non-work email, fine - but send an appropriate message to that effect, not a red herring about spam.

I think most people understand mistakes happen, and if you fix it quickly they'll get over it. In our case, since mail hits the white list first, even a 100% false positive RBL wouldn't impact email with anyone we have exchanged email with in the past. (The list auto-updates - anyone a user emails is on the list.) So it would only block any new senders, which is less of a problem.

I would never recommend relying on RBLs as your only line of defense, or thinking they're infallible. But I think they can be part of defense in depth. Spam is a huge issue, and no one solution will take care of it all.

At work mail goes through many filters - first the coarse whitelist on our relay that only allows email to valid addresses through, then commercial software on Exchange that runs through a number of filters - SPF, white list, a corporate blacklist (addresses we blocked), a phishing blacklist, a spam address BL, then a DNS RBL, keyword filter, header filter (malformed/forgeries), bayesian... probably others I forgot. And then I have Outlook's Junk Mail filter turned on, which catches a lot of the spam that makes it that far.

In the end I end up with only a little spam making it into my inbox - which is a lot better than a few years ago, before all of this was in place. Before the commercial system we had a lashed-up SpamAssassin setup, but that took a lot more labor to keep working well, and was never as effective really. And before that, when I first got here, we didn't have any anti-spam. That was suboptimal.

[qwrrty.livejournal.com]

As with open source software, some RBLs are run responsibly and with great diligence to accuracy, and some are.... not so much. You are familiar with this distinction in Usenet software, I believe.

For the purpose of comparison you may consider ORDB to be the FirstClassBBS (or perhaps the QWK Offline Reader) of realtime blackhole lists.

[also-huey.livejournal.com]

the history of RBLs has not been one of stability or respect

I don't think that's entirely fair. I'll grant you that the history of RBLs run by shitheads hasn't been one of stability or respect, but several (most notably Spamhaus and the CBL) are both well-run, mostly transparent, and generally not insulting except in the face of categorical stupidity. But to lump folks like Linford, and even Vixie and Rand, in the same category as complete fuckin snapperheads like Alan Brown, Ron Fucking Guilmette, Joe Jared and Brian (or Brielle, apparently) Bruns - that's apples and oranges. The folks in the former group are running a business that tries to help preserve the utility of email, while the guys in the latter are just waving penises.

[ronebofh.livejournal.com]

I was getting all het up about you calling Spamhaus well-run until i realized that i was thinking about SpamCop.

[also-huey.livejournal.com]

Believe it or not, Spamcop largely got unfucked a couple years back, when it stopped being run by Julian and started being run by Ironport. I too used to enjoy the whuppin-on-Spamcop, but after Petersen took over, holy crap, they fixed it. AFI has statistics (http://www.dnsbl.com/2007/05/spamcop-bl-another-look-its-accurate.html).

[motis.livejournal.com]

A couple of years ago, I had a problem with e-mail from my server in China being classified by an RBL as spam. I run a private, by-invitation-only site that requires deliberate registration, and the only e-mail we send is to people who definitely want to receive it and have explicitly requested it. Our mail was being blocked because Spamhaus listed the entire data center where my server is co-located as a spam source. It's true, there were some spammers using servers in that data center... IIRC, two or three. But the data center is in China, where there is no option of putting your colo box in a different ISP's data center, and the people running the place were unable to read the RBL notices being sent to them in English, and had no idea what was going on. Hundreds of servers in that very large data center were potentially impacted (I say 'potentially' because surely some of them don't send out e-mail), and 99.99% of those servers were owned or rented by people who had nothing to do with any spam being sent to anyone.

If putting pressure on customers to use a different ISP's data center is the object, then this particular action was one of those EPIC FAILS we all know and love, because there aren't other ISPs in China to give your business to. What ended up happening in this situation is that I was personally forced to act as an intermediary between Spamhaus and the Chinese administrators at my data center in order to get my e-mail working again. Don't get me wrong, I hate spammers just as much as anyone (and given my capacity for hate, probably much more than most)... but I deeply resent having my innocent server's e-mail held hostage, and being forced to work for Spamhaus unpaid. Had they NOT blacklisted the entire data center and instead ASKED me to work unpaid in an effort to rid the place of spammers, I would have been more than happy to cooperate. As it is, I feel like I've been strongarmed. Spamhaus used me and deprived me of any choice in the matter, and I had absolutely nothing to do with any spam sent to anyone from anywhere. The whole thing, in my opinion, was high-handed, arrogant, imperious, and deeply abusive of a power that should not exist in the unregulated form that it currently enjoys. If they want to block domains or IPs that are known to be spam sources, so be it... but this whole business of putting pressure on data centers via their innocent customers stinks like a ripe pig carcass left in a moldy sauna for the weekend.