Login
You're viewing the mastodon.iow.social public feed.
  • Deedee@treehouse.systems
    Oct 22, 2026, 6:54 AM

    It is a sad thing, but my oss and external websites have essentially been made private or projects disbanded.

    Between the UK online safety act, AI scrapers, AI "contributions", various critical vulnerabilities... I've just concluded that running small things in the open just isn't worth the time, and ceased to be fun a while ago. Especially if I'm the primary user and just wanted to share.

    I did pass a few things on to others, such as the Go HTML sanitizer, a large community platform, and a couple of other things... But if it wasn't large, it disappeared over the last year.

    Yesterday I discovered a few old WordPress sites dating back a decade... It's not worth doing the work to keep them online.

    And also yesterday when I installed a couple of new tools... Those became accessible via TailScale only.

    The web really got screwed huh.

    💬 1🔄 1⭐ 0

Replies

  • Deedee@treehouse.systems
    Oct 22, 2026, 7:10 AM

    By far the biggest attack on the web: AI scrapers.

    And Cloudflare is really not good at blocking it, but also I don't want to use them. And even with a lot of knowledge and skill at my disposal (I used to lead the DDoS protection team at Cloudflare, and was the person who introduced Rust there to improve the capabilities of the Firewall and DDoS protection)... Still the 11 requests per IP over tens of thousands of residential IPs from mostly real browsers at accumulative hundreds to thousands of requests per second. By the time you see the attack and add a block rule, it's pointless as it just looked like a single client accessing one page, and that client will never be seen again.

    There's no real way to defend against that except to make the website private and only allow access to those you validate IRL... Or to shoulder a massive financial cost trying to serve it all.

    What a world!

    💬 1🔄 0⭐ 0
  • 4 mice in a php filealgernon@come-from.mad-scientist.club
    Oct 23, 2026, 1:01 AM

    @dee

    There's no real way to defend against that except to make the website private and only allow access to those you validate IRL...

    There's a neat trick, though. As far as I observed, the vast majority of those real browsers from residential IP scrapers are all following a crawling queue collected by simpler, easier to identify bots. So if you can poison the queue, you can block an IP after looking at one request path. That's a big save. No need to serve those, no need to do a TLS handshake either.

    I'm trapping identifiable bots in a maze of garbage, where every link carries a poison ID I can identify trivially. The residential proxies virtually disappeared, my firewall takes care of them.

    I still see ~5m requests a day across the few dozen sites I host, but that's a far cry from the 110m1 I've seen before. I can handle a constant hum of 50-60 req/sec, that keeps their queues poisoned.

    It still sucks I have to do this, but the problem can be mitigated, in my experience.

    1. "only" 110m, because my server fell over under the load. ↩︎

    💬 0🔄 0⭐ 0