Login
You're viewing the functional.cafe public feed.

Replies

  • Nov 8, 2026, 9:19 AM

    @dcz How come you're completely ignoring the bit of ai bots hammering servers into death as a main argument for things like anubis? It's either your user wastes a little resource on visiting, or your waste a shit ton more then those users combined on ai-search-bots.

    💬 1🔄 0⭐ 0
  • Nov 8, 2026, 9:44 AM

    @wmd It's in the "Spam" section. Keep reading!

    The whole point is that by taking this path of least resistance, instead of spending your time, you're spending users'. And maybe you could find a better way, or a better balance.

    Or maybe the Internet is as good as dead and we need to build a new cool thing.

    💬 1🔄 0⭐ 0
  • Nov 8, 2026, 10:58 AM

    @dcz Hmm, I don't see it? the page pentions AI twice, only in a different section in a different way, see no mention of that or llm in the spam section.

    Also, I feel the entire read is over an hour?

    I'm no fatalist, but I would say the 90's internet is dead, or at least pretty much smells and acts like it. This is more a battle ground and our resistance comes at a high price. But the trust of the 90's is died a long while ago, with good reason. I won't be blaming the resistance though.

    💬 1🔄 0⭐ 0
  • Nov 8, 2026, 12:10 PM

    @wmd I mention bots hammering the website.
    Perhaps you're looking for me attributing that to AI, but I don't have any evidence for that, so I'm not implying it. The intentions of those behind the scrapers don't matter anyway to how to defend, at least I can't see how.

    I don't know how long it takes to read but writing it took over 10 hours :D

    How about we create a better Internet replacement that isn't dead? Learn from the history so that we can repeat it.

    💬 1🔄 0⭐ 0
  • Nov 8, 2026, 1:34 PM

    @dcz sounds like good plan, but everyone I know who switched to using anubis did this because of ai bot hammering. So from this perspective it seems crucial to mention. 🤷🏼‍♀️

    💬 1🔄 0⭐ 0
  • Nov 8, 2026, 1:55 PM

    @wmd I'd be interested in hearing about this in more detail - then I could write something about the link. So far, I haven't seen anyone present proof that AI is doing the hammering. I only heard conjectures.

    💬 1🔄 0⭐ 0
  • Nov 8, 2026, 2:51 PM

    @dcz
    There's the circumstantial evidence of the hammering coinciding with the broad availability of LLMs. There's also the fact that many of the requests doing the hammering reference an LLM in their user agent strings.
    Some of it is from LLMs doing work in response to a prompt but more is the collection of data for LLM training.
    @wmd

    💬 1🔄 0⭐ 0
  • Nov 8, 2026, 4:28 PM

    @cwilcox808 @wmd Very interested in researching that, had anyone published such logs?

    /me looks at my own logs...

    a couple AI search crawlers, but fewer than feed readers.

    💬 1🔄 0⭐ 0
  • 💬 2🔄 0⭐ 0
  • Nov 8, 2026, 8:38 PM

    Well, thank you Curtis for the suggestion that our wee blog is not "tiny" or "irrelevant".

    Nor is your site.

    But yeah. We get about 1,000pv/month from actual people. Roughly 10x that from AI bots.

    It's exhausting trying to swat them away. They're like a tornado of angry wasps. And they just lie and cheat and defraud.

    I'm a public servant, and I do a lot of this stuff in my own time so that you're not paying for it.

    But I am.

    @cwilcox808 @dcz @wmd

    💬 0🔄 0⭐ 0
  • Nov 11, 2026, 9:27 AM

    @cwilcox808 @wmd Interesting, thanks. I was always under the impression that Anubis got deployed as a last resort, when there's no identifiable pattern to block or signature contact abuse. If people deploy it those situation, it looks like even more of a path of least resistance than I suspected.

    I'll have to update my thinking for it.

    💬 1🔄 0⭐ 0
  • Nov 11, 2026, 9:42 AM

    @dcz @cwilcox808 wow, that is not a conclusion I would draw.

    No identifiable pattern for blocking doesn't mean it's identifiable as ai bots. Just that there is no way of effective blocking. And when simple blockw don't work, Anubis is what is left.

    Also reading more like victim blaming? The fault lies capitalism, big tech, corps, etc. not people defending against it.

    💬 1🔄 0⭐ 0
  • Nov 11, 2026, 11:49 AM

    @wmd @cwilcox808 The stats in the other post show that the huge AI traffic is attributed correctly, so there's a pattern that can be blocked, so in that situation Anubis would not be what's left.

    If unattributable traffic is the problem, and Anubis is the choice, then it can't be attributed to AI, so that's not what the aforementioned people must be doing.

    I mean, the entire post I wrote is victim blaming. We're victims of one problem, so we become perpetrators of another. What's the way out?

    💬 1🔄 0⭐ 0
  • Nov 11, 2026, 1:03 PM

    @dcz
    Some people do use Anubis because it's easy to implement and pure name recognition; they have this problem and they've heard of it as a solution.

    Some use Anubis because they do have load problems from more stealthy bot operators, those that don't identify themselves in their user-agent strings, instead using ones copied from real browsers plus using things like residential proxies (aka routing traffic through home devices infected with malware).

    Some use Anubis because they really don't want all their content (or their user's content) scraped and a proof-of-work (PoW) strategy slows the bots down enough to be effective (and maybe a deterrence). They're also happy about PoW wasting more of the bot operators' resources.

    @wmd

    💬 1🔄 0⭐ 0
  • 💬 0🔄 0⭐ 0
  • Nov 11, 2026, 3:21 PM

    @cwilcox808 @wmd The most effective blocking method is to block everyone and take your site offline.
    The problem with a lot of those clever solutions is the innocent people who are caught by it. Blocking IP addresses forever will catch a lot of legit people in the blast. They won't tell you that they are affected. Would you bother contacting the admin to get unblocked?
    That's why I wrote that essay: to speak for those who are blocked and frustrated.

    💬 0🔄 0⭐ 0
  • Nov 10, 2026, 12:53 AM

    @dcz this talk made me question something I was 100% sure about, great to see it published!

    About member induction - have you seen an approach you could recommend? I'm kind of worried it prevents people who tend to lurk before interacting from joining.

    💬 1🔄 0⭐ 0
  • Nov 10, 2026, 8:38 AM

    @klh_io Interviews work good in person but online I think they would be odd and scary.
    I already have friends who are scared of in-person interviews.
    Still, there are forums where you are required to make a "hello" post before jumping in. That maybe has the advantage of showing that the process is not scary?

    💬 1🔄 0⭐ 0
  • Nov 10, 2026, 8:39 AM

    @klh_io
    Maybe you could have some hybrid approach: personal induction and get full rights, or choose to do some other ladder (lurk with an account for a month, then trial period where you can get banned more easily to filter out bots).

    In the end, I'm usually on the user side of communities, so not much practical experience, sorry :(

    💬 1🔄 0⭐ 0
  • Nov 10, 2026, 10:45 AM

    @dcz oh I think user perspective is exactly what we need here, it's easy to say something works from the admin point of view if (unintentionally) it excluded the people who could disagree.

    The ladder could maybe be expanded - read only access everywhere with limited write in some places (in case someone is joining with an urgent problem, or the induction room)?

    💬 0🔄 0⭐ 0
  • Nov 10, 2026, 9:47 PM

    @dcz with regards to Anubis, free software people actually don't like it either: fsf.org/blogs/sysadmin/our-sma

    I fully agree with your shouting though and what makes it worse is that in my experience first world countries are treated way better by many of these services than other countries. This is likely also related to the unequal division of IPv4 addresses and CGNAT being more prevalent. Depending on my location I can access certain sites without JavaScript, but then from another country I'm forced to run the proprietary captcha JavaScript which means I need a heavier browser from there.

    I think that IPv6 could also help with separating legitimate users, but this does not seem to be picked up in general...

    💬 0🔄 0⭐ 0