Login
You're viewing the techdon.dev public feed.
  • Nov 14, 2026, 3:35 PM
    retooted lux 🦊:therian:

    @kouett @jaseg

    So, multiple points:

    • Linux repos are, in and of themselves, tar pits. They are gigantic. An oddball estimate i had made a year ago is that the entire tree history between root and the v6.17 tag generates at least 324,226,808,132 pages with unique URLs on a forge.
    • Scrapers are stupid, they will fetch every single one of those with whatever strategy they see fit.
    • For the same reason, they will readily accept any data. I feed them small random garbage pages, and what happens is that they saturate the machine that generates the garbage (because generating the garbage and returning it is faster than fetching the backend, so more bots come feast at the poison pit)
    • Like any denial-of-service attack, the only true way to fight it off is cutting connections down as soon as possible. I adopted a mixed approach where i would feed poison and then, every hour, i'd ban the entire /24 of the IPs that fetched poison. It only helps with big clusters of proxies, but they're very fucking distributed
    • The operating logic behind those bot swarms can detect when your server starts replying 504 but they'll usually keep close to that limit.
    • The only thing that's reliably worked for me was taking my entire forge down for a whole week and, when it came back, putting it behind a simple Basic Auth prompt.

    The state of self hosting in 2026..

    💬 1🔄 10⭐ 22