diana 🏳️⚧️🦋🌱dianea@lgbtqia.space
Oct 18, 2026, 1:24 PMretooted Dillo browser
Moved our git web server from cgit to stagit to reduce load on the server caused by crawlers, as it only uses static files.
https://git.dillo-browser.org/dillo/
Roughly 99.9% of requests are bots fetching commits and files over and over again at a rate of 50 req/s despite our robots.txt excluding any crawler. Stagit only exposes a small number of files and commits so they are not reachable to bots.
Sadly, many anti-bot protections rely on JS or fingerprinting which come with their own set of problems.