Aug 7, 2026, 9:35 AMCloudflare (>20% of all domains) has started automatically implementing DSM Dir art 4 rights reservation (ie copyright/TDM objection to scraping) for all sites that do not have a robots.txt file. They also adding purpose fields (new content-signal field with search, AI train, AI use) to robots.txt unilaterally and are enforcing it through monitoring and analysis. https://developers.cloudflare.com/bots/additional-configurations/managed-robots-txt/
Aug 7, 2026, 9:36 AMthis is quite a big deal and ramps up potential liability. it also is a move to put pressure on Google to split Googlebot into search and training (allowing people to block training but allow search).
Aug 7, 2026, 9:40 AMthey are also attempting to extend legal interpretation (potentially) with their 'use=reference' category, which their docs indicate is says allows you to index, make an excerpt and link back, but does NOT give permission to summarise.
@mikarv this is really interesting. I wonder if any systems are actually respecting it?
Aug 7, 2026, 11:22 AM@mdekstrand the point is that they have to unless they want to risk cloudflare blocking them, turning it from a request into technoregulation
@mikarv doesn't change the fact that #ClownFlare is a #RogueISP that has always been per of the Problem and never the solution!