New on :foojay: Foojay Today: Nicolas Frankel @frankel explains the levels of bot protection he met while searching for his ancestors online.

He covers what works at each level, from simple curl options (User-Agent, Referer, cookies) to JavaScript checks like Anubis and Cloudflare, up to headless browser detection and why a headed browser with Playwright often helps.

Scrape responsibly.

https://foojay.io/today/bypass-bot-protection/

#Scraping #Playwright #WebDev