I want to spend a few moments describing how Cloudflare is measuring bots. I am a Cloudflare Specialist Solutions Engineer (SASE/SSE).

Cloudflare sees around a third of the entire world’s HTTP/S traffic. It can only observe proxied traffic that operators have deliberately sent through the network. Otherwise Cloudflare only knows about BGP routes. In the proxied HTTP/S traffic there are various network signals. These are things like headers (e.g. user-agent), the order of the headers, and fingerprinting of attributes presented in the TLS handshake. Signals are evaluated by a machine learning model that has as its input the ~30% of the web cited above. This is used to calculate a bot score of 2-100, with 2 being likely automated and 99 likely human. The scores are divided into bands designated as like likely automated, likely human, and definitely bot or human. There are also heuristics (signatures) for known bots which peg the score to 1. If your user-agent contains “curl”, it will just get that score. No need to look at the ML. The machine learning model uses Adaptive Intelligence, which is Cloudflare’s commercial name for its frequently retrained model. What the model is really calculating is the probability that any given request will be able to solve a challenge. It can do this because every time you see a Cloudflare challenge and solve it, you are helping to train this system to be more accurate in determining whether or not your request was automated.

In addition to the heuristics and ML, Cloudflare can check browser fingerprints and user behaviors by injecting JavaScript dynamically into pages. Previous version was called JavaScript Detections, which performed the evaluation request-by-request, but a new Precursor feature can evaluate browser behavior continuously through a session, and invalidate any previously solved challenge.