Proxy tier benchmark

Which class of proxy does a public web target actually require?

Measured, not asserted. Published by BotProxy — see Bias and conflict of interest.

What class of proxy does a given public web target actually require?

This harness answers that question by measurement, not assertion. It runs an identical request pattern against each target through four arms that differ in one variable at a time, and records success as a content assertion rather than an HTTP status code.

direct         no proxy, straight from the benchmark host
dc_sticky      datacenter IP, one pinned exit for the whole run
dc_rotating    datacenter IP, a fresh exit per request
ad_sticky      + browser TLS/HTTP2 fingerprinting, one pinned exit
ad_rotating    + browser TLS/HTTP2 fingerprinting, fresh exit per request
residential    residential exit pool

The output is a per-target minimum viable tier: none, datacenter, datacenter + anti-detect, or residential.

Why this design

Cross-vendor proxy benchmarks confound plan tier, pool size, pool health and time of day, and are usually run on trial accounts that do not represent the paid product. This runs one harness, one set of credentials, one time window, and changes one variable — which makes it a controlled experiment rather than a comparison, and means there is no adverse party who can reasonably dispute it.

It also answers the question buyers actually ask. The decision is rarely "vendor A or vendor B"; it is "does this target need residential, or will datacenter do?"

Reproducing a run

cp config.example.json config.json     # fill in two credentials
python3 compliance.py --write          # gate every target on robots.txt
python3 harness.py --rounds 10 --group dryrun

compliance.py must run first — the harness refuses any target it has never seen. Run both from the host you intend to benchmark from: several targets answer differently by exit geography and ASN, so gating from a laptop and running from a cloud host can disagree (we observed exactly that; see below).

The two credentials must be identical except for the Anti-Detect setting — same account, same plan, same pinned exit location — or the two datacenter arms are not comparable.

Reading the results

results/<run>/raw.csv is one row per request:

column	meaning
outcome	the unit of analysis — see below
detail	which signature matched, e.g. cloudflare_challenge
bytes	response size, the input to cost per 1,000 pages
elapsed_ms	wall-clock, including proxy hops
tls_verified	false on the Anti-Detect arm only, and that is inherent
exit_ip	pseudonymised label for the exit the target saw (see redactions)

Outcomes: success, soft_block, challenge, blocked (403), rate_limited (429), server_error, auth_required, other_status, error.

success requires the target's content assertion to match. A 200 that does not contain what was asked for is recorded as soft_block. Silently degraded or poisoned content is the failure mode a status-code-only benchmark misses, and on protected targets it can be the dominant one — counting those as successes is the most common way a proxy benchmark overstates its numbers.

results/<run>/excluded.json lists every target the compliance gate dropped and why. Exclusions are published, never silently omitted.

Compliance rules encoded here

• Public, unauthenticated pages only. No logins, no paywalls, no access-control circumvention.
• robots.txt is resolved for every target and obeyed, both at selection time and again at run time.
• A robots.txt that cannot be read is not permission. Those targets are excluded and reported.
• The harness never runs faster than its politeness floor or the target's own Crawl-delay, whichever is slower.
• The User-Agent is contactable: BotProxyBenchmark/1.0 (+https://botproxy.com/benchmark; support@botproxy.net).
• Volume is deliberately low. This measures pass rate and efficiency, not throughput; it is not a load test of public infrastructure.
• Where an official API or bulk download exists it is recorded in targets.json and the report recommends it over scraping.

One judgement call, stated openly

Several targets answer 403 to the benchmark User-Agent for /robots.txt itself. Since robots.txt is a published policy document rather than protected content, the gate re-reads it with an ordinary browser User-Agent and then obeys whatever it says. Which User-Agent obtained the file is recorded in robots_fetched_as, because a site that will not serve its crawl policy to a self-identifying crawler is itself worth reporting. If neither User-Agent can read it, the target is excluded.

Two traps this harness exists to avoid

Do not compare BotProxy's own access-log byte counts across the two datacenter arms. A plain CONNECT log line covers a whole tunnel lifetime including the TLS handshake; a CONNECT-MITM line covers one HTTP response. A blocked request therefore logs several KB on one arm and a few hundred bytes on the other, and any threshold applied to both will show a difference that is pure log semantics. The same applies to the duration column.

Do not use RobotFileParser.read(). It fetches with Python's default User-Agent; targets that 403 that UA cause it to set disallow_all, so the harness silently skips targets the gate already cleared — a fingerprint block on the robots fetch masquerading as a policy decision. Both scripts share compliance.fetch_robots() for this reason.

Known limitations

• Arms differ in TLS verification (see tls_verified); this is a property of MITM fingerprinting, not a harness choice.
• Exit IPs rotate between arms by design, so a per-arm result blends pool quality with the variable under test. Exit labels are recorded so a reader can check whether an arm leaned on a single unlucky exit.
• X-Proxy-Ip is emitted on the CONNECT response, which HTTP clients do not expose for HTTPS tunnels, so per-request exit attribution is unavailable and every row carries the exit sampled at arm startup. Rotation was verified out-of-band (a pinned session returned 1 distinct address across 6 requests, a rotating one returned 6 of 6).
• Round count is small by design. This measures whether a tier works, not long-run stability under sustained load.

Results in this repository

File	Contents
REPORT.md	Full results, conclusions and scope limits
PROTECTION-CENSUS.md	What actually protects US public-data portals
DIMENSIONS.md	Per-IP rate limits, geography, and what a one-day run cannot see
DRY-RUN.md	The validation run and the harness defects it exposed
results/six-arm/	1,920 requests, 32 targets x 6 arms x 10 rounds — the headline run
results/full-sticky-baseline/	1,280 requests, single-pinned-IP control
results/dryrun/	200 requests, validation
targets-full.json	Target list with robots.txt decisions and probe evidence
probe_results.csv, probe2.csv	URL reconnaissance behind the census

Every run directory holds raw.csv (one row per request), summary.json, tiers.json where computed, and excluded.json listing what the compliance gate dropped and why.

Bias and conflict of interest

This benchmark was built and run by BotProxy, a proxy vendor, using its own infrastructure. Read it with that in mind — and then read the results, which are not flattering to the product:

• No proxied arm beat the unproxied control. The best arm in the study is direct, from a plain cloud host.
• 19 of 32 targets need no proxy at all.
• Exactly 1 of 32 targets is decided by a proxy tier.
• 14 target/arm pairs are worse with a proxy than without one.
• Where an official API or bulk download exists, this report recommends it over scraping.

We publish this because a vendor benchmark that the vendor wins is worthless. The harness, the target list and every raw result are here so the numbers can be checked, disputed, or re-run against different targets.

The design deliberately avoids testing competitors. Cross-vendor proxy benchmarks confound plan tier, pool health and time of day, and are usually run on trial accounts that do not represent the paid product. Measuring our own tiers against each other, with one variable changing at a time, is a controlled experiment; testing a competitor's free tier would not be.

Licence

MIT — see LICENSE. The results data is published under the same terms; cite the run date, as anti-bot behaviour drifts.

What is redacted, and why

Two things are removed from the published data. Both are stated here rather than done quietly:

Exit IP addresses are pseudonymised (dc-exit-1, res-exit-2, benchmark-host). Publishing a proxy pool's exit addresses would hand every target operator a blocklist, and the residential addresses belong to real households. The labels are stable within and across runs, so the data still shows how many distinct exits an arm used and which arm reused one — which is all the analysis needed. Nothing else in raw.csv is altered.

Figures that describe the size of BotProxy's business are omitted — customer counts, revenue, credential counts, total traffic volumes, and the absolute size of the exit pool. Where operational traffic is cited for context, it appears as a rate or a ratio rather than a total. No result in REPORT.md depends on a withheld number: everything in the benchmark itself, all 1,920 requests, is published in full.