AntiBot & WAF Cheatsheet

DIY request client
  Your code talks to the target with a browser-grade TLS stack,
  keeps a clean cookie jar, and replays challenge posts.
  Most “payload looks fine” failures are TLS / headers / IP / cookies.

BrowserSolver Website modes
  One HTTP API: AntiBot (classify), then Crawler / Rendering / WebUnlocker.
  Unlock playbooks run server-side on allowlisted hosts.
  You parse HTML/JSON; you do not maintain sensor posts.

1. Chrome TLS profile (latest available) + random extension order
2. HTTP/3 off when proxies do not support it
3. Exact browser header order (incl. HTTP/2 pseudo-headers)
4. Same UA + TLS + IP + hints for the whole flow
5. Sticky/session proxies (never rotate mid-challenge)
6. Clean cookie jar (no duplicate names / wrong domain)

Akamai Bot Manager
  Cookies _abck + bm_sz; dynamic <script src="/xxxx/yyyy/..."> near body end
  428 = SEC-CPT; 429 body with {"t":...} often SBSD

Imperva / Incapsula
  reese84 cookie or x-d-token: reese84
  /_Incapsula_Resource?SWJIYLWA=...: utmvc (___utmvc cookie)
  "Pardon Our Interruption": reese84 dynamic (+ possible PoW)

DataDome
  403 with inline var dd={...}; datadome cookie
  i.js / rt:'i': interstitial; c.js / rt:'c': slider
  tags = proactive /js telemetry (not a block page)

Kasada
  429 HTML referencing ips.js; x-kpsdk-* / tkrm_alpekz_* cookies
  x-is-human: Vercel BotID variant

Cloudflare
  Managed Challenge / Turnstile; cf-ray; __cf_bm / cf_clearance

PerimeterX / HUMAN
  px-cdn / client.perimeterx.net; _px* cookies; Access Denied walls

Fastly
  Edge 403 / empty bodies before origin; CDN JS challenges; Fastly headers

type VendorGuess =
  | "akamai" | "datadome" | "kasada" | "imperva"
  | "cloudflare" | "perimeterx" | "fastly" | "unknown"

function guessVendor(status: number, headers: Headers, body: string): VendorGuess {
  const setCookie = headers.get("set-cookie") ?? ""
  const server = (headers.get("server") ?? "").toLowerCase()
  const cfRay = headers.has("cf-ray")

  if (/_abck=|bm_sz=/.test(setCookie) || /sec-cp-challenge/.test(body)) return "akamai"
  if (status === 403 && /var\s+dd\s*=\s*\{/.test(body)) return "datadome"
  if (status === 429 && /ips\.js|x-kpsdk-|tkrm_alpekz_/.test(body + setCookie)) return "kasada"
  if (/reese84=|x-d-token|_Incapsula_Resource|Pardon Our Interruption/i.test(body + setCookie))
    return "imperva"
  if (cfRay || /__cf_bm=|cf_clearance=|cdn-cgi\/challenge/.test(body + setCookie))
    return "cloudflare"
  if (/perimeterx|px-cdn|_pxvid=|_pxhd=/i.test(body + setCookie)) return "perimeterx"
  if (server.includes("fastly") || /x-fastly|fastly-/.test([...headers.keys()].join(" ")))
    return "fastly"
  return "unknown"
}

Bad (stock client fingerprint):
  TLS ClientHello ≠ Chrome
  Header order alphabetical or framework-default
  sec-ch-ua / UA / platform version drift
  Datacenter IP on a residential-only target

Good (browser-grade session):
  Chrome TLS profile + random extension order
  Wire-captured header order for that request class
  Sticky IP for the whole challenge flow
  Clean cookie jar (no duplicates / wrong domain)

Chrome  :method :authority :scheme :path
Firefox :method :path :authority :scheme
Safari  :method :scheme :path :authority

Example Chrome navigation header order (illustrative):
:method, :authority, :scheme, :path,
sec-ch-ua, sec-ch-ua-mobile, sec-ch-ua-platform,
upgrade-insecure-requests, user-agent, accept,
sec-fetch-site, sec-fetch-mode, sec-fetch-user, sec-fetch-dest,
accept-encoding, accept-language, cookie, priority

Use sticky / session proxies for the entire challenge + protected action
Never rotate mid-flow (sensor, then verify, then checkout)
Residential or mobile when datacenter IPs are scored to zero
One clean cookie jar per sticky IP / session
If DataDome slider shows t=bv: IP hard-ban; change sticky IP first
Re-probe with Website AntiBot after an IP pool change

200 + challenge HTML     Cloudflare / soft blocks / empty shells
403 + var dd={...}       DataDome interstitial or slider
403 edge empty           Fastly / generic CDN ACL
428                      Akamai SEC-CPT (provider=crypto|behavioral|adaptive)
429 + ips.js             Kasada
429 + {"t":"..."}        Akamai SBSD hard block
429 / 503 rate shapes    vendor rate limits (retry with backoff + sticky IP)
401 from api.*           missing/wrong x-api-key on BrowserSolver

export BROWSERSOLVER_API_KEY="your-key"

curl -sS "https://api.browsersolver.com/v1/website/crawler" \
  -H "x-api-key: $BROWSERSOLVER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"query":"https://example.com"}'

const API = "https://api.browsersolver.com/v1"
const KEY = process.env.BROWSERSOLVER_API_KEY!

async function websitePost(path: string, body: Record<string, unknown>) {
  const res = await fetch(`${API}${path}`, {
    method: "POST",
    headers: {
      "x-api-key": KEY,
      "Content-Type": "application/json",
    },
    body: JSON.stringify(body),
  })
  if (!res.ok) {
    throw new Error(`${path} ${res.status}: ${await res.text()}`)
  }
  return res.json()
}

import os
import requests

API = "https://api.browsersolver.com/v1"
KEY = os.environ["BROWSERSOLVER_API_KEY"]

def website_post(path: str, body: dict):
    r = requests.post(
        f"{API}{path}",
        headers={"x-api-key": KEY, "Content-Type": "application/json"},
        json=body,
        timeout=120,
    )
    r.raise_for_status()
    return r.json()

HTTP 429 HTML referencing ips.js
Headers/cookies: x-kpsdk-*, tkrm_alpekz_s1.3, tkrm_alpekz_s1.3-ssn
Script path with two GUID segments, e.g.
  /149e9513-.../2d206a39-.../{ips.js|fp|tl|mfc}

Common flows:
  Flow 1: 429 on homepage, then GET ips.js, then POST /tl, then reload
  Flow 2: /fp fingerprint endpoint, then same ips.js /tl path
            (referer = /fp URL); protected calls may need fresh x-kpsdk-cd

Vercel BotID:
  x-is-human header; script often named c.js
  Stale token or IP change: regenerate after re-fetching c.js

Payload bodies are large: usually gzip/br/zstd + content-encoding

export BROWSERSOLVER_API_KEY="your-key"

curl -sS "https://api.browsersolver.com/v1/website/webunlocker" \
  -H "x-api-key: $BROWSERSOLVER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"query":"https://shop.example.com/collections/sneakers"}'

reese84
  Cookie named reese84 and/or x-d-token header
  Script path often static per site (POSTs with ?d=)
  Token JSON includes renewInSec: renew before expiry

reese84 dynamic
  "Pardon Our Interruption" page
  May require a PoW step before the token is accepted

utmvc
  Script like /_Incapsula_Resource?SWJIYLWA=...
  Cookie name is ___utmvc (three underscores)
  Large body: compress (gzip/br/zstd)

Also watch for Incapsula captcha blocks (hCaptcha / Geetest)
  after sensors when trust is still low

export BROWSERSOLVER_API_KEY="your-key"

curl -sS "https://api.browsersolver.com/v1/website/webunlocker" \
  -H "x-api-key: $BROWSERSOLVER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"query":"https://retail.example.com/catalog"}'

Managed Challenge / Turnstile interstitial instead of product HTML
cf-ray response header
__cf_bm / cf_clearance cookies after a successful browser challenge
Bot Fight Mode / Under Attack soft blocks (empty or cookie-gated body)
Soft “200 OK” pages that omit the data you need
cdn-cgi/challenge paths in HTML or network waterfalls

export BROWSERSOLVER_API_KEY="your-key"
URL='https://shop.example.com/collections/all'

# 1) cheapest
curl -sS -o /tmp/out.html -w "%{http_code}" "https://api.browsersolver.com/v1/website/crawler" \
  -H "x-api-key: $BROWSERSOLVER_API_KEY" \
  -H "Content-Type: application/json" -d "{\"query\":\"$URL\"}"

# 2) JS-heavy but reachable: /website/rendering
# 3) challenge wall on allowlisted host: /website/webunlocker

Scripts/hosts referencing px-cdn, client.perimeterx.net, or human security
Cookies: _px*, _pxhd, _pxvid (names vary by integration)
Block pages mentioning Access Denied with PX/HUMAN branding
Often paired with App / bot score on XHR after first paint
First paint can succeed while XHR data calls fail with PX scores

export BROWSERSOLVER_API_KEY="your-key"

curl -sS "https://api.browsersolver.com/v1/website/antibot" \
  -H "x-api-key: $BROWSERSOLVER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"query":"https://store.example.com/"}'

Edge 403 or empty bodies before origin content
JS challenges served from the CDN
Rate and ACL denials on catalog paths
Server / via headers mentioning Fastly
x-fastly-* or Fastly-specific cache headers on blocks

export BROWSERSOLVER_API_KEY="your-key"

curl -sS "https://api.browsersolver.com/v1/website/webunlocker" \
  -H "x-api-key: $BROWSERSOLVER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"query":"https://catalog.example.com/"}'

Website AntiBot    : name vendor / challenge / fingerprint gap
Website Crawler    : static HTML & JSON APIs (cheapest)
Website Rendering  : JS-heavy pages you can already reach
Website WebUnlocker: Cloudflare / DataDome / Akamai / PX / Kasada /
                      Imperva / Fastly on allowlisted domains

Rules:
  - Do not start on WebUnlocker for every URL
  - Re-probe with AntiBot when overnight pass rates collapse
  - Confirm domain support before production unlock volume
  - Failed unlock retries after exhaustion are not billed (see product docs)

export BROWSERSOLVER_API_KEY="your-key"

curl -sS "https://api.browsersolver.com/v1/website/antibot" \
  -H "x-api-key: $BROWSERSOLVER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"query":"https://example.com"}'

const API = "https://api.browsersolver.com/v1"
const KEY = process.env.BROWSERSOLVER_API_KEY!

const res = await fetch(`${API}/website/antibot`, {
  method: "POST",
  headers: { "x-api-key": KEY, "Content-Type": "application/json" },
  body: JSON.stringify({ query: process.argv[2] ?? "https://example.com" }),
})
console.log(await res.json())

import * as cheerio from "cheerio"

const API = "https://api.browsersolver.com/v1"
const KEY = process.env.BROWSERSOLVER_API_KEY!

const res = await fetch(`${API}/website/webunlocker`, {
  method: "POST",
  headers: { "x-api-key": KEY, "Content-Type": "application/json" },
  body: JSON.stringify({ query: "https://shop.example.com/p/sku-1" }),
})
const page = await res.json()
const html = typeof page === "string" ? page : String(page.html ?? page.body ?? page)
const $ = cheerio.load(html)
console.log({ title: $("h1").first().text().trim() })

type Mode = "crawler" | "rendering" | "webunlocker"
const modeByHost = new Map<string, Mode>()

async function fetchHtml(url: string): Promise<unknown> {
  const host = new URL(url).host
  const mode = modeByHost.get(host) ?? "crawler"
  const path = `/website/${mode}`
  try {
    return await websitePost(path, { query: url })
  } catch {
    if (mode === "crawler") {
      modeByHost.set(host, "rendering")
      return fetchHtml(url)
    }
    if (mode === "rendering") {
      modeByHost.set(host, "webunlocker")
      return fetchHtml(url)
    }
    // Optional: website/antibot once, then force webunlocker if allowlisted
    throw new Error(`exhausted modes for ${host}`)
  }
}

// websitePost: copy from “Authenticate BrowserSolver API calls”

/products/website-antibot
/products/website-webunlocker
/products/website-crawler
/products/website-rendering
/products/website-technology  : stack signals alongside AntiBot

1. Does AntiBot return a named vendor, or is the URL unreachable?
2. Did Crawler / Rendering already fail on the same URL?
3. Is the host on the WebUnlocker allowlist (or did support enable it)?
4. Are you reusing a sticky session/IP for the whole job, or rotating mid-flow?
5. Is the failure a challenge wall, empty soft-block, rate limit, or 5xx?

If 2–4 are “no / unsure”, fix routing before blaming unlock.

Symptom                              Likely cause                         Fix
Works locally, fails in prod         rotating proxy / IP drift            sticky session proxies
Intermittent blocks                  dirty cookie jar or mid-flow rotate  one jar + one IP per session
Blocked immediately                  DC IP or UA/sec-ch-ua mismatch       residential + coherent UA
DataDome rejects despite UA          hardcoded full-version-list          do not hardcode client hints
Unlock returns challenge HTML        domain not allowlisted / wrong mode  confirm coverage; re-probe
Crawler empty, browser works         JS-rendered content                  Website Rendering
Rendering challenge wall             vendor bot manager                   Website WebUnlocker
Pass rate drops overnight            vendor changed protection            AntiBot re-probe + mode bump
401 / missing key                    x-api-key absent or wrong            set header from dashboard key

Akamai
  _abck never valid after 3 sensors: TLS/headers/IP, not “more sensors”
  428 mid-session: SEC-CPT; keep hints coherent; honor chlg_duration waits
  429 {"t":…}: SBSD; reuse path/UUID/script, post a fresh payload

DataDome
  t=bv on slider: IP hard-ban; change sticky IP
  Still blocked after interstitial/slider: tags trust may be low

Imperva
  reese84 persists: token/IP issue; renew before renewInSec
  utmvc rejected: cookie must be ___utmvc; compress large bodies

Kasada
  429 loops: stale x-kpsdk-cd POW reused across requests
  x-is-human rejected: re-fetch BotID c.js after IP/UA change

Fastly
  Edge 403 with empty body: unlock / allowlist before origin parsers

Include:
  - target URL + timestamp (UTC)
  - AntiBot JSON (vendor / challenge / mode hints)
  - mode used (crawler | rendering | webunlocker)
  - HTTP status + first 2–4 KB of body (redact PII)
  - response headers: cf-ray, set-cookie names (not values), server
  - whether the domain is allowlisted
  - for local clients: TLS profile id + header-order list from a wire proxy

Skip:
  - full cookie values in public channels
  - API keys
  - unrelated analytics / ad noise

function redact(html: string, max = 4000) {
  return html
    .replace(/datadome=[^;\s]+/gi, "datadome=REDACTED")
    .replace(/_abck=[^;\s]+/gi, "_abck=REDACTED")
    .replace(/reese84=[^;\s]+/gi, "reese84=REDACTED")
    .replace(/___utmvc=[^;\s]+/gi, "___utmvc=REDACTED")
    .replace(/cf_clearance=[^;\s]+/gi, "cf_clearance=REDACTED")
    .slice(0, max)
}

Use Website AntiBot + WebUnlocker when:
  - you want one HTTP API for crawl / render / unlock
  - you do not want to maintain TLS clients + challenge posts
  - the domain is (or can be) allowlisted

Use a local browser-grade client only when:
  - you are researching a new host before requesting allowlist
  - you need interactive debugging of a single challenge flow

Do not expect Website modes to return raw sensor payloads.
Do not post DIY challenge bodies through Website Crawler / Unlock.