From 8e9d21e1110876e0c8f09caaf9b6cd86442ff5d5 Mon Sep 17 00:00:00 2001
From: dprevoznik <58714078+dprevoznik@users.noreply.github.com>
Date: Fri, 4 Sep 2026 16:59:00 +0000
Subject: [PATCH 01/20] Add cookbook for pausing agents on captcha telemetry
events
Shows how to use browser-loop's abort/resume pattern with the
captcha_solve_started / captcha_solve_result / captcha_challenge_result
telemetry events, so an agent stops acting while Kernel's solver is
working and resumes once it reports a terminal outcome.
---
browsers/playwright-computer-use-fallback.mdx | 1 +
.../telemetry/pausing-for-captcha-solves.mdx | 215 ++++++++++++++++++
docs.json | 3 +-
3 files changed, 218 insertions(+), 1 deletion(-)
create mode 100644 browsers/telemetry/pausing-for-captcha-solves.mdx
diff --git a/browsers/playwright-computer-use-fallback.mdx b/browsers/playwright-computer-use-fallback.mdx
index f70688d3..bbc99adb 100644
--- a/browsers/playwright-computer-use-fallback.mdx
+++ b/browsers/playwright-computer-use-fallback.mdx
@@ -506,3 +506,4 @@ As a starting rule, reach for **Per-Tool Limit** first, even if it costs you one
- [Computer Use overview](/integrations/computer-use/overview) — running computer-use models on KERNEL more generally
- [Replays](/browsers/replays) — record a session end to end
- [Stealth mode](/browsers/bot-detection/stealth) — reduce how often a page notices the automation in the first place
+- [Pausing for Captcha Solves](/browsers/telemetry/pausing-for-captcha-solves) — the same abort-and-resume pattern, applied to captcha telemetry instead of toolset handoffs
diff --git a/browsers/telemetry/pausing-for-captcha-solves.mdx b/browsers/telemetry/pausing-for-captcha-solves.mdx
new file mode 100644
index 00000000..68d04d9e
--- /dev/null
+++ b/browsers/telemetry/pausing-for-captcha-solves.mdx
@@ -0,0 +1,215 @@
+---
+title: "Pausing for Captcha Solves"
+description: "Use captcha telemetry events to pause an agent while Kernel's solver works, and resume once it succeeds"
+---
+
+Kernel's [stealth mode](/browsers/bot-detection/stealth) includes an automatic captcha solver that attempts supported challenges — reCAPTCHA, hCaptcha, Cloudflare/Turnstile, and press-and-hold — without any action from your agent. The solver runs in the VM; your agent's job is just to not get in its way while it works.
+
+The naive way to handle that is a system-prompt instruction: [`"If you see a CAPTCHA or similar test, just wait for it to get solved automatically."`](/browsers/bot-detection/stealth#anthropic-computer-use) That works, but it's a soft constraint. The model decides for itself how long to wait and how to tell a solve is still running, from a screenshot alone. It can act too soon — a click during an active solve can land on the widget and interfere with it — or wait long after the page has already moved on.
+
+[Captcha telemetry](/browsers/telemetry/categories#correlate-captcha-tasks-and-challenges) turns that guess into a signal: `captcha_solve_started` fires when the solver accepts a task, and `captcha_solve_result` / `captcha_challenge_result` fire when it reaches a terminal outcome. This cookbook uses [`@onkernel/browser-loop`](https://www.npmjs.com/package/@onkernel/browser-loop) — the same package used in [Playwright with Computer Use Fallback](/browsers/playwright-computer-use-fallback) — to build an automation that streams those events in real time, pauses the agent the instant a solve starts, and resumes it once the result comes back successful.
+
+## The approach
+
+Run the agent through a `browser-loop`-compiled harness as usual. Alongside it, open the session's [telemetry stream](/browsers/telemetry/streaming) and watch for `captcha` events:
+
+1. On `captcha_solve_started`, abort the harness's current turn so the agent stops taking actions.
+2. Wait for the matching terminal event — `captcha_challenge_result` when a `challenge_id` is available, otherwise `captcha_solve_result` on `task_id` — within a bounded timeout.
+3. Re-prompt the harness with what happened, so it picks up with full context instead of starting over.
+
+This mirrors the abort-and-resume pattern from the [Per-Tool Limit](/browsers/playwright-computer-use-fallback#per-tool-limit) recipe: `harness.abort()` interrupts the in-flight run, and a fresh `harness.prompt()` call continues the same conversation once the agent is safe to act again.
+
+## Setup
+
+```bash
+npm install @onkernel/browser-loop @onkernel/sdk @earendil-works/pi-agent-core tsx
+```
+
+
+`browser-loop` depends on an older, exact `@onkernel/sdk` version that predates the [telemetry API](/browsers/telemetry/overview) — installing `@onkernel/sdk` at its current version alongside it can leave two copies in `node_modules`, which produces two incompatible `KERNEL` client types (`attach({ client, browser })` won't type-check, and `client.browsers.telemetry` won't exist on the version `browser-loop` sees). Run `npm ls @onkernel/sdk`; if you see two versions, force a single one with an `overrides` entry in `package.json`:
+
+```json
+{
+ "overrides": {
+ "@onkernel/sdk": "$@onkernel/sdk"
+ }
+}
+```
+
+That pins every copy in the tree to whatever version your own `dependencies` entry resolves to, so `attach()` and `client.browsers.telemetry.stream()` both see the same client type.
+
+
+Every script needs a `KERNEL_API_KEY` and a provider key for whichever model `LOOP_MODEL` points at (`anthropic:claude-sonnet-5` by default, so `ANTHROPIC_API_KEY`):
+
+```bash
+KERNEL_API_KEY=... ANTHROPIC_API_KEY=... npx tsx captcha-solve-wait.ts
+```
+
+## The script
+
+`createCaptchaGate` is the only piece of custom logic. It opens the telemetry stream, tracks whether a captcha episode is currently in flight, and exposes one method the driver awaits after pausing the agent.
+
+A `captcha_solve_started` event marks the start of an episode and calls `onSolveStarted`. Duplicate `captcha_solve_started` events for the same episode (an image-grid challenge can run several solver tasks) don't fire it again — the driver only needs to know once that it should stop. The episode ends on whichever terminal event actually arrives: `captcha_challenge_result` when the task carried a `challenge_id`, or `captcha_solve_result` when it didn't. Per the [telemetry docs](/browsers/telemetry/categories#correlate-captcha-tasks-and-challenges), delivery is best-effort and unordered, so `waitForOutcome` takes a timeout and resolves to `"unknown"` rather than hanging forever if a terminal event never shows up.
+
+The driver compiles a Playwright-toolset harness the same way as the [fallback cookbook](/browsers/playwright-computer-use-fallback), wires the gate's `onSolveStarted` callback to `harness.abort()`, and loops on `stopReason === "aborted"` — re-prompting with the outcome each time a captcha resolves, until the agent finishes for a reason other than a captcha pause.
+
+```ts
+/**
+ * Run an agent against a page that may present a captcha, pausing on
+ * `captcha_solve_started` and resuming once Kernel's solver reports a
+ * terminal outcome for that episode.
+ *
+ * Demo target: the public reCAPTCHA v2 demo at google.com/recaptcha/api2/demo.
+ * Kernel's stealth captcha solver handles it automatically; this script's job
+ * is only to keep the agent from acting on the page while that's in progress.
+ *
+ * Usage:
+ * KERNEL_API_KEY=... ANTHROPIC_API_KEY=... npx tsx captcha-solve-wait.ts
+ *
+ * Env:
+ * KERNEL_API_KEY required, KERNEL browser API key
+ * LOOP_MODEL optional, defaults to anthropic:claude-sonnet-5
+ * (needs the matching provider API key, e.g. ANTHROPIC_API_KEY)
+ */
+import KERNEL from "@onkernel/sdk";
+import { AgentHarness, InMemorySessionRepo, type AgentHarnessEvent } from "@earendil-works/pi-agent-core";
+import { loop } from "@onkernel/browser-loop";
+import { attach, requireLoopEnvApiKeyForModel, type LoopModelRef } from "@onkernel/browser-loop/pi";
+
+type CaptchaOutcome = "solved" | "failure" | "timeout" | "abandoned" | "unknown";
+
+// Tracks one captcha episode at a time: an episode opens on captcha_solve_started
+// and closes on whichever terminal event actually arrives for it.
+function createCaptchaGate(client: KERNEL, sessionId: string, onSolveStarted: () => void) {
+ let taskId: string | null = null;
+ let challengeId: string | null = null;
+ let waiter: ((outcome: CaptchaOutcome) => void) | null = null;
+
+ function settle(outcome: CaptchaOutcome) {
+ taskId = null;
+ challengeId = null;
+ waiter?.(outcome);
+ waiter = null;
+ }
+
+ (async () => {
+ const stream = await client.browsers.telemetry.stream(sessionId);
+ for await (const { event } of stream) {
+ if (event.category !== "captcha") continue;
+
+ if (event.type === "captcha_solve_started") {
+ const isNewEpisode = taskId === null && challengeId === null;
+ taskId = event.data.task_id ?? taskId;
+ challengeId = event.data.challenge_id ?? challengeId;
+ if (isNewEpisode) onSolveStarted();
+ } else if (event.type === "captcha_challenge_result" && event.data.challenge_id === challengeId) {
+ settle(event.data.status);
+ } else if (event.type === "captcha_solve_result" && !challengeId && event.data?.task_id === taskId) {
+ settle(event.data.status === "success" ? "solved" : event.data.status);
+ }
+ }
+ })();
+
+ return {
+ waitForOutcome(timeoutMs: number): Promise {
+ if (taskId === null && challengeId === null) return Promise.resolve("solved");
+ return new Promise((resolve) => {
+ waiter = resolve;
+ setTimeout(() => {
+ if (waiter === resolve) {
+ waiter = null;
+ resolve("unknown");
+ }
+ }, timeoutMs);
+ });
+ },
+ };
+}
+
+const TASK_PROMPT =
+ "Go to google.com/recaptcha/api2/demo, solve the reCAPTCHA challenge, and submit the form.";
+
+const MODEL = (process.env.LOOP_MODEL as LoopModelRef | undefined) ?? "anthropic:claude-sonnet-5";
+
+// How long to wait for a terminal captcha event before giving up and letting
+// the agent reassess the page on its own.
+const CAPTCHA_WAIT_TIMEOUT_MS = 45_000;
+
+async function main(): Promise {
+ const kernelApiKey = process.env.KERNEL_API_KEY;
+ if (!kernelApiKey) throw new Error("KERNEL_API_KEY is required");
+ requireLoopEnvApiKeyForModel(MODEL);
+
+ const client = new KERNEL({ apiKey: kernelApiKey });
+ const browser = await client.browsers.create({
+ stealth: true,
+ telemetry: { browser: { captcha: { enabled: true } } },
+ });
+ const kb = attach({ client, browser });
+
+ try {
+ const session = await new InMemorySessionRepo().create({ id: "captcha-solve-wait" });
+
+ const compiled = kb.compile({ model: MODEL, tools: loop.toolsets.browser() });
+ const harness = new AgentHarness({
+ session,
+ model: compiled.model,
+ models: compiled.models,
+ tools: [...compiled.tools],
+ activeToolNames: compiled.tools.map((tool) => tool.name),
+ systemPrompt: "Use the supplied browser tools to complete the task efficiently.",
+ });
+ compiled.activate(harness);
+
+ let actionTurns = 0;
+ harness.subscribe((event: AgentHarnessEvent) => {
+ if (event.type !== "tool_execution_end") return;
+ actionTurns += 1;
+ console.log(`[agent ${actionTurns}] ${event.toolName} error=${event.isError}`);
+ });
+
+ const gate = createCaptchaGate(client, browser.session_id, () => {
+ console.log("[captcha] solve started, pausing agent");
+ void harness.abort();
+ });
+
+ console.log(`model=${MODEL} prompt=${JSON.stringify(TASK_PROMPT)}`);
+ let final = await harness.prompt(TASK_PROMPT);
+
+ while (final.stopReason === "aborted") {
+ const outcome = await gate.waitForOutcome(CAPTCHA_WAIT_TIMEOUT_MS);
+ console.log(`[captcha] resolved: ${outcome}`);
+ const followUp =
+ outcome === "solved"
+ ? "Kernel's captcha solver resolved the challenge successfully. Continue the task."
+ : `The captcha challenge ended as "${outcome}". Take a screenshot to check the current ` +
+ "page state, then retry the challenge yourself or report the blocker.";
+ final = await harness.prompt(followUp);
+ }
+
+ console.log(`final stopReason: ${final.stopReason}`);
+ for (const block of final.content) {
+ if (block.type === "text") console.log(block.text);
+ }
+ } finally {
+ await kb.dispose();
+ await client.browsers.deleteByID(browser.session_id);
+ }
+}
+
+void main();
+```
+
+## Notes
+
+- **The gate only matters once a run is in flight.** If a captcha appears and clears before the first `harness.prompt()` call starts, there's nothing to pause — which is fine, since there was nothing for the agent to interfere with either.
+- **`harness.abort()` interrupts the current turn, not just the next tool call.** The agent's in-flight action still completes on the browser side; the harness stops before starting another one. This is the same mechanism the [fallback cookbook](/browsers/playwright-computer-use-fallback#per-tool-limit) uses to hand off between toolsets.
+- **Multiple captchas in one run just loop.** Each terminal event resets the gate's state, so a second `captcha_solve_started` later in the task pauses and resumes the same way.
+- **A bounded wait beats an indefinite one.** Per the [telemetry docs](/browsers/telemetry/categories#correlate-captcha-tasks-and-challenges), event delivery isn't guaranteed or ordered — `waitForOutcome`'s timeout is what keeps a dropped event from stalling the run forever.
+
+## Next steps
+
+- [Telemetry Categories](/browsers/telemetry/categories#correlate-captcha-tasks-and-challenges) — the full captcha event schema, `task_id` / `challenge_id` correlation, and outcome meanings
+- [Stream Telemetry](/browsers/telemetry/streaming) — resuming a dropped stream, filtering by category
+- [Stealth mode](/browsers/bot-detection/stealth) — what the automatic captcha solver covers
+- [Playwright with Computer Use Fallback](/browsers/playwright-computer-use-fallback) — the abort-and-resume pattern this cookbook reuses
diff --git a/docs.json b/docs.json
index d89c7161..398b28ac 100644
--- a/docs.json
+++ b/docs.json
@@ -168,7 +168,8 @@
"pages": [
"browsers/telemetry/overview",
"browsers/telemetry/categories",
- "browsers/telemetry/streaming"
+ "browsers/telemetry/streaming",
+ "browsers/telemetry/pausing-for-captcha-solves"
]
},
"browsers/pools"
From 3712d4068ff6d3e3d37ee8bc290507a5a505ef76 Mon Sep 17 00:00:00 2001
From: dprevoznik <58714078+dprevoznik@users.noreply.github.com>
Date: Fri, 4 Sep 2026 17:11:12 +0000
Subject: [PATCH 02/20] Fix captcha gate race conditions and outcome semantics
Buffers a terminal telemetry event that arrives before waitForOutcome
is called, clears tracked IDs on timeout so the next captcha still
triggers a pause, stops a same-challenge task from overwriting the
tracked task_id, and distinguishes a task-level success from a
challenge actually clearing.
---
.../telemetry/pausing-for-captcha-solves.mdx | 84 +++++++++++++------
1 file changed, 59 insertions(+), 25 deletions(-)
diff --git a/browsers/telemetry/pausing-for-captcha-solves.mdx b/browsers/telemetry/pausing-for-captcha-solves.mdx
index 68d04d9e..4615d0b4 100644
--- a/browsers/telemetry/pausing-for-captcha-solves.mdx
+++ b/browsers/telemetry/pausing-for-captcha-solves.mdx
@@ -7,7 +7,7 @@ Kernel's [stealth mode](/browsers/bot-detection/stealth) includes an automatic c
The naive way to handle that is a system-prompt instruction: [`"If you see a CAPTCHA or similar test, just wait for it to get solved automatically."`](/browsers/bot-detection/stealth#anthropic-computer-use) That works, but it's a soft constraint. The model decides for itself how long to wait and how to tell a solve is still running, from a screenshot alone. It can act too soon — a click during an active solve can land on the widget and interfere with it — or wait long after the page has already moved on.
-[Captcha telemetry](/browsers/telemetry/categories#correlate-captcha-tasks-and-challenges) turns that guess into a signal: `captcha_solve_started` fires when the solver accepts a task, and `captcha_solve_result` / `captcha_challenge_result` fire when it reaches a terminal outcome. This cookbook uses [`@onkernel/browser-loop`](https://www.npmjs.com/package/@onkernel/browser-loop) — the same package used in [Playwright with Computer Use Fallback](/browsers/playwright-computer-use-fallback) — to build an automation that streams those events in real time, pauses the agent the instant a solve starts, and resumes it once the result comes back successful.
+[Captcha telemetry](/browsers/telemetry/categories#correlate-captcha-tasks-and-challenges) turns that guess into a signal: `captcha_solve_started` fires when the solver accepts a task, and `captcha_solve_result` / `captcha_challenge_result` fire when it reaches a terminal outcome. This cookbook uses [`@onkernel/browser-loop`](https://www.npmjs.com/package/@onkernel/browser-loop) — the same package used in [Playwright with Computer Use Fallback](/browsers/playwright-computer-use-fallback) — to build an automation that streams those events in real time, pauses the agent the instant a solve starts, and resumes it once telemetry reports a terminal outcome for that solve.
## The approach
@@ -49,7 +49,9 @@ KERNEL_API_KEY=... ANTHROPIC_API_KEY=... npx tsx captcha-solve-wait.ts
`createCaptchaGate` is the only piece of custom logic. It opens the telemetry stream, tracks whether a captcha episode is currently in flight, and exposes one method the driver awaits after pausing the agent.
-A `captcha_solve_started` event marks the start of an episode and calls `onSolveStarted`. Duplicate `captcha_solve_started` events for the same episode (an image-grid challenge can run several solver tasks) don't fire it again — the driver only needs to know once that it should stop. The episode ends on whichever terminal event actually arrives: `captcha_challenge_result` when the task carried a `challenge_id`, or `captcha_solve_result` when it didn't. Per the [telemetry docs](/browsers/telemetry/categories#correlate-captcha-tasks-and-challenges), delivery is best-effort and unordered, so `waitForOutcome` takes a timeout and resolves to `"unknown"` rather than hanging forever if a terminal event never shows up.
+A `captcha_solve_started` event marks the start of an episode and calls `onSolveStarted`. Further `captcha_solve_started` events for the same episode — an image-grid challenge can run several solver tasks under one `challenge_id` — don't fire it again or overwrite the tracked IDs; the driver only needs to know once that it should stop, and losing the original `task_id` would break the pairing described in the [telemetry docs](/browsers/telemetry/categories#correlate-captcha-tasks-and-challenges). The episode ends on whichever terminal event actually arrives: `captcha_challenge_result` when a `challenge_id` was seen, or `captcha_solve_result` on `task_id` when it wasn't. Those two terminal events mean different things — a `captcha_challenge_result` of `solved` means the visible challenge actually cleared, while a `captcha_solve_result` of `success` only means the solver returned a usable answer for that one task, so the gate keeps them as distinct outcomes (`challenge_solved` vs. `task_success`) instead of collapsing both into "solved."
+
+Because delivery is best-effort and events can arrive out of order, a terminal event can land before the driver even calls `waitForOutcome` (right after `harness.abort()` returns). The gate buffers that outcome instead of dropping it, and `waitForOutcome` checks the buffer before it starts waiting. If nothing ever arrives, the timeout resolves to `"unknown"` and clears the tracked IDs, so a later captcha still triggers a fresh pause instead of being mistaken for the one that timed out.
The driver compiles a Playwright-toolset harness the same way as the [fallback cookbook](/browsers/playwright-computer-use-fallback), wires the gate's `onSolveStarted` callback to `harness.abort()`, and loops on `stopReason === "aborted"` — re-prompting with the outcome each time a captcha resolves, until the agent finishes for a reason other than a captcha pause.
@@ -76,20 +78,32 @@ import { AgentHarness, InMemorySessionRepo, type AgentHarnessEvent } from "@eare
import { loop } from "@onkernel/browser-loop";
import { attach, requireLoopEnvApiKeyForModel, type LoopModelRef } from "@onkernel/browser-loop/pi";
-type CaptchaOutcome = "solved" | "failure" | "timeout" | "abandoned" | "unknown";
+type CaptchaOutcome = "challenge_solved" | "task_success" | "failure" | "timeout" | "abandoned" | "unknown";
-// Tracks one captcha episode at a time: an episode opens on captcha_solve_started
-// and closes on whichever terminal event actually arrives for it.
+// Tracks one open captcha episode at a time: it opens on captcha_solve_started
+// and closes on whichever terminal event actually arrives for it. A second,
+// unrelated episode starting while one is already open is ignored rather than
+// overwriting the tracked IDs -- overlapping captcha widgets are rare, and the
+// telemetry docs note that even Kernel's own event model can't always
+// attribute a result when multiple same-provider challenges are open at once.
function createCaptchaGate(client: KERNEL, sessionId: string, onSolveStarted: () => void) {
let taskId: string | null = null;
let challengeId: string | null = null;
+ let bufferedOutcome: CaptchaOutcome | null = null;
let waiter: ((outcome: CaptchaOutcome) => void) | null = null;
- function settle(outcome: CaptchaOutcome) {
+ function finish(outcome: CaptchaOutcome) {
taskId = null;
challengeId = null;
- waiter?.(outcome);
- waiter = null;
+ if (waiter) {
+ const resolve = waiter;
+ waiter = null;
+ resolve(outcome);
+ } else {
+ // waitForOutcome hasn't been called yet -- buffer the outcome so a
+ // fast terminal event isn't dropped by the race with harness.abort().
+ bufferedOutcome = outcome;
+ }
}
(async () => {
@@ -98,28 +112,43 @@ function createCaptchaGate(client: KERNEL, sessionId: string, onSolveStarted: ()
if (event.category !== "captcha") continue;
if (event.type === "captcha_solve_started") {
- const isNewEpisode = taskId === null && challengeId === null;
- taskId = event.data.task_id ?? taskId;
- challengeId = event.data.challenge_id ?? challengeId;
- if (isNewEpisode) onSolveStarted();
+ const isSameEpisode =
+ (event.data.challenge_id !== undefined && event.data.challenge_id === challengeId) ||
+ (event.data.task_id !== undefined && event.data.task_id === taskId);
+ if (isSameEpisode) continue;
+ const isIdle = taskId === null && challengeId === null;
+ if (isIdle) {
+ onSolveStarted();
+ taskId = event.data.task_id ?? null;
+ challengeId = event.data.challenge_id ?? null;
+ }
} else if (event.type === "captcha_challenge_result" && event.data.challenge_id === challengeId) {
- settle(event.data.status);
+ finish(event.data.status === "solved" ? "challenge_solved" : event.data.status);
} else if (event.type === "captcha_solve_result" && !challengeId && event.data?.task_id === taskId) {
- settle(event.data.status === "success" ? "solved" : event.data.status);
+ finish(event.data.status === "success" ? "task_success" : event.data.status);
}
}
})();
return {
waitForOutcome(timeoutMs: number): Promise {
- if (taskId === null && challengeId === null) return Promise.resolve("solved");
+ if (bufferedOutcome !== null) {
+ const outcome = bufferedOutcome;
+ bufferedOutcome = null;
+ return Promise.resolve(outcome);
+ }
+ if (taskId === null && challengeId === null) return Promise.resolve("challenge_solved");
return new Promise((resolve) => {
waiter = resolve;
setTimeout(() => {
- if (waiter === resolve) {
- waiter = null;
- resolve("unknown");
- }
+ if (waiter !== resolve) return;
+ waiter = null;
+ // Give up waiting -- clear the tracked IDs so a later
+ // captcha_solve_started still triggers a fresh pause instead
+ // of being mistaken for the episode that just timed out.
+ taskId = null;
+ challengeId = null;
+ resolve("unknown");
}, timeoutMs);
});
},
@@ -180,10 +209,14 @@ async function main(): Promise {
const outcome = await gate.waitForOutcome(CAPTCHA_WAIT_TIMEOUT_MS);
console.log(`[captcha] resolved: ${outcome}`);
const followUp =
- outcome === "solved"
- ? "Kernel's captcha solver resolved the challenge successfully. Continue the task."
- : `The captcha challenge ended as "${outcome}". Take a screenshot to check the current ` +
- "page state, then retry the challenge yourself or report the blocker.";
+ outcome === "challenge_solved"
+ ? "Kernel's captcha solver reported that the visible challenge cleared. Continue the task."
+ : outcome === "task_success"
+ ? "Kernel's captcha solver returned a usable solution, but there's no visible-challenge " +
+ "confirmation for this page. Take a screenshot to confirm the challenge actually " +
+ "cleared, then continue or retry."
+ : `The captcha challenge ended as "${outcome}". Take a screenshot to check the current ` +
+ "page state, then retry the challenge yourself or report the blocker.";
final = await harness.prompt(followUp);
}
@@ -204,8 +237,9 @@ void main();
- **The gate only matters once a run is in flight.** If a captcha appears and clears before the first `harness.prompt()` call starts, there's nothing to pause — which is fine, since there was nothing for the agent to interfere with either.
- **`harness.abort()` interrupts the current turn, not just the next tool call.** The agent's in-flight action still completes on the browser side; the harness stops before starting another one. This is the same mechanism the [fallback cookbook](/browsers/playwright-computer-use-fallback#per-tool-limit) uses to hand off between toolsets.
-- **Multiple captchas in one run just loop.** Each terminal event resets the gate's state, so a second `captcha_solve_started` later in the task pauses and resumes the same way.
-- **A bounded wait beats an indefinite one.** Per the [telemetry docs](/browsers/telemetry/categories#correlate-captcha-tasks-and-challenges), event delivery isn't guaranteed or ordered — `waitForOutcome`'s timeout is what keeps a dropped event from stalling the run forever.
+- **Sequential captchas in one run just loop.** Each terminal event (or timeout) resets the gate's state, so a second `captcha_solve_started` later in the task pauses and resumes the same way.
+- **Overlapping captchas aren't split apart.** The gate tracks one open episode at a time; a second, unrelated `captcha_solve_started` while the first is still open is ignored rather than risking a mismatched pairing. Two genuinely concurrent visible challenges are rare, and the [telemetry docs](/browsers/telemetry/categories#correlate-captcha-tasks-and-challenges) note that Kernel's own event model can't always attribute a result in that case either.
+- **A bounded wait beats an indefinite one.** Event delivery isn't guaranteed or ordered, so `waitForOutcome`'s timeout is what keeps a dropped event from stalling the run forever — and it clears the gate's state on the way out so the next captcha isn't mistaken for the one that timed out.
## Next steps
From 9df87e2109a1876400fc8029fb974f6809eb38ab Mon Sep 17 00:00:00 2001
From: dprevoznik <58714078+dprevoznik@users.noreply.github.com>
Date: Fri, 4 Sep 2026 17:23:15 +0000
Subject: [PATCH 03/20] Discard stale buffered captcha outcome when a new
episode opens
If an episode's outcome buffers before waitForOutcome is called and a
new episode starts before the driver gets back to it, the buffer is
now cleared so the driver waits on the current episode instead of
resuming on the stale one. Also breaks the dense gate-behavior
paragraph into bullets for scannability.
---
browsers/telemetry/pausing-for-captcha-solves.mdx | 11 ++++++++++-
1 file changed, 10 insertions(+), 1 deletion(-)
diff --git a/browsers/telemetry/pausing-for-captcha-solves.mdx b/browsers/telemetry/pausing-for-captcha-solves.mdx
index 4615d0b4..68460cc7 100644
--- a/browsers/telemetry/pausing-for-captcha-solves.mdx
+++ b/browsers/telemetry/pausing-for-captcha-solves.mdx
@@ -49,7 +49,11 @@ KERNEL_API_KEY=... ANTHROPIC_API_KEY=... npx tsx captcha-solve-wait.ts
`createCaptchaGate` is the only piece of custom logic. It opens the telemetry stream, tracks whether a captcha episode is currently in flight, and exposes one method the driver awaits after pausing the agent.
-A `captcha_solve_started` event marks the start of an episode and calls `onSolveStarted`. Further `captcha_solve_started` events for the same episode — an image-grid challenge can run several solver tasks under one `challenge_id` — don't fire it again or overwrite the tracked IDs; the driver only needs to know once that it should stop, and losing the original `task_id` would break the pairing described in the [telemetry docs](/browsers/telemetry/categories#correlate-captcha-tasks-and-challenges). The episode ends on whichever terminal event actually arrives: `captcha_challenge_result` when a `challenge_id` was seen, or `captcha_solve_result` on `task_id` when it wasn't. Those two terminal events mean different things — a `captcha_challenge_result` of `solved` means the visible challenge actually cleared, while a `captcha_solve_result` of `success` only means the solver returned a usable answer for that one task, so the gate keeps them as distinct outcomes (`challenge_solved` vs. `task_success`) instead of collapsing both into "solved."
+A `captcha_solve_started` event marks the start of an episode and calls `onSolveStarted`. From there:
+
+- **Further `captcha_solve_started` events for the same episode don't fire it again or overwrite the tracked IDs.** An image-grid challenge can run several solver tasks under one `challenge_id`; the driver only needs to know once that it should stop, and losing the original `task_id` would break the pairing described in the [telemetry docs](/browsers/telemetry/categories#correlate-captcha-tasks-and-challenges).
+- **The episode ends on whichever terminal event actually arrives** — `captcha_challenge_result` when a `challenge_id` was seen, or `captcha_solve_result` on `task_id` when it wasn't.
+- **Those two terminal events mean different things**, so the gate keeps them as distinct outcomes instead of collapsing both into "solved": a `captcha_challenge_result` of `solved` means the visible challenge actually cleared, while a `captcha_solve_result` of `success` only means the solver returned a usable answer for that one task (`challenge_solved` vs. `task_success`).
Because delivery is best-effort and events can arrive out of order, a terminal event can land before the driver even calls `waitForOutcome` (right after `harness.abort()` returns). The gate buffers that outcome instead of dropping it, and `waitForOutcome` checks the buffer before it starts waiting. If nothing ever arrives, the timeout resolves to `"unknown"` and clears the tracked IDs, so a later captcha still triggers a fresh pause instead of being mistaken for the one that timed out.
@@ -118,6 +122,11 @@ function createCaptchaGate(client: KERNEL, sessionId: string, onSolveStarted: ()
if (isSameEpisode) continue;
const isIdle = taskId === null && challengeId === null;
if (isIdle) {
+ // Discard any outcome still sitting in the buffer: it's from an
+ // episode that finished before the driver got back to
+ // waitForOutcome, and a new episode has since opened, so it no
+ // longer describes what the driver is now waiting on.
+ bufferedOutcome = null;
onSolveStarted();
taskId = event.data.task_id ?? null;
challengeId = event.data.challenge_id ?? null;
From d7f6e03c13f62da71ca42e2b3c1193c6f6bcf4db Mon Sep 17 00:00:00 2001
From: dprevoznik <58714078+dprevoznik@users.noreply.github.com>
Date: Fri, 4 Sep 2026 17:30:32 +0000
Subject: [PATCH 04/20] Track buffered outcome identity to survive late
duplicate starts
Delivery is unordered, so a captcha_solve_started for an episode can
arrive after that episode's own terminal result already buffered.
The gate now remembers which task_id/challenge_id a buffered outcome
belongs to, so a late duplicate for that same episode is recognized
and left alone instead of being mistaken for a new episode (which
previously discarded the buffer and re-triggered a spurious pause).
---
.../telemetry/pausing-for-captcha-solves.mdx | 47 ++++++++++++++-----
1 file changed, 36 insertions(+), 11 deletions(-)
diff --git a/browsers/telemetry/pausing-for-captcha-solves.mdx b/browsers/telemetry/pausing-for-captcha-solves.mdx
index 68460cc7..0f2a96eb 100644
--- a/browsers/telemetry/pausing-for-captcha-solves.mdx
+++ b/browsers/telemetry/pausing-for-captcha-solves.mdx
@@ -93,20 +93,45 @@ type CaptchaOutcome = "challenge_solved" | "task_success" | "failure" | "timeout
function createCaptchaGate(client: KERNEL, sessionId: string, onSolveStarted: () => void) {
let taskId: string | null = null;
let challengeId: string | null = null;
- let bufferedOutcome: CaptchaOutcome | null = null;
let waiter: ((outcome: CaptchaOutcome) => void) | null = null;
+ // A finished episode's outcome, kept alongside the IDs it belongs to. Delivery
+ // is unordered, so a captcha_solve_started for this same episode can still
+ // arrive after its result; matching on these IDs tells that late duplicate
+ // apart from a genuinely new episode.
+ let bufferedOutcome: CaptchaOutcome | null = null;
+ let bufferedTaskId: string | null = null;
+ let bufferedChallengeId: string | null = null;
+
+ function matchesBuffer(eventTaskId?: string, eventChallengeId?: string): boolean {
+ if (bufferedOutcome === null) return false;
+ return (
+ (eventChallengeId !== undefined && eventChallengeId === bufferedChallengeId) ||
+ (eventTaskId !== undefined && eventTaskId === bufferedTaskId)
+ );
+ }
+
+ function clearBuffer() {
+ bufferedOutcome = null;
+ bufferedTaskId = null;
+ bufferedChallengeId = null;
+ }
+
function finish(outcome: CaptchaOutcome) {
- taskId = null;
- challengeId = null;
if (waiter) {
const resolve = waiter;
waiter = null;
+ taskId = null;
+ challengeId = null;
resolve(outcome);
} else {
// waitForOutcome hasn't been called yet -- buffer the outcome so a
// fast terminal event isn't dropped by the race with harness.abort().
bufferedOutcome = outcome;
+ bufferedTaskId = taskId;
+ bufferedChallengeId = challengeId;
+ taskId = null;
+ challengeId = null;
}
}
@@ -116,17 +141,17 @@ function createCaptchaGate(client: KERNEL, sessionId: string, onSolveStarted: ()
if (event.category !== "captcha") continue;
if (event.type === "captcha_solve_started") {
- const isSameEpisode =
+ const isSameOpenEpisode =
(event.data.challenge_id !== undefined && event.data.challenge_id === challengeId) ||
(event.data.task_id !== undefined && event.data.task_id === taskId);
- if (isSameEpisode) continue;
+ if (isSameOpenEpisode) continue;
+ if (matchesBuffer(event.data.task_id, event.data.challenge_id)) continue;
const isIdle = taskId === null && challengeId === null;
if (isIdle) {
- // Discard any outcome still sitting in the buffer: it's from an
- // episode that finished before the driver got back to
- // waitForOutcome, and a new episode has since opened, so it no
- // longer describes what the driver is now waiting on.
- bufferedOutcome = null;
+ // A genuinely new episode makes any buffered outcome stale: it
+ // belongs to an earlier episode the driver hasn't collected yet,
+ // and this new one is what waitForOutcome should wait on instead.
+ clearBuffer();
onSolveStarted();
taskId = event.data.task_id ?? null;
challengeId = event.data.challenge_id ?? null;
@@ -143,7 +168,7 @@ function createCaptchaGate(client: KERNEL, sessionId: string, onSolveStarted: ()
waitForOutcome(timeoutMs: number): Promise {
if (bufferedOutcome !== null) {
const outcome = bufferedOutcome;
- bufferedOutcome = null;
+ clearBuffer();
return Promise.resolve(outcome);
}
if (taskId === null && challengeId === null) return Promise.resolve("challenge_solved");
From 0ed73cca92b9950da645d469d4dcd82766b1c137 Mon Sep 17 00:00:00 2001
From: dprevoznik <58714078+dprevoznik@users.noreply.github.com>
Date: Fri, 4 Sep 2026 18:02:53 +0000
Subject: [PATCH 05/20] Switch cookbook demo to duckduckgo -> 2captcha
Turnstile demo
Starts on duckduckgo.com, navigates to 2captcha.com's public
Cloudflare Turnstile demo, and has the agent summarize the page's
explanation of how Turnstile is solved once the challenge clears.
---
browsers/telemetry/pausing-for-captcha-solves.mdx | 12 ++++++++----
1 file changed, 8 insertions(+), 4 deletions(-)
diff --git a/browsers/telemetry/pausing-for-captcha-solves.mdx b/browsers/telemetry/pausing-for-captcha-solves.mdx
index 0f2a96eb..97eddda4 100644
--- a/browsers/telemetry/pausing-for-captcha-solves.mdx
+++ b/browsers/telemetry/pausing-for-captcha-solves.mdx
@@ -65,9 +65,11 @@ The driver compiles a Playwright-toolset harness the same way as the [fallback c
* `captcha_solve_started` and resuming once Kernel's solver reports a
* terminal outcome for that episode.
*
- * Demo target: the public reCAPTCHA v2 demo at google.com/recaptcha/api2/demo.
- * Kernel's stealth captcha solver handles it automatically; this script's job
- * is only to keep the agent from acting on the page while that's in progress.
+ * Demo target: start on duckduckgo.com, then navigate to 2captcha.com's public
+ * Cloudflare Turnstile demo. Kernel's stealth captcha solver handles the
+ * Turnstile challenge automatically; this script's job is only to keep the
+ * agent from acting on the page while that's in progress, then have it
+ * summarize the page once the challenge clears.
*
* Usage:
* KERNEL_API_KEY=... ANTHROPIC_API_KEY=... npx tsx captcha-solve-wait.ts
@@ -190,7 +192,9 @@ function createCaptchaGate(client: KERNEL, sessionId: string, onSolveStarted: ()
}
const TASK_PROMPT =
- "Go to google.com/recaptcha/api2/demo, solve the reCAPTCHA challenge, and submit the form.";
+ "Start at duckduckgo.com, then navigate to 2captcha.com/demo/cloudflare-turnstile. Once the " +
+ "Turnstile challenge finishes solving, summarize the page's content, explaining how Turnstile " +
+ "challenges get solved.";
const MODEL = (process.env.LOOP_MODEL as LoopModelRef | undefined) ?? "anthropic:claude-sonnet-5";
From 911955e8e708f776a63075e3db86f8339a864aec Mon Sep 17 00:00:00 2001
From: dprevoznik <58714078+dprevoznik@users.noreply.github.com>
Date: Fri, 4 Sep 2026 18:15:26 +0000
Subject: [PATCH 06/20] Make cookbook self-contained and use a placeholder task
prompt
Removes inline references to the Playwright/computer-use fallback
cookbook from the body text (kept only in Next Steps), and replaces
the concrete duckduckgo/2captcha demo task with a placeholder the
reader fills in with their own task.
---
.../telemetry/pausing-for-captcha-solves.mdx | 25 ++++++++-----------
1 file changed, 11 insertions(+), 14 deletions(-)
diff --git a/browsers/telemetry/pausing-for-captcha-solves.mdx b/browsers/telemetry/pausing-for-captcha-solves.mdx
index 97eddda4..401425da 100644
--- a/browsers/telemetry/pausing-for-captcha-solves.mdx
+++ b/browsers/telemetry/pausing-for-captcha-solves.mdx
@@ -7,17 +7,17 @@ Kernel's [stealth mode](/browsers/bot-detection/stealth) includes an automatic c
The naive way to handle that is a system-prompt instruction: [`"If you see a CAPTCHA or similar test, just wait for it to get solved automatically."`](/browsers/bot-detection/stealth#anthropic-computer-use) That works, but it's a soft constraint. The model decides for itself how long to wait and how to tell a solve is still running, from a screenshot alone. It can act too soon — a click during an active solve can land on the widget and interfere with it — or wait long after the page has already moved on.
-[Captcha telemetry](/browsers/telemetry/categories#correlate-captcha-tasks-and-challenges) turns that guess into a signal: `captcha_solve_started` fires when the solver accepts a task, and `captcha_solve_result` / `captcha_challenge_result` fire when it reaches a terminal outcome. This cookbook uses [`@onkernel/browser-loop`](https://www.npmjs.com/package/@onkernel/browser-loop) — the same package used in [Playwright with Computer Use Fallback](/browsers/playwright-computer-use-fallback) — to build an automation that streams those events in real time, pauses the agent the instant a solve starts, and resumes it once telemetry reports a terminal outcome for that solve.
+[Captcha telemetry](/browsers/telemetry/categories#correlate-captcha-tasks-and-challenges) turns that guess into a signal: `captcha_solve_started` fires when the solver accepts a task, and `captcha_solve_result` / `captcha_challenge_result` fire when it reaches a terminal outcome. This cookbook uses [`@onkernel/browser-loop`](https://www.npmjs.com/package/@onkernel/browser-loop) to build an automation that streams those events in real time, pauses the agent the instant a solve starts, and resumes it once telemetry reports a terminal outcome for that solve.
## The approach
-Run the agent through a `browser-loop`-compiled harness as usual. Alongside it, open the session's [telemetry stream](/browsers/telemetry/streaming) and watch for `captcha` events:
+Run the agent through a `browser-loop`-compiled harness. Alongside it, open the session's [telemetry stream](/browsers/telemetry/streaming) and watch for `captcha` events:
1. On `captcha_solve_started`, abort the harness's current turn so the agent stops taking actions.
2. Wait for the matching terminal event — `captcha_challenge_result` when a `challenge_id` is available, otherwise `captcha_solve_result` on `task_id` — within a bounded timeout.
3. Re-prompt the harness with what happened, so it picks up with full context instead of starting over.
-This mirrors the abort-and-resume pattern from the [Per-Tool Limit](/browsers/playwright-computer-use-fallback#per-tool-limit) recipe: `harness.abort()` interrupts the in-flight run, and a fresh `harness.prompt()` call continues the same conversation once the agent is safe to act again.
+`harness.abort()` interrupts the in-flight run, and a fresh `harness.prompt()` call continues the same conversation once the agent is safe to act again.
## Setup
@@ -57,7 +57,7 @@ A `captcha_solve_started` event marks the start of an episode and calls `onSolve
Because delivery is best-effort and events can arrive out of order, a terminal event can land before the driver even calls `waitForOutcome` (right after `harness.abort()` returns). The gate buffers that outcome instead of dropping it, and `waitForOutcome` checks the buffer before it starts waiting. If nothing ever arrives, the timeout resolves to `"unknown"` and clears the tracked IDs, so a later captcha still triggers a fresh pause instead of being mistaken for the one that timed out.
-The driver compiles a Playwright-toolset harness the same way as the [fallback cookbook](/browsers/playwright-computer-use-fallback), wires the gate's `onSolveStarted` callback to `harness.abort()`, and loops on `stopReason === "aborted"` — re-prompting with the outcome each time a captcha resolves, until the agent finishes for a reason other than a captcha pause.
+The driver compiles a Playwright-toolset harness, wires the gate's `onSolveStarted` callback to `harness.abort()`, and loops on `stopReason === "aborted"` — re-prompting with the outcome each time a captcha resolves, until the agent finishes for a reason other than a captcha pause.
```ts
/**
@@ -65,11 +65,11 @@ The driver compiles a Playwright-toolset harness the same way as the [fallback c
* `captcha_solve_started` and resuming once Kernel's solver reports a
* terminal outcome for that episode.
*
- * Demo target: start on duckduckgo.com, then navigate to 2captcha.com's public
- * Cloudflare Turnstile demo. Kernel's stealth captcha solver handles the
- * Turnstile challenge automatically; this script's job is only to keep the
- * agent from acting on the page while that's in progress, then have it
- * summarize the page once the challenge clears.
+ * Set TASK_PROMPT below to whatever task the agent should run -- anything
+ * that might land it on a captcha-protected page. Kernel's stealth captcha
+ * solver handles supported challenge types automatically; this script's job
+ * is only to keep the agent from acting on the page while a solve is in
+ * progress.
*
* Usage:
* KERNEL_API_KEY=... ANTHROPIC_API_KEY=... npx tsx captcha-solve-wait.ts
@@ -191,10 +191,7 @@ function createCaptchaGate(client: KERNEL, sessionId: string, onSolveStarted: ()
};
}
-const TASK_PROMPT =
- "Start at duckduckgo.com, then navigate to 2captcha.com/demo/cloudflare-turnstile. Once the " +
- "Turnstile challenge finishes solving, summarize the page's content, explaining how Turnstile " +
- "challenges get solved.";
+const TASK_PROMPT = "[ENTER TASK FOR AGENT THAT MAY NEED CAPTCHA SOLVING]";
const MODEL = (process.env.LOOP_MODEL as LoopModelRef | undefined) ?? "anthropic:claude-sonnet-5";
@@ -274,7 +271,7 @@ void main();
## Notes
- **The gate only matters once a run is in flight.** If a captcha appears and clears before the first `harness.prompt()` call starts, there's nothing to pause — which is fine, since there was nothing for the agent to interfere with either.
-- **`harness.abort()` interrupts the current turn, not just the next tool call.** The agent's in-flight action still completes on the browser side; the harness stops before starting another one. This is the same mechanism the [fallback cookbook](/browsers/playwright-computer-use-fallback#per-tool-limit) uses to hand off between toolsets.
+- **`harness.abort()` interrupts the current turn, not just the next tool call.** The agent's in-flight action still completes on the browser side; the harness stops before starting another one.
- **Sequential captchas in one run just loop.** Each terminal event (or timeout) resets the gate's state, so a second `captcha_solve_started` later in the task pauses and resumes the same way.
- **Overlapping captchas aren't split apart.** The gate tracks one open episode at a time; a second, unrelated `captcha_solve_started` while the first is still open is ignored rather than risking a mismatched pairing. Two genuinely concurrent visible challenges are rare, and the [telemetry docs](/browsers/telemetry/categories#correlate-captcha-tasks-and-challenges) note that Kernel's own event model can't always attribute a result in that case either.
- **A bounded wait beats an indefinite one.** Event delivery isn't guaranteed or ordered, so `waitForOutcome`'s timeout is what keeps a dropped event from stalling the run forever — and it clears the gate's state on the way out so the next captcha isn't mistaken for the one that timed out.
From 924f68ab6cc7038a0f26fa0cf9795cac82e03c6a Mon Sep 17 00:00:00 2001
From: dprevoznik <58714078+dprevoznik@users.noreply.github.com>
Date: Fri, 4 Sep 2026 18:21:21 +0000
Subject: [PATCH 07/20] Add captcha event summary table to the introduction
Quick-reference table of the three captcha events and their possible
terminal outcomes, right after they're introduced in prose.
---
browsers/telemetry/pausing-for-captcha-solves.mdx | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/browsers/telemetry/pausing-for-captcha-solves.mdx b/browsers/telemetry/pausing-for-captcha-solves.mdx
index 401425da..b113114b 100644
--- a/browsers/telemetry/pausing-for-captcha-solves.mdx
+++ b/browsers/telemetry/pausing-for-captcha-solves.mdx
@@ -9,6 +9,12 @@ The naive way to handle that is a system-prompt instruction: [`"If you see a CAP
[Captcha telemetry](/browsers/telemetry/categories#correlate-captcha-tasks-and-challenges) turns that guess into a signal: `captcha_solve_started` fires when the solver accepts a task, and `captcha_solve_result` / `captcha_challenge_result` fire when it reaches a terminal outcome. This cookbook uses [`@onkernel/browser-loop`](https://www.npmjs.com/package/@onkernel/browser-loop) to build an automation that streams those events in real time, pauses the agent the instant a solve starts, and resumes it once telemetry reports a terminal outcome for that solve.
+| Event | Fires when | Possible outcomes |
+| --- | --- | --- |
+| `captcha_solve_started` | The solver accepts a task. Doesn't mean a solve is currently in flight. | — |
+| `captcha_solve_result` | A solver task reaches a terminal state. A `success` means the solver returned a usable answer, not that the visible challenge cleared. | `success`, `failure`, `timeout`, `abandoned` |
+| `captcha_challenge_result` | The visible challenge itself reaches its overall terminal state. Only emitted for challenge types Kernel can track as a widget. | `solved`, `failure`, `timeout`, `abandoned` |
+
## The approach
Run the agent through a `browser-loop`-compiled harness. Alongside it, open the session's [telemetry stream](/browsers/telemetry/streaming) and watch for `captcha` events:
From 738ec65eff6bc81e56843364c04ff6afb26f62e5 Mon Sep 17 00:00:00 2001
From: dprevoznik <58714078+dprevoznik@users.noreply.github.com>
Date: Fri, 4 Sep 2026 19:22:30 +0000
Subject: [PATCH 08/20] Remove stale @onkernel/sdk version-pin warning
browser-loop's exact @onkernel/sdk pin is relaxed to a caret range
(kernel/browser-loop#94), matching the same removal already done for
the Playwright/computer-use fallback cookbook (kernel/docs#548).
---
browsers/telemetry/pausing-for-captcha-solves.mdx | 14 --------------
1 file changed, 14 deletions(-)
diff --git a/browsers/telemetry/pausing-for-captcha-solves.mdx b/browsers/telemetry/pausing-for-captcha-solves.mdx
index b113114b..dbd9deee 100644
--- a/browsers/telemetry/pausing-for-captcha-solves.mdx
+++ b/browsers/telemetry/pausing-for-captcha-solves.mdx
@@ -31,20 +31,6 @@ Run the agent through a `browser-loop`-compiled harness. Alongside it, open the
npm install @onkernel/browser-loop @onkernel/sdk @earendil-works/pi-agent-core tsx
```
-
-`browser-loop` depends on an older, exact `@onkernel/sdk` version that predates the [telemetry API](/browsers/telemetry/overview) — installing `@onkernel/sdk` at its current version alongside it can leave two copies in `node_modules`, which produces two incompatible `KERNEL` client types (`attach({ client, browser })` won't type-check, and `client.browsers.telemetry` won't exist on the version `browser-loop` sees). Run `npm ls @onkernel/sdk`; if you see two versions, force a single one with an `overrides` entry in `package.json`:
-
-```json
-{
- "overrides": {
- "@onkernel/sdk": "$@onkernel/sdk"
- }
-}
-```
-
-That pins every copy in the tree to whatever version your own `dependencies` entry resolves to, so `attach()` and `client.browsers.telemetry.stream()` both see the same client type.
-
-
Every script needs a `KERNEL_API_KEY` and a provider key for whichever model `LOOP_MODEL` points at (`anthropic:claude-sonnet-5` by default, so `ANTHROPIC_API_KEY`):
```bash
From 4d7645fd5a6dbc63c413ac2bb1846ccbafbb5c12 Mon Sep 17 00:00:00 2001
From: dprevoznik <58714078+dprevoznik@users.noreply.github.com>
Date: Fri, 4 Sep 2026 22:11:17 +0000
Subject: [PATCH 09/20] Rebuild the captcha cookbook around the correlation
rules
The previous gate paused on captcha_solve_started, joined challenge
results only when a start had carried a challenge_id, and had no
fallback when an event never arrived. On a real reCAPTCHA that dropped
the challenge-level outcome entirely and re-prompted the agent once per
solver task.
The gate now pairs tasks on task_id only, records every challenge result
and marks whether it can be attributed, bounds every wait, and falls
back to a read-only page probe. It holds at the tool_call hook instead
of aborting the turn, so an action is stopped before it reaches the
page and no re-prompt is needed.
Split into four named steps with the complete file in an accordion, and
pin pi-agent-core to the version browser-loop depends on -- the previous
install line resolved a newer one that the script doesn't compile
against.
---
browsers/playwright-computer-use-fallback.mdx | 2 +-
.../telemetry/pausing-for-captcha-solves.mdx | 619 ++++++++++++------
2 files changed, 432 insertions(+), 189 deletions(-)
diff --git a/browsers/playwright-computer-use-fallback.mdx b/browsers/playwright-computer-use-fallback.mdx
index bbc99adb..27c74c2c 100644
--- a/browsers/playwright-computer-use-fallback.mdx
+++ b/browsers/playwright-computer-use-fallback.mdx
@@ -506,4 +506,4 @@ As a starting rule, reach for **Per-Tool Limit** first, even if it costs you one
- [Computer Use overview](/integrations/computer-use/overview) — running computer-use models on KERNEL more generally
- [Replays](/browsers/replays) — record a session end to end
- [Stealth mode](/browsers/bot-detection/stealth) — reduce how often a page notices the automation in the first place
-- [Pausing for Captcha Solves](/browsers/telemetry/pausing-for-captcha-solves) — the same abort-and-resume pattern, applied to captcha telemetry instead of toolset handoffs
+- [Pausing for Captcha Solves](/browsers/telemetry/pausing-for-captcha-solves) — holding an agent mid-run on captcha telemetry instead of toolset handoffs
diff --git a/browsers/telemetry/pausing-for-captcha-solves.mdx b/browsers/telemetry/pausing-for-captcha-solves.mdx
index dbd9deee..d64baa7e 100644
--- a/browsers/telemetry/pausing-for-captcha-solves.mdx
+++ b/browsers/telemetry/pausing-for-captcha-solves.mdx
@@ -1,214 +1,258 @@
---
title: "Pausing for Captcha Solves"
-description: "Use captcha telemetry events to pause an agent while Kernel's solver works, and resume once it succeeds"
+description: "Use captcha telemetry to hold an agent while Kernel's solver works, and tell it what actually happened"
---
-Kernel's [stealth mode](/browsers/bot-detection/stealth) includes an automatic captcha solver that attempts supported challenges — reCAPTCHA, hCaptcha, Cloudflare/Turnstile, and press-and-hold — without any action from your agent. The solver runs in the VM; your agent's job is just to not get in its way while it works.
+Kernel's [stealth mode](/browsers/bot-detection/stealth) includes an automatic captcha solver that attempts supported challenges — reCAPTCHA, hCaptcha, Cloudflare/Turnstile, and press-and-hold — without any action from your agent. The solver runs in the VM; your agent's job is to not get in its way, and to know what happened when it finishes.
-The naive way to handle that is a system-prompt instruction: [`"If you see a CAPTCHA or similar test, just wait for it to get solved automatically."`](/browsers/bot-detection/stealth#anthropic-computer-use) That works, but it's a soft constraint. The model decides for itself how long to wait and how to tell a solve is still running, from a screenshot alone. It can act too soon — a click during an active solve can land on the widget and interfere with it — or wait long after the page has already moved on.
+The usual approach is a system-prompt instruction: [`"If you see a CAPTCHA or similar test, just wait for it to get solved automatically."`](/browsers/bot-detection/stealth#anthropic-computer-use) That works, but the model decides for itself how long to wait and how to tell a solve is still running, from a screenshot alone.
-[Captcha telemetry](/browsers/telemetry/categories#correlate-captcha-tasks-and-challenges) turns that guess into a signal: `captcha_solve_started` fires when the solver accepts a task, and `captcha_solve_result` / `captcha_challenge_result` fire when it reaches a terminal outcome. This cookbook uses [`@onkernel/browser-loop`](https://www.npmjs.com/package/@onkernel/browser-loop) to build an automation that streams those events in real time, pauses the agent the instant a solve starts, and resumes it once telemetry reports a terminal outcome for that solve.
+[Captcha telemetry](/browsers/telemetry/categories#correlate-captcha-tasks-and-challenges) gives you the signal directly. This cookbook wires it into a [`@onkernel/browser-loop`](https://www.npmjs.com/package/@onkernel/browser-loop) agent so that browser actions are held while a solve is outstanding, and the agent is told the outcome in terms it can act on.
-| Event | Fires when | Possible outcomes |
-| --- | --- | --- |
-| `captcha_solve_started` | The solver accepts a task. Doesn't mean a solve is currently in flight. | — |
-| `captcha_solve_result` | A solver task reaches a terminal state. A `success` means the solver returned a usable answer, not that the visible challenge cleared. | `success`, `failure`, `timeout`, `abandoned` |
-| `captcha_challenge_result` | The visible challenge itself reaches its overall terminal state. Only emitted for challenge types Kernel can track as a widget. | `solved`, `failure`, `timeout`, `abandoned` |
+The design follows one split:
+
+
+Telemetry decides **what to say**. The live page decides **whether to interrupt the agent at all**. A solver task succeeding and a challenge clearing are different facts, so the gate never reports one as the other.
+
-## The approach
+## What the events tell you
-Run the agent through a `browser-loop`-compiled harness. Alongside it, open the session's [telemetry stream](/browsers/telemetry/streaming) and watch for `captcha` events:
+| Event | Scope | What it means |
+| --- | --- | --- |
+| `captcha_solve_started` | Solver task | The solver accepted a task. It does **not** mean a solve is currently in flight. |
+| `captcha_solve_result` | Solver task | A task ended `success`, `failure`, `timeout`, or `abandoned`. Success means the solver returned a usable answer, not that the challenge cleared. |
+| `captcha_challenge_result` | Visible challenge | The challenge reached its overall outcome. Only emitted for challenge types Kernel tracks as a widget. |
-1. On `captcha_solve_started`, abort the harness's current turn so the agent stops taking actions.
-2. Wait for the matching terminal event — `captcha_challenge_result` when a `challenge_id` is available, otherwise `captcha_solve_result` on `task_id` — within a bounded timeout.
-3. Re-prompt the harness with what happened, so it picks up with full context instead of starting over.
+Three rules from [Correlate captcha tasks and challenges](/browsers/telemetry/categories#correlate-captcha-tasks-and-challenges) shape everything below:
-`harness.abort()` interrupts the in-flight run, and a fresh `harness.prompt()` call continues the same conversation once the agent is safe to act again.
+- **`task_id` is the only join.** Pair a start with its result on it. `challenge_id` groups tasks for one visible challenge and is present only when Kernel tracked the widget — tasks without one can never be joined to a `captcha_challenge_result`, even when one is emitted for the same page.
+- **Delivery is best-effort and unordered.** A start can arrive after its result, and any event can be absent. Nothing may depend on arrival order, and every wait needs a deadline.
+- **Fall back to the page.** When you need a challenge-level outcome and don't have one, use the available task results and the current page state.
## Setup
+
+Pin `@earendil-works/pi-agent-core` to the version `@onkernel/browser-loop` depends on. A newer one renames the exports this script uses and the script won't compile.
+
+
```bash
-npm install @onkernel/browser-loop @onkernel/sdk @earendil-works/pi-agent-core tsx
+npm install @onkernel/browser-loop @onkernel/sdk @earendil-works/pi-agent-core@0.83.0 tsx
```
-Every script needs a `KERNEL_API_KEY` and a provider key for whichever model `LOOP_MODEL` points at (`anthropic:claude-sonnet-5` by default, so `ANTHROPIC_API_KEY`):
+Every run needs a `KERNEL_API_KEY` and a provider key for whichever model `LOOP_MODEL` points at (`anthropic:claude-sonnet-5` by default, so `ANTHROPIC_API_KEY`):
```bash
-KERNEL_API_KEY=... ANTHROPIC_API_KEY=... npx tsx captcha-solve-wait.ts
+KERNEL_API_KEY=... ANTHROPIC_API_KEY=... npx tsx captcha-gate.ts "your task"
```
-## The script
+## Build the gate
-`createCaptchaGate` is the only piece of custom logic. It opens the telemetry stream, tracks whether a captcha episode is currently in flight, and exposes one method the driver awaits after pausing the agent.
+The four snippets below are one file, `captcha-gate.ts`, in order.
-A `captcha_solve_started` event marks the start of an episode and calls `onSolveStarted`. From there:
+
+
+ Three maps, one per thing the telemetry can tell you, all keyed so nothing depends on arrival order. `joinable` is the important one: it holds only the `challenge_id`s that actually appeared on a task event, which are the only ones a challenge result may be attributed to.
-- **Further `captcha_solve_started` events for the same episode don't fire it again or overwrite the tracked IDs.** An image-grid challenge can run several solver tasks under one `challenge_id`; the driver only needs to know once that it should stop, and losing the original `task_id` would break the pairing described in the [telemetry docs](/browsers/telemetry/categories#correlate-captcha-tasks-and-challenges).
-- **The episode ends on whichever terminal event actually arrives** — `captcha_challenge_result` when a `challenge_id` was seen, or `captcha_solve_result` on `task_id` when it wasn't.
-- **Those two terminal events mean different things**, so the gate keeps them as distinct outcomes instead of collapsing both into "solved": a `captcha_challenge_result` of `solved` means the visible challenge actually cleared, while a `captcha_solve_result` of `success` only means the solver returned a usable answer for that one task (`challenge_solved` vs. `task_success`).
+ A task record created by a *result* already has a status, so a `captcha_solve_started` that arrives afterwards finds a closed task and leaves it closed.
-Because delivery is best-effort and events can arrive out of order, a terminal event can land before the driver even calls `waitForOutcome` (right after `harness.abort()` returns). The gate buffers that outcome instead of dropping it, and `waitForOutcome` checks the buffer before it starts waiting. If nothing ever arrives, the timeout resolves to `"unknown"` and clears the tracked IDs, so a later captcha still triggers a fresh pause instead of being mistaken for the one that timed out.
-
-The driver compiles a Playwright-toolset harness, wires the gate's `onSolveStarted` callback to `harness.abort()`, and loops on `stopReason === "aborted"` — re-prompting with the outcome each time a captcha resolves, until the agent finishes for a reason other than a captcha pause.
-
-```ts
+```ts captcha-gate.ts
/**
- * Run an agent against a page that may present a captcha, pausing on
- * `captcha_solve_started` and resuming once Kernel's solver reports a
- * terminal outcome for that episode.
- *
- * Set TASK_PROMPT below to whatever task the agent should run -- anything
- * that might land it on a captcha-protected page. Kernel's stealth captcha
- * solver handles supported challenge types automatically; this script's job
- * is only to keep the agent from acting on the page while a solve is in
- * progress.
+ * Pause a browser-loop agent while Kernel's captcha solver works, and tell it
+ * what actually happened when the solve ends.
*
* Usage:
- * KERNEL_API_KEY=... ANTHROPIC_API_KEY=... npx tsx captcha-solve-wait.ts
- *
- * Env:
- * KERNEL_API_KEY required, KERNEL browser API key
- * LOOP_MODEL optional, defaults to anthropic:claude-sonnet-5
- * (needs the matching provider API key, e.g. ANTHROPIC_API_KEY)
+ * KERNEL_API_KEY=... ANTHROPIC_API_KEY=... npx tsx captcha-gate.ts "your task"
*/
import KERNEL from "@onkernel/sdk";
-import { AgentHarness, InMemorySessionRepo, type AgentHarnessEvent } from "@earendil-works/pi-agent-core";
+import { AgentHarness, InMemorySessionRepo } from "@earendil-works/pi-agent-core";
import { loop } from "@onkernel/browser-loop";
import { attach, requireLoopEnvApiKeyForModel, type LoopModelRef } from "@onkernel/browser-loop/pi";
-type CaptchaOutcome = "challenge_solved" | "task_success" | "failure" | "timeout" | "abandoned" | "unknown";
-
-// Tracks one open captcha episode at a time: it opens on captcha_solve_started
-// and closes on whichever terminal event actually arrives for it. A second,
-// unrelated episode starting while one is already open is ignored rather than
-// overwriting the tracked IDs -- overlapping captcha widgets are rare, and the
-// telemetry docs note that even Kernel's own event model can't always
-// attribute a result when multiple same-provider challenges are open at once.
-function createCaptchaGate(client: KERNEL, sessionId: string, onSolveStarted: () => void) {
- let taskId: string | null = null;
- let challengeId: string | null = null;
- let waiter: ((outcome: CaptchaOutcome) => void) | null = null;
-
- // A finished episode's outcome, kept alongside the IDs it belongs to. Delivery
- // is unordered, so a captcha_solve_started for this same episode can still
- // arrive after its result; matching on these IDs tells that late duplicate
- // apart from a genuinely new episode.
- let bufferedOutcome: CaptchaOutcome | null = null;
- let bufferedTaskId: string | null = null;
- let bufferedChallengeId: string | null = null;
-
- function matchesBuffer(eventTaskId?: string, eventChallengeId?: string): boolean {
- if (bufferedOutcome === null) return false;
- return (
- (eventChallengeId !== undefined && eventChallengeId === bufferedChallengeId) ||
- (eventTaskId !== undefined && eventTaskId === bufferedTaskId)
- );
- }
+const TASK_SETTLE_MS = 20_000;
+const CHALLENGE_GRACE_MS = 8_000;
+const POLL_MS = 250;
- function clearBuffer() {
- bufferedOutcome = null;
- bufferedTaskId = null;
- bufferedChallengeId = null;
- }
+const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms));
- function finish(outcome: CaptchaOutcome) {
- if (waiter) {
- const resolve = waiter;
- waiter = null;
- taskId = null;
- challengeId = null;
- resolve(outcome);
- } else {
- // waitForOutcome hasn't been called yet -- buffer the outcome so a
- // fast terminal event isn't dropped by the race with harness.abort().
- bufferedOutcome = outcome;
- bufferedTaskId = taskId;
- bufferedChallengeId = challengeId;
- taskId = null;
- challengeId = null;
- }
- }
+interface PageState {
+ widgets: string[];
+ tokenPresent: boolean;
+}
+
+interface Outcome {
+ status?: string;
+ durationMs?: number;
+ captchaType?: string;
+}
- (async () => {
- const stream = await client.browsers.telemetry.stream(sessionId);
+interface Verdict extends Outcome {
+ source: "challenge" | "task" | "page";
+ status: string;
+ joined: boolean;
+ page: PageState;
+ message: string;
+}
+
+function createCaptchaGate(kernel: KERNEL, sessionId: string, probePage: () => Promise) {
+ const tasks = new Map();
+ const challenges = new Map();
+ // Only challenge_ids that appeared on a task event may be attributed to it.
+ const joinable = new Set();
+
+ void (async () => {
+ const stream = await kernel.browsers.telemetry.stream(sessionId);
for await (const { event } of stream) {
if (event.category !== "captcha") continue;
- if (event.type === "captcha_solve_started") {
- const isSameOpenEpisode =
- (event.data.challenge_id !== undefined && event.data.challenge_id === challengeId) ||
- (event.data.task_id !== undefined && event.data.task_id === taskId);
- if (isSameOpenEpisode) continue;
- if (matchesBuffer(event.data.task_id, event.data.challenge_id)) continue;
- const isIdle = taskId === null && challengeId === null;
- if (isIdle) {
- // A genuinely new episode makes any buffered outcome stale: it
- // belongs to an earlier episode the driver hasn't collected yet,
- // and this new one is what waitForOutcome should wait on instead.
- clearBuffer();
- onSolveStarted();
- taskId = event.data.task_id ?? null;
- challengeId = event.data.challenge_id ?? null;
- }
- } else if (event.type === "captcha_challenge_result" && event.data.challenge_id === challengeId) {
- finish(event.data.status === "solved" ? "challenge_solved" : event.data.status);
- } else if (event.type === "captcha_solve_result" && !challengeId && event.data?.task_id === taskId) {
- finish(event.data.status === "success" ? "task_success" : event.data.status);
+ if (event.type === "captcha_solve_started" && event.data?.task_id) {
+ // A result for an unseen task lands already closed, so a start that
+ // arrives after its own result never reopens it.
+ const task = tasks.get(event.data.task_id) ?? { openedAt: Date.now() };
+ task.captchaType = event.data.captcha_type ?? task.captchaType;
+ tasks.set(event.data.task_id, task);
+ if (event.data.challenge_id) joinable.add(event.data.challenge_id);
+ } else if (event.type === "captcha_solve_result" && event.data?.task_id) {
+ const task = tasks.get(event.data.task_id) ?? { openedAt: Date.now() };
+ task.status = event.data.status;
+ task.durationMs = event.data.duration_ms;
+ task.captchaType = event.data.captcha_type ?? task.captchaType;
+ tasks.set(event.data.task_id, task);
+ if (event.data.challenge_id) joinable.add(event.data.challenge_id);
+ } else if (event.type === "captcha_challenge_result" && event.data?.challenge_id) {
+ challenges.set(event.data.challenge_id, {
+ status: event.data.status,
+ durationMs: event.data.duration_ms,
+ captchaType: event.data.captcha_type,
+ });
}
}
})();
+```
+
+
+
+ A task counts as open while it has no terminal status and is still inside its deadline — that deadline is what stops a missing `captcha_solve_result` from holding the agent forever. `until` is the only waiting primitive, and it always takes a timeout.
+
+```ts captcha-gate.ts
+ const openTasks = () => [...tasks.values()].filter((t) => !t.status && Date.now() - t.openedAt < TASK_SETTLE_MS);
+ const joinedResult = () => [...challenges].find(([id]) => joinable.has(id));
+
+ async function until(done: () => boolean, timeoutMs: number) {
+ const deadline = Date.now() + timeoutMs;
+ while (!done() && Date.now() < deadline) await sleep(POLL_MS);
+ }
+```
+
+
+
+ `resolve` settles what it can, waits a bounded interval for a challenge-level result *only* when a task actually carried a `challenge_id`, then reads the page and picks the best available source: an attributable challenge result, then any challenge result (labelled as a page observation rather than a join), then task results, then the page alone.
+
+ `holding` and `pending` are separate on purpose. `pending` covers a terminal outcome that lands while no action is in flight — without it, a challenge result arriving between tool calls is recorded and never told to anyone.
+
+```ts captcha-gate.ts
+ async function resolve(): Promise {
+ // A challenge result covers every task under it, so stop waiting once one lands.
+ await until(() => openTasks().length === 0 || Boolean(joinedResult()), TASK_SETTLE_MS);
+ if (joinable.size > 0 && !joinedResult()) await until(() => Boolean(joinedResult()), CHALLENGE_GRACE_MS);
+
+ const page = await probePage();
+ const challenge = joinedResult() ?? [...challenges][0];
+ if (challenge) return describe("challenge", challenge[1].status, joinable.has(challenge[0]), challenge[1], page);
+
+ const finished = [...tasks.values()].filter((t) => t.status);
+ const task = finished.find((t) => t.status !== "success") ?? finished[0];
+ if (task) return describe("task", task.status!, false, task, page);
+
+ return describe("page", "unknown", false, {}, page);
+ }
return {
- waitForOutcome(timeoutMs: number): Promise {
- if (bufferedOutcome !== null) {
- const outcome = bufferedOutcome;
- clearBuffer();
- return Promise.resolve(outcome);
- }
- if (taskId === null && challengeId === null) return Promise.resolve("challenge_solved");
- return new Promise((resolve) => {
- waiter = resolve;
- setTimeout(() => {
- if (waiter !== resolve) return;
- waiter = null;
- // Give up waiting -- clear the tracked IDs so a later
- // captcha_solve_started still triggers a fresh pause instead
- // of being mistaken for the episode that just timed out.
- taskId = null;
- challengeId = null;
- resolve("unknown");
- }, timeoutMs);
- });
+ /** A task the solver accepted has no terminal result yet. */
+ holding: () => openTasks().length > 0,
+ /** A terminal outcome is recorded that the agent hasn't been told about. */
+ pending: () => challenges.size > 0 || [...tasks.values()].some((t) => t.status && t.status !== "success"),
+ resolve,
+ reset: () => {
+ tasks.clear();
+ challenges.clear();
+ joinable.clear();
},
};
}
-const TASK_PROMPT = "[ENTER TASK FOR AGENT THAT MAY NEED CAPTCHA SOLVING]";
+function describe(source: Verdict["source"], status: string, joined: boolean, outcome: Outcome, page: PageState): Verdict {
+ const took = outcome.durationMs ? ` after ${(outcome.durationMs / 1000).toFixed(1)}s` : "";
+ const kind = outcome.captchaType ? `${outcome.captchaType} ` : "";
+ const headline =
+ source === "page"
+ ? "No terminal captcha telemetry arrived."
+ : source === "task"
+ ? status === "success"
+ ? `The solver returned an answer for a ${kind}task${took}. No challenge-level outcome was reported, so this is not a cleared challenge.`
+ : `A ${kind}solver task ended as "${status}"${took}.`
+ : status === "solved"
+ ? `Kernel observed the ${kind}challenge clear${took}. That is not proof the site accepted the solution.`
+ : `Kernel reported the ${kind}challenge as "${status}"${took}.`;
+ const caveat =
+ source === "challenge" && !joined
+ ? " It could not be joined to this page's solver tasks, so treat it as an observation about the page."
+ : "";
+ const where = page.widgets.length
+ ? `A captcha widget is still on the page (${page.widgets.join(", ")}).`
+ : "No captcha widget is visible on the page.";
+ return { source, status, joined, captchaType: outcome.captchaType, durationMs: outcome.durationMs, page, message: `${headline}${caveat} ${where}` };
+}
+```
+
+
+
+ `harness.on("tool_call", …)` is awaited before the tool is dispatched, so returning from it late holds the action and returning `{ block: true, reason }` replaces it with a message the model reads. That is cheaper than aborting the turn and it stops the action *before* it reaches the page.
+
+ The page probe is a read-only Playwright snippet. Its result is what decides whether the agent is interrupted at all — a cleared page just resumes with no extra turn.
+
+```ts captcha-gate.ts
+const PROBE = `
+return await page.evaluate(() => {
+ const groups = {
+ recaptcha: 'iframe[src*="recaptcha"], .g-recaptcha',
+ hcaptcha: 'iframe[src*="hcaptcha"], .h-captcha',
+ turnstile: 'iframe[src*="challenges.cloudflare.com"], .cf-turnstile',
+ };
+ const onScreen = (el) => { const r = el.getBoundingClientRect(); return r.width > 0 && r.height > 0; };
+ return {
+ widgets: Object.entries(groups)
+ .filter(([, sel]) => [...document.querySelectorAll(sel)].some(onScreen))
+ .map(([name]) => name),
+ tokenPresent: [...document.querySelectorAll('[name="cf-turnstile-response"], [name="g-recaptcha-response"]')]
+ .some((el) => el.value.length > 0),
+ };
+});
+`;
const MODEL = (process.env.LOOP_MODEL as LoopModelRef | undefined) ?? "anthropic:claude-sonnet-5";
-// How long to wait for a terminal captcha event before giving up and letting
-// the agent reassess the page on its own.
-const CAPTCHA_WAIT_TIMEOUT_MS = 45_000;
-
async function main(): Promise {
- const kernelApiKey = process.env.KERNEL_API_KEY;
- if (!kernelApiKey) throw new Error("KERNEL_API_KEY is required");
+ const task = process.argv[2];
+ if (!task) throw new Error("pass the task as the first argument");
requireLoopEnvApiKeyForModel(MODEL);
- const client = new KERNEL({ apiKey: kernelApiKey });
- const browser = await client.browsers.create({
+ const kernel = new KERNEL();
+ const browser = await kernel.browsers.create({
stealth: true,
telemetry: { browser: { captcha: { enabled: true } } },
});
- const kb = attach({ client, browser });
+ const kb = attach({ client: kernel, browser });
- try {
- const session = await new InMemorySessionRepo().create({ id: "captcha-solve-wait" });
+ const probePage = async (): Promise => {
+ const { result } = await kernel.browsers.playwright.execute(browser.session_id, { code: PROBE });
+ return (result as PageState | undefined) ?? { widgets: [], tokenPresent: false };
+ };
+ const gate = createCaptchaGate(kernel, browser.session_id, probePage);
+ try {
const compiled = kb.compile({ model: MODEL, tools: loop.toolsets.browser() });
const harness = new AgentHarness({
- session,
+ session: await new InMemorySessionRepo().create({ id: browser.session_id }),
model: compiled.model,
models: compiled.models,
tools: [...compiled.tools],
@@ -217,60 +261,259 @@ async function main(): Promise {
});
compiled.activate(harness);
- let actionTurns = 0;
- harness.subscribe((event: AgentHarnessEvent) => {
- if (event.type !== "tool_execution_end") return;
- actionTurns += 1;
- console.log(`[agent ${actionTurns}] ${event.toolName} error=${event.isError}`);
+ harness.on("tool_call", async () => {
+ if (!gate.holding() && !gate.pending()) return undefined;
+ const verdict = await gate.resolve();
+ gate.reset();
+ console.log(`[captcha] ${verdict.source}:${verdict.status} — ${verdict.message}`);
+ // Telemetry decides what to say; the page decides whether to interrupt.
+ if (verdict.page.widgets.length === 0) return undefined;
+ return { block: true, reason: verdict.message };
});
- const gate = createCaptchaGate(client, browser.session_id, () => {
- console.log("[captcha] solve started, pausing agent");
- void harness.abort();
- });
+ const final = await harness.prompt(task);
+ for (const block of final.content) if (block.type === "text") console.log(block.text);
+ } finally {
+ await kb.dispose();
+ await kernel.browsers.deleteByID(browser.session_id);
+ }
+}
- console.log(`model=${MODEL} prompt=${JSON.stringify(TASK_PROMPT)}`);
- let final = await harness.prompt(TASK_PROMPT);
-
- while (final.stopReason === "aborted") {
- const outcome = await gate.waitForOutcome(CAPTCHA_WAIT_TIMEOUT_MS);
- console.log(`[captcha] resolved: ${outcome}`);
- const followUp =
- outcome === "challenge_solved"
- ? "Kernel's captcha solver reported that the visible challenge cleared. Continue the task."
- : outcome === "task_success"
- ? "Kernel's captcha solver returned a usable solution, but there's no visible-challenge " +
- "confirmation for this page. Take a screenshot to confirm the challenge actually " +
- "cleared, then continue or retry."
- : `The captcha challenge ended as "${outcome}". Take a screenshot to check the current ` +
- "page state, then retry the challenge yourself or report the blocker.";
- final = await harness.prompt(followUp);
- }
+void main();
+```
+
+
+
+
+```ts captcha-gate.ts
+/**
+ * Pause a browser-loop agent while Kernel's captcha solver works, and tell it
+ * what actually happened when the solve ends.
+ *
+ * Usage:
+ * KERNEL_API_KEY=... ANTHROPIC_API_KEY=... npx tsx captcha-gate.ts "your task"
+ */
+import KERNEL from "@onkernel/sdk";
+import { AgentHarness, InMemorySessionRepo } from "@earendil-works/pi-agent-core";
+import { loop } from "@onkernel/browser-loop";
+import { attach, requireLoopEnvApiKeyForModel, type LoopModelRef } from "@onkernel/browser-loop/pi";
- console.log(`final stopReason: ${final.stopReason}`);
- for (const block of final.content) {
- if (block.type === "text") console.log(block.text);
+const TASK_SETTLE_MS = 20_000;
+const CHALLENGE_GRACE_MS = 8_000;
+const POLL_MS = 250;
+
+const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms));
+
+interface PageState {
+ widgets: string[];
+ tokenPresent: boolean;
+}
+
+interface Outcome {
+ status?: string;
+ durationMs?: number;
+ captchaType?: string;
+}
+
+interface Verdict extends Outcome {
+ source: "challenge" | "task" | "page";
+ status: string;
+ joined: boolean;
+ page: PageState;
+ message: string;
+}
+
+function createCaptchaGate(kernel: KERNEL, sessionId: string, probePage: () => Promise) {
+ const tasks = new Map();
+ const challenges = new Map();
+ // Only challenge_ids that appeared on a task event may be attributed to it.
+ const joinable = new Set();
+
+ void (async () => {
+ const stream = await kernel.browsers.telemetry.stream(sessionId);
+ for await (const { event } of stream) {
+ if (event.category !== "captcha") continue;
+
+ if (event.type === "captcha_solve_started" && event.data?.task_id) {
+ // A result for an unseen task lands already closed, so a start that
+ // arrives after its own result never reopens it.
+ const task = tasks.get(event.data.task_id) ?? { openedAt: Date.now() };
+ task.captchaType = event.data.captcha_type ?? task.captchaType;
+ tasks.set(event.data.task_id, task);
+ if (event.data.challenge_id) joinable.add(event.data.challenge_id);
+ } else if (event.type === "captcha_solve_result" && event.data?.task_id) {
+ const task = tasks.get(event.data.task_id) ?? { openedAt: Date.now() };
+ task.status = event.data.status;
+ task.durationMs = event.data.duration_ms;
+ task.captchaType = event.data.captcha_type ?? task.captchaType;
+ tasks.set(event.data.task_id, task);
+ if (event.data.challenge_id) joinable.add(event.data.challenge_id);
+ } else if (event.type === "captcha_challenge_result" && event.data?.challenge_id) {
+ challenges.set(event.data.challenge_id, {
+ status: event.data.status,
+ durationMs: event.data.duration_ms,
+ captchaType: event.data.captcha_type,
+ });
+ }
}
+ })();
+
+ const openTasks = () => [...tasks.values()].filter((t) => !t.status && Date.now() - t.openedAt < TASK_SETTLE_MS);
+ const joinedResult = () => [...challenges].find(([id]) => joinable.has(id));
+
+ async function until(done: () => boolean, timeoutMs: number) {
+ const deadline = Date.now() + timeoutMs;
+ while (!done() && Date.now() < deadline) await sleep(POLL_MS);
+ }
+
+ async function resolve(): Promise {
+ // A challenge result covers every task under it, so stop waiting once one lands.
+ await until(() => openTasks().length === 0 || Boolean(joinedResult()), TASK_SETTLE_MS);
+ if (joinable.size > 0 && !joinedResult()) await until(() => Boolean(joinedResult()), CHALLENGE_GRACE_MS);
+
+ const page = await probePage();
+ const challenge = joinedResult() ?? [...challenges][0];
+ if (challenge) return describe("challenge", challenge[1].status, joinable.has(challenge[0]), challenge[1], page);
+
+ const finished = [...tasks.values()].filter((t) => t.status);
+ const task = finished.find((t) => t.status !== "success") ?? finished[0];
+ if (task) return describe("task", task.status!, false, task, page);
+
+ return describe("page", "unknown", false, {}, page);
+ }
+
+ return {
+ /** A task the solver accepted has no terminal result yet. */
+ holding: () => openTasks().length > 0,
+ /** A terminal outcome is recorded that the agent hasn't been told about. */
+ pending: () => challenges.size > 0 || [...tasks.values()].some((t) => t.status && t.status !== "success"),
+ resolve,
+ reset: () => {
+ tasks.clear();
+ challenges.clear();
+ joinable.clear();
+ },
+ };
+}
+
+function describe(source: Verdict["source"], status: string, joined: boolean, outcome: Outcome, page: PageState): Verdict {
+ const took = outcome.durationMs ? ` after ${(outcome.durationMs / 1000).toFixed(1)}s` : "";
+ const kind = outcome.captchaType ? `${outcome.captchaType} ` : "";
+ const headline =
+ source === "page"
+ ? "No terminal captcha telemetry arrived."
+ : source === "task"
+ ? status === "success"
+ ? `The solver returned an answer for a ${kind}task${took}. No challenge-level outcome was reported, so this is not a cleared challenge.`
+ : `A ${kind}solver task ended as "${status}"${took}.`
+ : status === "solved"
+ ? `Kernel observed the ${kind}challenge clear${took}. That is not proof the site accepted the solution.`
+ : `Kernel reported the ${kind}challenge as "${status}"${took}.`;
+ const caveat =
+ source === "challenge" && !joined
+ ? " It could not be joined to this page's solver tasks, so treat it as an observation about the page."
+ : "";
+ const where = page.widgets.length
+ ? `A captcha widget is still on the page (${page.widgets.join(", ")}).`
+ : "No captcha widget is visible on the page.";
+ return { source, status, joined, captchaType: outcome.captchaType, durationMs: outcome.durationMs, page, message: `${headline}${caveat} ${where}` };
+}
+
+const PROBE = `
+return await page.evaluate(() => {
+ const groups = {
+ recaptcha: 'iframe[src*="recaptcha"], .g-recaptcha',
+ hcaptcha: 'iframe[src*="hcaptcha"], .h-captcha',
+ turnstile: 'iframe[src*="challenges.cloudflare.com"], .cf-turnstile',
+ };
+ const onScreen = (el) => { const r = el.getBoundingClientRect(); return r.width > 0 && r.height > 0; };
+ return {
+ widgets: Object.entries(groups)
+ .filter(([, sel]) => [...document.querySelectorAll(sel)].some(onScreen))
+ .map(([name]) => name),
+ tokenPresent: [...document.querySelectorAll('[name="cf-turnstile-response"], [name="g-recaptcha-response"]')]
+ .some((el) => el.value.length > 0),
+ };
+});
+`;
+
+const MODEL = (process.env.LOOP_MODEL as LoopModelRef | undefined) ?? "anthropic:claude-sonnet-5";
+
+async function main(): Promise {
+ const task = process.argv[2];
+ if (!task) throw new Error("pass the task as the first argument");
+ requireLoopEnvApiKeyForModel(MODEL);
+
+ const kernel = new KERNEL();
+ const browser = await kernel.browsers.create({
+ stealth: true,
+ telemetry: { browser: { captcha: { enabled: true } } },
+ });
+ const kb = attach({ client: kernel, browser });
+
+ const probePage = async (): Promise => {
+ const { result } = await kernel.browsers.playwright.execute(browser.session_id, { code: PROBE });
+ return (result as PageState | undefined) ?? { widgets: [], tokenPresent: false };
+ };
+ const gate = createCaptchaGate(kernel, browser.session_id, probePage);
+
+ try {
+ const compiled = kb.compile({ model: MODEL, tools: loop.toolsets.browser() });
+ const harness = new AgentHarness({
+ session: await new InMemorySessionRepo().create({ id: browser.session_id }),
+ model: compiled.model,
+ models: compiled.models,
+ tools: [...compiled.tools],
+ activeToolNames: compiled.tools.map((tool) => tool.name),
+ systemPrompt: "Use the supplied browser tools to complete the task efficiently.",
+ });
+ compiled.activate(harness);
+
+ harness.on("tool_call", async () => {
+ if (!gate.holding() && !gate.pending()) return undefined;
+ const verdict = await gate.resolve();
+ gate.reset();
+ console.log(`[captcha] ${verdict.source}:${verdict.status} — ${verdict.message}`);
+ // Telemetry decides what to say; the page decides whether to interrupt.
+ if (verdict.page.widgets.length === 0) return undefined;
+ return { block: true, reason: verdict.message };
+ });
+
+ const final = await harness.prompt(task);
+ for (const block of final.content) if (block.type === "text") console.log(block.text);
} finally {
await kb.dispose();
- await client.browsers.deleteByID(browser.session_id);
+ await kernel.browsers.deleteByID(browser.session_id);
}
}
void main();
```
+
+
+## What the agent is told
+
+Every verdict pairs a telemetry claim with the page state it was checked against:
+
+| Source | Message |
+| --- | --- |
+| Attributable challenge result | "Kernel observed the challenge clear after 18.0s. That is not proof the site accepted the solution. No captcha widget is visible on the page." |
+| Unjoinable challenge result | Same, plus "It could not be joined to this page's solver tasks, so treat it as an observation about the page." |
+| Task result only | "The solver returned an answer for a turnstile task after 3.9s. No challenge-level outcome was reported, so this is not a cleared challenge. A captcha widget is still on the page (turnstile)." |
+| Nothing terminal arrived | "No terminal captcha telemetry arrived. A captcha widget is still on the page (turnstile)." |
+
+Durations come straight from each event's `duration_ms`, which is authoritative; don't compute them from event timestamps.
-## Notes
+## Limits
-- **The gate only matters once a run is in flight.** If a captcha appears and clears before the first `harness.prompt()` call starts, there's nothing to pause — which is fine, since there was nothing for the agent to interfere with either.
-- **`harness.abort()` interrupts the current turn, not just the next tool call.** The agent's in-flight action still completes on the browser side; the harness stops before starting another one.
-- **Sequential captchas in one run just loop.** Each terminal event (or timeout) resets the gate's state, so a second `captcha_solve_started` later in the task pauses and resumes the same way.
-- **Overlapping captchas aren't split apart.** The gate tracks one open episode at a time; a second, unrelated `captcha_solve_started` while the first is still open is ignored rather than risking a mismatched pairing. Two genuinely concurrent visible challenges are rare, and the [telemetry docs](/browsers/telemetry/categories#correlate-captcha-tasks-and-challenges) note that Kernel's own event model can't always attribute a result in that case either.
-- **A bounded wait beats an indefinite one.** Event delivery isn't guaranteed or ordered, so `waitForOutcome`'s timeout is what keeps a dropped event from stalling the run forever — and it clears the gate's state on the way out so the next captcha isn't mistaken for the one that timed out.
+- **`TASK_SETTLE_MS` and `CHALLENGE_GRACE_MS` are the safety net.** Events can be absent, so both waits are bounded and the gate falls through to the page rather than stalling. Raise `TASK_SETTLE_MS` if your solves routinely run longer than 20s.
+- **Challenge results aren't emitted for every widget type.** Turnstile, for instance, reports task events only, so the task-plus-page path is the one that runs there.
+- **One episode at a time.** The gate resets after each verdict; overlapping visible challenges from the same provider aren't split apart, and the telemetry docs note Kernel's own event model can't always attribute a result in that case either.
+- **The page probe is best-effort.** It reads the DOM for known widget markers and a response token. A site that renders its challenge somewhere those selectors miss will fall back to the telemetry verdict alone.
## Next steps
-- [Telemetry Categories](/browsers/telemetry/categories#correlate-captcha-tasks-and-challenges) — the full captcha event schema, `task_id` / `challenge_id` correlation, and outcome meanings
+- [Telemetry Categories](/browsers/telemetry/categories#correlate-captcha-tasks-and-challenges) — the full captcha event schema and correlation rules
- [Stream Telemetry](/browsers/telemetry/streaming) — resuming a dropped stream, filtering by category
- [Stealth mode](/browsers/bot-detection/stealth) — what the automatic captcha solver covers
-- [Playwright with Computer Use Fallback](/browsers/playwright-computer-use-fallback) — the abort-and-resume pattern this cookbook reuses
+- [Playwright with Computer Use Fallback](/browsers/playwright-computer-use-fallback) — holding and redirecting an agent mid-run for a different reason
From dbdabaacafa6a6da1e1f00c96a4a37631c96522c Mon Sep 17 00:00:00 2001
From: dprevoznik <58714078+dprevoznik@users.noreply.github.com>
Date: Tue, 8 Sep 2026 17:30:55 +0000
Subject: [PATCH 10/20] Fold the telemetry/page-observation split into body
prose
Was in a Note callout, which undersold content essential to
understanding the cookbook's design rather than a skippable aside.
---
browsers/telemetry/pausing-for-captcha-solves.mdx | 6 +-----
1 file changed, 1 insertion(+), 5 deletions(-)
diff --git a/browsers/telemetry/pausing-for-captcha-solves.mdx b/browsers/telemetry/pausing-for-captcha-solves.mdx
index d64baa7e..29c389b2 100644
--- a/browsers/telemetry/pausing-for-captcha-solves.mdx
+++ b/browsers/telemetry/pausing-for-captcha-solves.mdx
@@ -9,11 +9,7 @@ The usual approach is a system-prompt instruction: [`"If you see a CAPTCHA or si
[Captcha telemetry](/browsers/telemetry/categories#correlate-captcha-tasks-and-challenges) gives you the signal directly. This cookbook wires it into a [`@onkernel/browser-loop`](https://www.npmjs.com/package/@onkernel/browser-loop) agent so that browser actions are held while a solve is outstanding, and the agent is told the outcome in terms it can act on.
-The design follows one split:
-
-
-Telemetry decides **what to say**. The live page decides **whether to interrupt the agent at all**. A solver task succeeding and a challenge clearing are different facts, so the gate never reports one as the other.
-
+The design follows one split: telemetry decides **what to say**, while the live page decides **whether to interrupt the agent at all**. A solver task succeeding and a challenge clearing are different facts, so the gate never reports one as the other.
## What the events tell you
From 454748b90f9d36ff8b216005ee665eb40206e0c7 Mon Sep 17 00:00:00 2001
From: dprevoznik <58714078+dprevoznik@users.noreply.github.com>
Date: Fri, 11 Sep 2026 20:46:28 +0000
Subject: [PATCH 11/20] Fix ERR_PACKAGE_PATH_NOT_EXPORTED in the captcha-gate
Setup instructions
@earendil-works/pi-agent-core ships ESM only, so tsx resolves the
script as CommonJS and fails immediately without "type": "module" in
package.json. Reproduced the exact error following the documented
setup steps as written, then verified npm pkg set type=module fixes
it.
---
browsers/telemetry/pausing-for-captcha-solves.mdx | 3 +++
1 file changed, 3 insertions(+)
diff --git a/browsers/telemetry/pausing-for-captcha-solves.mdx b/browsers/telemetry/pausing-for-captcha-solves.mdx
index 29c389b2..52785f84 100644
--- a/browsers/telemetry/pausing-for-captcha-solves.mdx
+++ b/browsers/telemetry/pausing-for-captcha-solves.mdx
@@ -33,8 +33,11 @@ Pin `@earendil-works/pi-agent-core` to the version `@onkernel/browser-loop` depe
```bash
npm install @onkernel/browser-loop @onkernel/sdk @earendil-works/pi-agent-core@0.83.0 tsx
+npm pkg set type=module
```
+`@earendil-works/pi-agent-core` ships ESM only. Without `type: module` in `package.json`, `tsx` resolves `captcha-gate.ts`'s import of it as CommonJS and fails immediately with `ERR_PACKAGE_PATH_NOT_EXPORTED`.
+
Every run needs a `KERNEL_API_KEY` and a provider key for whichever model `LOOP_MODEL` points at (`anthropic:claude-sonnet-5` by default, so `ANTHROPIC_API_KEY`):
```bash
From b6ef14205e75ba58809b122672a5ab30db2289c7 Mon Sep 17 00:00:00 2001
From: dprevoznik <58714078+dprevoznik@users.noreply.github.com>
Date: Fri, 11 Sep 2026 20:47:48 +0000
Subject: [PATCH 12/20] Close the telemetry stream before deleting the browser
The gate's telemetry loop runs detached and was never explicitly
stopped, so it keeps its connection open past the browser's deletion.
Reproduced: the documented script hangs indefinitely after finishing
its task instead of exiting (confirmed via a live run, deleteByID
completes fine but the process never returns). Fixed by saving the
stream's AbortController and calling gate.close() in main()'s finally
block before deletion; verified the same live run now exits cleanly
in ~5s.
---
browsers/telemetry/pausing-for-captcha-solves.mdx | 14 ++++++++++++++
1 file changed, 14 insertions(+)
diff --git a/browsers/telemetry/pausing-for-captcha-solves.mdx b/browsers/telemetry/pausing-for-captcha-solves.mdx
index 52785f84..587ca193 100644
--- a/browsers/telemetry/pausing-for-captcha-solves.mdx
+++ b/browsers/telemetry/pausing-for-captcha-solves.mdx
@@ -54,6 +54,8 @@ The four snippets below are one file, `captcha-gate.ts`, in order.
A task record created by a *result* already has a status, so a `captcha_solve_started` that arrives afterwards finds a closed task and leaves it closed.
+ The stream's `AbortController` is saved so the gate can close it later — the loop runs detached from the caller, and leaving it open past the browser's deletion keeps the process alive waiting on a connection that will never produce another event.
+
```ts captcha-gate.ts
/**
* Pause a browser-loop agent while Kernel's captcha solver works, and tell it
@@ -97,9 +99,11 @@ function createCaptchaGate(kernel: KERNEL, sessionId: string, probePage: () => P
const challenges = new Map();
// Only challenge_ids that appeared on a task event may be attributed to it.
const joinable = new Set();
+ let streamController: AbortController | undefined;
void (async () => {
const stream = await kernel.browsers.telemetry.stream(sessionId);
+ streamController = stream.controller;
for await (const { event } of stream) {
if (event.category !== "captcha") continue;
@@ -176,6 +180,8 @@ function createCaptchaGate(kernel: KERNEL, sessionId: string, probePage: () => P
challenges.clear();
joinable.clear();
},
+ /** Stop the telemetry stream. Call before deleting the browser. */
+ close: () => streamController?.abort(),
};
}
@@ -209,6 +215,8 @@ function describe(source: Verdict["source"], status: string, joined: boolean, ou
The page probe is a read-only Playwright snippet. Its result is what decides whether the agent is interrupted at all — a cleared page just resumes with no extra turn.
+ `gate.close()` runs before the browser is deleted, so the detached telemetry loop from step one gets torn down instead of hanging on a connection to a session that no longer exists.
+
```ts captcha-gate.ts
const PROBE = `
return await page.evaluate(() => {
@@ -273,6 +281,7 @@ async function main(): Promise {
const final = await harness.prompt(task);
for (const block of final.content) if (block.type === "text") console.log(block.text);
} finally {
+ gate.close();
await kb.dispose();
await kernel.browsers.deleteByID(browser.session_id);
}
@@ -327,9 +336,11 @@ function createCaptchaGate(kernel: KERNEL, sessionId: string, probePage: () => P
const challenges = new Map();
// Only challenge_ids that appeared on a task event may be attributed to it.
const joinable = new Set();
+ let streamController: AbortController | undefined;
void (async () => {
const stream = await kernel.browsers.telemetry.stream(sessionId);
+ streamController = stream.controller;
for await (const { event } of stream) {
if (event.category !== "captcha") continue;
@@ -392,6 +403,8 @@ function createCaptchaGate(kernel: KERNEL, sessionId: string, probePage: () => P
challenges.clear();
joinable.clear();
},
+ /** Stop the telemetry stream. Call before deleting the browser. */
+ close: () => streamController?.abort(),
};
}
@@ -481,6 +494,7 @@ async function main(): Promise {
const final = await harness.prompt(task);
for (const block of final.content) if (block.type === "text") console.log(block.text);
} finally {
+ gate.close();
await kb.dispose();
await kernel.browsers.deleteByID(browser.session_id);
}
From 5f8899b3f2b1ba2d00172e7e947edd3f28a2bb74 Mon Sep 17 00:00:00 2001
From: dprevoznik <58714078+dprevoznik@users.noreply.github.com>
Date: Fri, 11 Sep 2026 20:48:35 +0000
Subject: [PATCH 13/20] Fix pending() dropping successful and expired
unresolved tasks
pending() only counted a settled task if its status was a non-success
terminal value, so a successful task-only result (Turnstile's usual
path, per the cookbook's own documented example) or a task whose
deadline lapsed with no result at all were neither holding() nor
pending() -- resolve() and the page probe never ran for them, and the
agent was never told. Replaced the status-based check with the same
isOpen() predicate openTasks() already uses, so pending() means "some
task settled, one way or another." Verified against a scripted
turnstile-success case: it now reproduces the exact message documented
in the "What the agent is told" table, which the prior logic could
never actually produce.
---
browsers/telemetry/pausing-for-captcha-solves.mdx | 12 +++++++-----
1 file changed, 7 insertions(+), 5 deletions(-)
diff --git a/browsers/telemetry/pausing-for-captcha-solves.mdx b/browsers/telemetry/pausing-for-captcha-solves.mdx
index 587ca193..07b9369c 100644
--- a/browsers/telemetry/pausing-for-captcha-solves.mdx
+++ b/browsers/telemetry/pausing-for-captcha-solves.mdx
@@ -137,7 +137,8 @@ function createCaptchaGate(kernel: KERNEL, sessionId: string, probePage: () => P
A task counts as open while it has no terminal status and is still inside its deadline — that deadline is what stops a missing `captcha_solve_result` from holding the agent forever. `until` is the only waiting primitive, and it always takes a timeout.
```ts captcha-gate.ts
- const openTasks = () => [...tasks.values()].filter((t) => !t.status && Date.now() - t.openedAt < TASK_SETTLE_MS);
+ const isOpen = (t: Outcome & { openedAt: number }) => !t.status && Date.now() - t.openedAt < TASK_SETTLE_MS;
+ const openTasks = () => [...tasks.values()].filter(isOpen);
const joinedResult = () => [...challenges].find(([id]) => joinable.has(id));
async function until(done: () => boolean, timeoutMs: number) {
@@ -150,7 +151,7 @@ function createCaptchaGate(kernel: KERNEL, sessionId: string, probePage: () => P
`resolve` settles what it can, waits a bounded interval for a challenge-level result *only* when a task actually carried a `challenge_id`, then reads the page and picks the best available source: an attributable challenge result, then any challenge result (labelled as a page observation rather than a join), then task results, then the page alone.
- `holding` and `pending` are separate on purpose. `pending` covers a terminal outcome that lands while no action is in flight — without it, a challenge result arriving between tool calls is recorded and never told to anyone.
+ `holding` and `pending` are separate on purpose. `pending` covers a terminal outcome that lands while no action is in flight — without it, a challenge result arriving between tool calls is recorded and never told to anyone. It has to catch *every* settled task, not just failures: a successful task-only result (Turnstile's usual path) and a task whose deadline lapsed with no result at all are both settled outcomes the agent needs to hear about.
```ts captcha-gate.ts
async function resolve(): Promise {
@@ -173,7 +174,7 @@ function createCaptchaGate(kernel: KERNEL, sessionId: string, probePage: () => P
/** A task the solver accepted has no terminal result yet. */
holding: () => openTasks().length > 0,
/** A terminal outcome is recorded that the agent hasn't been told about. */
- pending: () => challenges.size > 0 || [...tasks.values()].some((t) => t.status && t.status !== "success"),
+ pending: () => challenges.size > 0 || [...tasks.values()].some((t) => !isOpen(t)),
resolve,
reset: () => {
tasks.clear();
@@ -368,7 +369,8 @@ function createCaptchaGate(kernel: KERNEL, sessionId: string, probePage: () => P
}
})();
- const openTasks = () => [...tasks.values()].filter((t) => !t.status && Date.now() - t.openedAt < TASK_SETTLE_MS);
+ const isOpen = (t: Outcome & { openedAt: number }) => !t.status && Date.now() - t.openedAt < TASK_SETTLE_MS;
+ const openTasks = () => [...tasks.values()].filter(isOpen);
const joinedResult = () => [...challenges].find(([id]) => joinable.has(id));
async function until(done: () => boolean, timeoutMs: number) {
@@ -396,7 +398,7 @@ function createCaptchaGate(kernel: KERNEL, sessionId: string, probePage: () => P
/** A task the solver accepted has no terminal result yet. */
holding: () => openTasks().length > 0,
/** A terminal outcome is recorded that the agent hasn't been told about. */
- pending: () => challenges.size > 0 || [...tasks.values()].some((t) => t.status && t.status !== "success"),
+ pending: () => challenges.size > 0 || [...tasks.values()].some((t) => !isOpen(t)),
resolve,
reset: () => {
tasks.clear();
From 669abf98d7d0e67cab8f4b5a48d9bcf650963c83 Mon Sep 17 00:00:00 2001
From: dprevoznik <58714078+dprevoznik@users.noreply.github.com>
Date: Fri, 11 Sep 2026 20:49:15 +0000
Subject: [PATCH 14/20] Use tokenPresent, not just widget visibility, to decide
whether to block
A solved reCAPTCHA or hCaptcha widget commonly stays rendered on the
page, so gating solely on verdict.page.widgets.length blocks the
agent indefinitely after the challenge already produced a usable
response token. The block decision now also clears when a token is
present. Also added hCaptcha's response field name -- the probe was
only checking Turnstile's and reCAPTCHA's -- so tokenPresent can
actually see an hCaptcha solve.
---
browsers/telemetry/pausing-for-captcha-solves.mdx | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/browsers/telemetry/pausing-for-captcha-solves.mdx b/browsers/telemetry/pausing-for-captcha-solves.mdx
index 07b9369c..485eaf25 100644
--- a/browsers/telemetry/pausing-for-captcha-solves.mdx
+++ b/browsers/telemetry/pausing-for-captcha-solves.mdx
@@ -214,7 +214,7 @@ function describe(source: Verdict["source"], status: string, joined: boolean, ou
`harness.on("tool_call", …)` is awaited before the tool is dispatched, so returning from it late holds the action and returning `{ block: true, reason }` replaces it with a message the model reads. That is cheaper than aborting the turn and it stops the action *before* it reaches the page.
- The page probe is a read-only Playwright snippet. Its result is what decides whether the agent is interrupted at all — a cleared page just resumes with no extra turn.
+ The page probe is a read-only Playwright snippet. Its result is what decides whether the agent is interrupted at all — a cleared page just resumes with no extra turn. That decision checks `tokenPresent`, not just `widgets`: a solved reCAPTCHA or hCaptcha widget commonly stays rendered on the page, so a response token is the signal that the challenge is actually behind the agent, not the widget disappearing.
`gate.close()` runs before the browser is deleted, so the detached telemetry loop from step one gets torn down instead of hanging on a connection to a session that no longer exists.
@@ -231,7 +231,7 @@ return await page.evaluate(() => {
widgets: Object.entries(groups)
.filter(([, sel]) => [...document.querySelectorAll(sel)].some(onScreen))
.map(([name]) => name),
- tokenPresent: [...document.querySelectorAll('[name="cf-turnstile-response"], [name="g-recaptcha-response"]')]
+ tokenPresent: [...document.querySelectorAll('[name="cf-turnstile-response"], [name="g-recaptcha-response"], [name="h-captcha-response"]')]
.some((el) => el.value.length > 0),
};
});
@@ -275,7 +275,7 @@ async function main(): Promise {
gate.reset();
console.log(`[captcha] ${verdict.source}:${verdict.status} — ${verdict.message}`);
// Telemetry decides what to say; the page decides whether to interrupt.
- if (verdict.page.widgets.length === 0) return undefined;
+ if (verdict.page.widgets.length === 0 || verdict.page.tokenPresent) return undefined;
return { block: true, reason: verdict.message };
});
@@ -445,7 +445,7 @@ return await page.evaluate(() => {
widgets: Object.entries(groups)
.filter(([, sel]) => [...document.querySelectorAll(sel)].some(onScreen))
.map(([name]) => name),
- tokenPresent: [...document.querySelectorAll('[name="cf-turnstile-response"], [name="g-recaptcha-response"]')]
+ tokenPresent: [...document.querySelectorAll('[name="cf-turnstile-response"], [name="g-recaptcha-response"], [name="h-captcha-response"]')]
.some((el) => el.value.length > 0),
};
});
@@ -489,7 +489,7 @@ async function main(): Promise {
gate.reset();
console.log(`[captcha] ${verdict.source}:${verdict.status} — ${verdict.message}`);
// Telemetry decides what to say; the page decides whether to interrupt.
- if (verdict.page.widgets.length === 0) return undefined;
+ if (verdict.page.widgets.length === 0 || verdict.page.tokenPresent) return undefined;
return { block: true, reason: verdict.message };
});
From c1066d8d0b7f620678e04a0f8d55a57421607b76 Mon Sep 17 00:00:00 2001
From: dprevoznik <58714078+dprevoznik@users.noreply.github.com>
Date: Fri, 11 Sep 2026 20:49:54 +0000
Subject: [PATCH 15/20] Add a system-prompt fallback for when telemetry reports
nothing
The gate only interrupts the agent when telemetry gives it something
to act on, but every captcha event can be absent. Without a fallback,
a captcha telemetry misses entirely leaves the model with no guidance
at all. Added the same automatic-solve wait instruction already
documented for the no-telemetry case elsewhere, so there's a floor
under the gate rather than nothing. Verified the addition doesn't
change behavior on a normal, captcha-free run.
---
browsers/telemetry/pausing-for-captcha-solves.mdx | 12 ++++++++++--
1 file changed, 10 insertions(+), 2 deletions(-)
diff --git a/browsers/telemetry/pausing-for-captcha-solves.mdx b/browsers/telemetry/pausing-for-captcha-solves.mdx
index 485eaf25..7978d1d0 100644
--- a/browsers/telemetry/pausing-for-captcha-solves.mdx
+++ b/browsers/telemetry/pausing-for-captcha-solves.mdx
@@ -218,6 +218,8 @@ function describe(source: Verdict["source"], status: string, joined: boolean, ou
`gate.close()` runs before the browser is deleted, so the detached telemetry loop from step one gets torn down instead of hanging on a connection to a session that no longer exists.
+ The gate only speaks up when telemetry gives it something to report. Since any event can be absent, the system prompt keeps a fallback instruction for the case where nothing ever arrives — the model still knows to wait out a captcha it can see but the gate never heard about.
+
```ts captcha-gate.ts
const PROBE = `
return await page.evaluate(() => {
@@ -265,7 +267,10 @@ async function main(): Promise {
models: compiled.models,
tools: [...compiled.tools],
activeToolNames: compiled.tools.map((tool) => tool.name),
- systemPrompt: "Use the supplied browser tools to complete the task efficiently.",
+ systemPrompt:
+ "Use the supplied browser tools to complete the task efficiently. If you see a captcha or " +
+ "similar challenge and nothing has told you otherwise, wait for it to be solved automatically " +
+ "before acting.",
});
compiled.activate(harness);
@@ -479,7 +484,10 @@ async function main(): Promise {
models: compiled.models,
tools: [...compiled.tools],
activeToolNames: compiled.tools.map((tool) => tool.name),
- systemPrompt: "Use the supplied browser tools to complete the task efficiently.",
+ systemPrompt:
+ "Use the supplied browser tools to complete the task efficiently. If you see a captcha or " +
+ "similar challenge and nothing has told you otherwise, wait for it to be solved automatically " +
+ "before acting.",
});
compiled.activate(harness);
From a2af5b3a371d2f9a95c8034e9d90a3bef6c64e3e Mon Sep 17 00:00:00 2001
From: dprevoznik <58714078+dprevoznik@users.noreply.github.com>
Date: Fri, 11 Sep 2026 20:50:39 +0000
Subject: [PATCH 16/20] Actually pin pi-agent-core in the Setup install command
The Warning says to pin @earendil-works/pi-agent-core, but
"pkg@0.83.0" in a plain npm install still saves a caret range
(^0.83.0), which a patch release could still break against. Split it
into its own npm install --save-exact call, verified it now writes a
literal "0.83.0" with no operator while @onkernel/sdk stays on a
flexible range, and confirmed the resulting install still runs the
script end to end.
---
browsers/telemetry/pausing-for-captcha-solves.mdx | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/browsers/telemetry/pausing-for-captcha-solves.mdx b/browsers/telemetry/pausing-for-captcha-solves.mdx
index 7978d1d0..661b23b9 100644
--- a/browsers/telemetry/pausing-for-captcha-solves.mdx
+++ b/browsers/telemetry/pausing-for-captcha-solves.mdx
@@ -32,7 +32,8 @@ Pin `@earendil-works/pi-agent-core` to the version `@onkernel/browser-loop` depe
```bash
-npm install @onkernel/browser-loop @onkernel/sdk @earendil-works/pi-agent-core@0.83.0 tsx
+npm install @onkernel/browser-loop @onkernel/sdk tsx
+npm install @earendil-works/pi-agent-core@0.83.0 --save-exact
npm pkg set type=module
```
From 778f23bcfb7e7b3b60e6ded8e24af3babdd9bc00 Mon Sep 17 00:00:00 2001
From: dprevoznik <58714078+dprevoznik@users.noreply.github.com>
Date: Fri, 11 Sep 2026 20:51:11 +0000
Subject: [PATCH 17/20] Document that press-and-hold challenges never actually
hold the agent
Press-and-hold emits captcha_challenge_result like reCAPTCHA and
hCaptcha, but PROBE's selector groups have no entry for it, so
widgets always comes back empty for it and the tool-call handler lets
the agent proceed even while the challenge is genuinely still open.
Documented the gap explicitly rather than leaving it implied by the
generic "best-effort" bullet, since it's a real class of false
negative, not just an edge case of missed markup.
---
browsers/telemetry/pausing-for-captcha-solves.mdx | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/browsers/telemetry/pausing-for-captcha-solves.mdx b/browsers/telemetry/pausing-for-captcha-solves.mdx
index 661b23b9..f725ad7d 100644
--- a/browsers/telemetry/pausing-for-captcha-solves.mdx
+++ b/browsers/telemetry/pausing-for-captcha-solves.mdx
@@ -533,7 +533,8 @@ Durations come straight from each event's `duration_ms`, which is authoritative;
- **`TASK_SETTLE_MS` and `CHALLENGE_GRACE_MS` are the safety net.** Events can be absent, so both waits are bounded and the gate falls through to the page rather than stalling. Raise `TASK_SETTLE_MS` if your solves routinely run longer than 20s.
- **Challenge results aren't emitted for every widget type.** Turnstile, for instance, reports task events only, so the task-plus-page path is the one that runs there.
- **One episode at a time.** The gate resets after each verdict; overlapping visible challenges from the same provider aren't split apart, and the telemetry docs note Kernel's own event model can't always attribute a result in that case either.
-- **The page probe is best-effort.** It reads the DOM for known widget markers and a response token. A site that renders its challenge somewhere those selectors miss will fall back to the telemetry verdict alone.
+- **The page probe is best-effort.** It reads the DOM for known widget markers and a response token. A site that renders its challenge somewhere those selectors miss won't be held past the logged message alone.
+- **Press-and-hold has no selector at all.** It emits `captcha_challenge_result` telemetry like reCAPTCHA and hCaptcha, but `PROBE`'s `groups` has no entry for it, so `widgets` never reports one -- the tool-call handler treats that as clear and lets the agent proceed even while the challenge is still open. Add a selector for your target's specific implementation if you need it to actually hold.
## Next steps
From f8d982b508f2a6304816d0e142b12b5d3aeb0ef0 Mon Sep 17 00:00:00 2001
From: dprevoznik <58714078+dprevoznik@users.noreply.github.com>
Date: Fri, 11 Sep 2026 21:13:46 +0000
Subject: [PATCH 18/20] Don't discard a still-open follow-up task on reset
reset() unconditionally cleared every task, so a follow-up solver
task that started during resolve()'s challenge-grace wait (only
watching for captcha_challenge_result there, not new tasks) got wiped
out while still genuinely in flight -- holding() went false right
after, letting the agent act while Kernel's solver was still working.
Reproduced with a scripted case (a task still open, no covering
challenge result, at the moment reset() runs) showing holding()
incorrectly flip to false; fixed by only clearing a task once
something actually covers it -- its own terminal status, or a
challenge result for its episode. Verified the fix against that case
and a regression case where a challenge result legitimately does
cover a leftover open task (multi-task-same-challenge), which still
clears correctly.
---
.../telemetry/pausing-for-captcha-solves.mdx | 28 ++++++++++++++++---
1 file changed, 24 insertions(+), 4 deletions(-)
diff --git a/browsers/telemetry/pausing-for-captcha-solves.mdx b/browsers/telemetry/pausing-for-captcha-solves.mdx
index f725ad7d..10f61764 100644
--- a/browsers/telemetry/pausing-for-captcha-solves.mdx
+++ b/browsers/telemetry/pausing-for-captcha-solves.mdx
@@ -154,6 +154,8 @@ function createCaptchaGate(kernel: KERNEL, sessionId: string, probePage: () => P
`holding` and `pending` are separate on purpose. `pending` covers a terminal outcome that lands while no action is in flight — without it, a challenge result arriving between tool calls is recorded and never told to anyone. It has to catch *every* settled task, not just failures: a successful task-only result (Turnstile's usual path) and a task whose deadline lapsed with no result at all are both settled outcomes the agent needs to hear about.
+ `reset` only clears a task once something actually covers it — a terminal status of its own, or a challenge result for the episode it belongs to. A task the solver hasn't finished with survives, so `holding` stays true and the next tool call keeps waiting on it instead of losing track of a solve that's still in progress.
+
```ts captcha-gate.ts
async function resolve(): Promise {
// A challenge result covers every task under it, so stop waiting once one lands.
@@ -178,9 +180,18 @@ function createCaptchaGate(kernel: KERNEL, sessionId: string, probePage: () => P
pending: () => challenges.size > 0 || [...tasks.values()].some((t) => !isOpen(t)),
resolve,
reset: () => {
- tasks.clear();
+ // A challenge result covers every task under it, open or not, so an
+ // open task isn't preserved once its challenge already resolved --
+ // only when there's no other signal covering it yet, meaning the
+ // solver may still be working via a follow-up task resolve() hasn't
+ // seen close out.
+ const covered = Boolean(joinedResult());
+ for (const [id, t] of tasks) {
+ if (isOpen(t) && !covered) continue;
+ tasks.delete(id);
+ }
challenges.clear();
- joinable.clear();
+ if (openTasks().length === 0) joinable.clear();
},
/** Stop the telemetry stream. Call before deleting the browser. */
close: () => streamController?.abort(),
@@ -407,9 +418,18 @@ function createCaptchaGate(kernel: KERNEL, sessionId: string, probePage: () => P
pending: () => challenges.size > 0 || [...tasks.values()].some((t) => !isOpen(t)),
resolve,
reset: () => {
- tasks.clear();
+ // A challenge result covers every task under it, open or not, so an
+ // open task isn't preserved once its challenge already resolved --
+ // only when there's no other signal covering it yet, meaning the
+ // solver may still be working via a follow-up task resolve() hasn't
+ // seen close out.
+ const covered = Boolean(joinedResult());
+ for (const [id, t] of tasks) {
+ if (isOpen(t) && !covered) continue;
+ tasks.delete(id);
+ }
challenges.clear();
- joinable.clear();
+ if (openTasks().length === 0) joinable.clear();
},
/** Stop the telemetry stream. Call before deleting the browser. */
close: () => streamController?.abort(),
From e394a41f86585808230ff4c191063d3d99d2873f Mon Sep 17 00:00:00 2001
From: dprevoznik <58714078+dprevoznik@users.noreply.github.com>
Date: Fri, 11 Sep 2026 21:15:22 +0000
Subject: [PATCH 19/20] Ignore late-arriving events for already-reported
episodes
reset() cleared every task_id and challenge_id it consumed, so a
captcha_solve_started that arrives after its own result -- delivery
is unordered, this is explicitly possible -- gets treated as a
brand-new task in the now-empty map, and the next tool call holds for
a full TASK_SETTLE_MS waiting on a result that was already seen and
reported. Added a settled set of IDs reset() has cleared; a late
event matching one is dropped instead of reopening it. Reproduced the
phantom-reopen with a scripted late-duplicate-start case and verified
it's now ignored, plus regression-checked the covered-leftover-task
and task-only-success cases.
---
.../telemetry/pausing-for-captcha-solves.mdx | 26 ++++++++++++++-----
1 file changed, 19 insertions(+), 7 deletions(-)
diff --git a/browsers/telemetry/pausing-for-captcha-solves.mdx b/browsers/telemetry/pausing-for-captcha-solves.mdx
index 10f61764..c87376c6 100644
--- a/browsers/telemetry/pausing-for-captcha-solves.mdx
+++ b/browsers/telemetry/pausing-for-captcha-solves.mdx
@@ -53,7 +53,7 @@ The four snippets below are one file, `captcha-gate.ts`, in order.
Three maps, one per thing the telemetry can tell you, all keyed so nothing depends on arrival order. `joinable` is the important one: it holds only the `challenge_id`s that actually appeared on a task event, which are the only ones a challenge result may be attributed to.
- A task record created by a *result* already has a status, so a `captcha_solve_started` that arrives afterwards finds a closed task and leaves it closed.
+ A task record created by a *result* already has a status, so a `captcha_solve_started` that arrives afterwards finds a closed task and leaves it closed. That only works while the task is still in `tasks`, though — once `reset` has cleared it, a late start for the same `task_id` would otherwise look like a brand-new one. `settled` remembers which IDs `reset` already cleared, so a late arrival for one is dropped instead of reopening an episode the agent has already been told about.
The stream's `AbortController` is saved so the gate can close it later — the loop runs detached from the caller, and leaving it open past the browser's deletion keeps the process alive waiting on a connection that will never produce another event.
@@ -100,6 +100,10 @@ function createCaptchaGate(kernel: KERNEL, sessionId: string, probePage: () => P
const challenges = new Map();
// Only challenge_ids that appeared on a task event may be attributed to it.
const joinable = new Set();
+ // task_ids and challenge_ids already reported and cleared by reset(), so a
+ // late-arriving duplicate for one of them -- delivery is unordered, a start
+ // can arrive after its own result -- isn't mistaken for a new episode.
+ const settled = new Set();
let streamController: AbortController | undefined;
void (async () => {
@@ -108,21 +112,21 @@ function createCaptchaGate(kernel: KERNEL, sessionId: string, probePage: () => P
for await (const { event } of stream) {
if (event.category !== "captcha") continue;
- if (event.type === "captcha_solve_started" && event.data?.task_id) {
+ if (event.type === "captcha_solve_started" && event.data?.task_id && !settled.has(event.data.task_id)) {
// A result for an unseen task lands already closed, so a start that
// arrives after its own result never reopens it.
const task = tasks.get(event.data.task_id) ?? { openedAt: Date.now() };
task.captchaType = event.data.captcha_type ?? task.captchaType;
tasks.set(event.data.task_id, task);
if (event.data.challenge_id) joinable.add(event.data.challenge_id);
- } else if (event.type === "captcha_solve_result" && event.data?.task_id) {
+ } else if (event.type === "captcha_solve_result" && event.data?.task_id && !settled.has(event.data.task_id)) {
const task = tasks.get(event.data.task_id) ?? { openedAt: Date.now() };
task.status = event.data.status;
task.durationMs = event.data.duration_ms;
task.captchaType = event.data.captcha_type ?? task.captchaType;
tasks.set(event.data.task_id, task);
if (event.data.challenge_id) joinable.add(event.data.challenge_id);
- } else if (event.type === "captcha_challenge_result" && event.data?.challenge_id) {
+ } else if (event.type === "captcha_challenge_result" && event.data?.challenge_id && !settled.has(event.data.challenge_id)) {
challenges.set(event.data.challenge_id, {
status: event.data.status,
durationMs: event.data.duration_ms,
@@ -188,8 +192,10 @@ function createCaptchaGate(kernel: KERNEL, sessionId: string, probePage: () => P
const covered = Boolean(joinedResult());
for (const [id, t] of tasks) {
if (isOpen(t) && !covered) continue;
+ settled.add(id);
tasks.delete(id);
}
+ for (const id of challenges.keys()) settled.add(id);
challenges.clear();
if (openTasks().length === 0) joinable.clear();
},
@@ -354,6 +360,10 @@ function createCaptchaGate(kernel: KERNEL, sessionId: string, probePage: () => P
const challenges = new Map();
// Only challenge_ids that appeared on a task event may be attributed to it.
const joinable = new Set();
+ // task_ids and challenge_ids already reported and cleared by reset(), so a
+ // late-arriving duplicate for one of them -- delivery is unordered, a start
+ // can arrive after its own result -- isn't mistaken for a new episode.
+ const settled = new Set();
let streamController: AbortController | undefined;
void (async () => {
@@ -362,21 +372,21 @@ function createCaptchaGate(kernel: KERNEL, sessionId: string, probePage: () => P
for await (const { event } of stream) {
if (event.category !== "captcha") continue;
- if (event.type === "captcha_solve_started" && event.data?.task_id) {
+ if (event.type === "captcha_solve_started" && event.data?.task_id && !settled.has(event.data.task_id)) {
// A result for an unseen task lands already closed, so a start that
// arrives after its own result never reopens it.
const task = tasks.get(event.data.task_id) ?? { openedAt: Date.now() };
task.captchaType = event.data.captcha_type ?? task.captchaType;
tasks.set(event.data.task_id, task);
if (event.data.challenge_id) joinable.add(event.data.challenge_id);
- } else if (event.type === "captcha_solve_result" && event.data?.task_id) {
+ } else if (event.type === "captcha_solve_result" && event.data?.task_id && !settled.has(event.data.task_id)) {
const task = tasks.get(event.data.task_id) ?? { openedAt: Date.now() };
task.status = event.data.status;
task.durationMs = event.data.duration_ms;
task.captchaType = event.data.captcha_type ?? task.captchaType;
tasks.set(event.data.task_id, task);
if (event.data.challenge_id) joinable.add(event.data.challenge_id);
- } else if (event.type === "captcha_challenge_result" && event.data?.challenge_id) {
+ } else if (event.type === "captcha_challenge_result" && event.data?.challenge_id && !settled.has(event.data.challenge_id)) {
challenges.set(event.data.challenge_id, {
status: event.data.status,
durationMs: event.data.duration_ms,
@@ -426,8 +436,10 @@ function createCaptchaGate(kernel: KERNEL, sessionId: string, probePage: () => P
const covered = Boolean(joinedResult());
for (const [id, t] of tasks) {
if (isOpen(t) && !covered) continue;
+ settled.add(id);
tasks.delete(id);
}
+ for (const id of challenges.keys()) settled.add(id);
challenges.clear();
if (openTasks().length === 0) joinable.clear();
},
From 62834048ea92abec0866a026ba9116b11ec40c39 Mon Sep 17 00:00:00 2001
From: dprevoznik <58714078+dprevoznik@users.noreply.github.com>
Date: Fri, 11 Sep 2026 21:29:57 +0000
Subject: [PATCH 20/20] Give settled IDs a TTL instead of blocking them forever
reset() permanently blacklisted every reported task_id and
challenge_id, but Kernel can reuse a challenge_id when an episode
continues across a page reload -- a genuinely new outcome for one
would be silently dropped forever, and a new task under it would
still cost a full CHALLENGE_GRACE_MS wait for a result that could
never be recorded. Replaced the permanent set with a map of when each
ID was settled; a duplicate is only dropped within TASK_SETTLE_MS of
that, then treated as fresh again. Verified the original short-window
duplicate is still blocked, a reload's new outcome past the window is
no longer dropped, and reran the existing regression scenarios
(task-only success, follow-up task survives, covered leftover task).
Documented the resulting edge case (a reload within TASK_SETTLE_MS of
the prior verdict can still be missed) in Limits.
---
.../telemetry/pausing-for-captcha-solves.mdx | 55 ++++++++++++-------
1 file changed, 36 insertions(+), 19 deletions(-)
diff --git a/browsers/telemetry/pausing-for-captcha-solves.mdx b/browsers/telemetry/pausing-for-captcha-solves.mdx
index c87376c6..19ff0f85 100644
--- a/browsers/telemetry/pausing-for-captcha-solves.mdx
+++ b/browsers/telemetry/pausing-for-captcha-solves.mdx
@@ -53,7 +53,7 @@ The four snippets below are one file, `captcha-gate.ts`, in order.
Three maps, one per thing the telemetry can tell you, all keyed so nothing depends on arrival order. `joinable` is the important one: it holds only the `challenge_id`s that actually appeared on a task event, which are the only ones a challenge result may be attributed to.
- A task record created by a *result* already has a status, so a `captcha_solve_started` that arrives afterwards finds a closed task and leaves it closed. That only works while the task is still in `tasks`, though — once `reset` has cleared it, a late start for the same `task_id` would otherwise look like a brand-new one. `settled` remembers which IDs `reset` already cleared, so a late arrival for one is dropped instead of reopening an episode the agent has already been told about.
+ A task record created by a *result* already has a status, so a `captcha_solve_started` that arrives afterwards finds a closed task and leaves it closed. That only works while the task is still in `tasks`, though — once `reset` has cleared it, a late start for the same `task_id` would otherwise look like a brand-new one. `settledAt` remembers when `reset` cleared each ID, so a late arrival within `TASK_SETTLE_MS` of that is dropped instead of reopening an episode the agent has already been told about. Past that window the same ID is treated as fresh again, since Kernel can reuse a `challenge_id` when an episode continues across a page reload.
The stream's `AbortController` is saved so the gate can close it later — the loop runs detached from the caller, and leaving it open past the browser's deletion keeps the process alive waiting on a connection that will never produce another event.
@@ -100,10 +100,17 @@ function createCaptchaGate(kernel: KERNEL, sessionId: string, probePage: () => P
const challenges = new Map();
// Only challenge_ids that appeared on a task event may be attributed to it.
const joinable = new Set();
- // task_ids and challenge_ids already reported and cleared by reset(), so a
- // late-arriving duplicate for one of them -- delivery is unordered, a start
- // can arrive after its own result -- isn't mistaken for a new episode.
- const settled = new Set();
+ // task_ids and challenge_ids reported and cleared by reset(), each stamped
+ // with when that happened. A late-arriving duplicate within TASK_SETTLE_MS
+ // of that -- delivery is unordered, a start can arrive after its own
+ // result -- is dropped instead of reopening an already-told episode. Past
+ // that window it's treated as fresh, since Kernel can reuse the same
+ // challenge_id when an episode continues across a page reload.
+ const settledAt = new Map();
+ const isSettled = (id: string) => {
+ const at = settledAt.get(id);
+ return at !== undefined && Date.now() - at < TASK_SETTLE_MS;
+ };
let streamController: AbortController | undefined;
void (async () => {
@@ -112,21 +119,21 @@ function createCaptchaGate(kernel: KERNEL, sessionId: string, probePage: () => P
for await (const { event } of stream) {
if (event.category !== "captcha") continue;
- if (event.type === "captcha_solve_started" && event.data?.task_id && !settled.has(event.data.task_id)) {
+ if (event.type === "captcha_solve_started" && event.data?.task_id && !isSettled(event.data.task_id)) {
// A result for an unseen task lands already closed, so a start that
// arrives after its own result never reopens it.
const task = tasks.get(event.data.task_id) ?? { openedAt: Date.now() };
task.captchaType = event.data.captcha_type ?? task.captchaType;
tasks.set(event.data.task_id, task);
if (event.data.challenge_id) joinable.add(event.data.challenge_id);
- } else if (event.type === "captcha_solve_result" && event.data?.task_id && !settled.has(event.data.task_id)) {
+ } else if (event.type === "captcha_solve_result" && event.data?.task_id && !isSettled(event.data.task_id)) {
const task = tasks.get(event.data.task_id) ?? { openedAt: Date.now() };
task.status = event.data.status;
task.durationMs = event.data.duration_ms;
task.captchaType = event.data.captcha_type ?? task.captchaType;
tasks.set(event.data.task_id, task);
if (event.data.challenge_id) joinable.add(event.data.challenge_id);
- } else if (event.type === "captcha_challenge_result" && event.data?.challenge_id && !settled.has(event.data.challenge_id)) {
+ } else if (event.type === "captcha_challenge_result" && event.data?.challenge_id && !isSettled(event.data.challenge_id)) {
challenges.set(event.data.challenge_id, {
status: event.data.status,
durationMs: event.data.duration_ms,
@@ -190,12 +197,13 @@ function createCaptchaGate(kernel: KERNEL, sessionId: string, probePage: () => P
// solver may still be working via a follow-up task resolve() hasn't
// seen close out.
const covered = Boolean(joinedResult());
+ const now = Date.now();
for (const [id, t] of tasks) {
if (isOpen(t) && !covered) continue;
- settled.add(id);
+ settledAt.set(id, now);
tasks.delete(id);
}
- for (const id of challenges.keys()) settled.add(id);
+ for (const id of challenges.keys()) settledAt.set(id, now);
challenges.clear();
if (openTasks().length === 0) joinable.clear();
},
@@ -360,10 +368,17 @@ function createCaptchaGate(kernel: KERNEL, sessionId: string, probePage: () => P
const challenges = new Map();
// Only challenge_ids that appeared on a task event may be attributed to it.
const joinable = new Set();
- // task_ids and challenge_ids already reported and cleared by reset(), so a
- // late-arriving duplicate for one of them -- delivery is unordered, a start
- // can arrive after its own result -- isn't mistaken for a new episode.
- const settled = new Set();
+ // task_ids and challenge_ids reported and cleared by reset(), each stamped
+ // with when that happened. A late-arriving duplicate within TASK_SETTLE_MS
+ // of that -- delivery is unordered, a start can arrive after its own
+ // result -- is dropped instead of reopening an already-told episode. Past
+ // that window it's treated as fresh, since Kernel can reuse the same
+ // challenge_id when an episode continues across a page reload.
+ const settledAt = new Map();
+ const isSettled = (id: string) => {
+ const at = settledAt.get(id);
+ return at !== undefined && Date.now() - at < TASK_SETTLE_MS;
+ };
let streamController: AbortController | undefined;
void (async () => {
@@ -372,21 +387,21 @@ function createCaptchaGate(kernel: KERNEL, sessionId: string, probePage: () => P
for await (const { event } of stream) {
if (event.category !== "captcha") continue;
- if (event.type === "captcha_solve_started" && event.data?.task_id && !settled.has(event.data.task_id)) {
+ if (event.type === "captcha_solve_started" && event.data?.task_id && !isSettled(event.data.task_id)) {
// A result for an unseen task lands already closed, so a start that
// arrives after its own result never reopens it.
const task = tasks.get(event.data.task_id) ?? { openedAt: Date.now() };
task.captchaType = event.data.captcha_type ?? task.captchaType;
tasks.set(event.data.task_id, task);
if (event.data.challenge_id) joinable.add(event.data.challenge_id);
- } else if (event.type === "captcha_solve_result" && event.data?.task_id && !settled.has(event.data.task_id)) {
+ } else if (event.type === "captcha_solve_result" && event.data?.task_id && !isSettled(event.data.task_id)) {
const task = tasks.get(event.data.task_id) ?? { openedAt: Date.now() };
task.status = event.data.status;
task.durationMs = event.data.duration_ms;
task.captchaType = event.data.captcha_type ?? task.captchaType;
tasks.set(event.data.task_id, task);
if (event.data.challenge_id) joinable.add(event.data.challenge_id);
- } else if (event.type === "captcha_challenge_result" && event.data?.challenge_id && !settled.has(event.data.challenge_id)) {
+ } else if (event.type === "captcha_challenge_result" && event.data?.challenge_id && !isSettled(event.data.challenge_id)) {
challenges.set(event.data.challenge_id, {
status: event.data.status,
durationMs: event.data.duration_ms,
@@ -434,12 +449,13 @@ function createCaptchaGate(kernel: KERNEL, sessionId: string, probePage: () => P
// solver may still be working via a follow-up task resolve() hasn't
// seen close out.
const covered = Boolean(joinedResult());
+ const now = Date.now();
for (const [id, t] of tasks) {
if (isOpen(t) && !covered) continue;
- settled.add(id);
+ settledAt.set(id, now);
tasks.delete(id);
}
- for (const id of challenges.keys()) settled.add(id);
+ for (const id of challenges.keys()) settledAt.set(id, now);
challenges.clear();
if (openTasks().length === 0) joinable.clear();
},
@@ -567,6 +583,7 @@ Durations come straight from each event's `duration_ms`, which is authoritative;
- **One episode at a time.** The gate resets after each verdict; overlapping visible challenges from the same provider aren't split apart, and the telemetry docs note Kernel's own event model can't always attribute a result in that case either.
- **The page probe is best-effort.** It reads the DOM for known widget markers and a response token. A site that renders its challenge somewhere those selectors miss won't be held past the logged message alone.
- **Press-and-hold has no selector at all.** It emits `captcha_challenge_result` telemetry like reCAPTCHA and hCaptcha, but `PROBE`'s `groups` has no entry for it, so `widgets` never reports one -- the tool-call handler treats that as clear and lets the agent proceed even while the challenge is still open. Add a selector for your target's specific implementation if you need it to actually hold.
+- **A reload within `TASK_SETTLE_MS` of the last verdict can still be missed.** `settledAt` treats an ID as fresh again once that window passes, so a genuinely new outcome for a reused `challenge_id` eventually gets through -- but if the reload happens sooner than that, its result is indistinguishable from a stale duplicate and gets dropped.
## Next steps