Add deterministic Ask-to-Act handoff action by 16Miku · Pull Request #3030 · webbrain-one/webbrain

16Miku · 2026-09-13T13:38:00Z

Summary

Fixes #3029 by adding a structured Ask-to-Act handoff signal and a deterministic one-click transition from Ask mode to Act mode.

What changed

Add a strict mode_handoff (act / none) classifier schema, prompt, message builder, and parser.
Run a lightweight, independent handoff classification after Ask responses without modifying or embedding protocol markers in the assistant answer.
Emit an ask_mode_handoff update only for a validated act result.
Add a Switch to Act & retry action that reuses the existing retry payload and mode-switching flow.
Keep failure paths fail-closed: unsupported providers, invalid output, cancellation, and classifier errors do not block or alter the original Ask response.
Mirror the implementation across Chrome and Firefox, including locale keys and CSS.
Add Chrome/Firefox regression coverage for strict parsing, mode guards, skip conditions, and silent failure behavior.

Validation

node test/run.js — 2306 passed, 0 failed
JavaScript syntax checks — all changed JavaScript files passed node --check

Manual verification (WebBrain Compass)

Test page:
https://the-internet.herokuapp.com/login

Request:

Enter username test-user and password test-pass, but do not click Login or submit the form.

Observed:

Ask mode correctly remained read-only and explained that Act mode was required.
The Switch to Act & retry button appeared below the answer.
Clicking the button switched to Act mode and automatically retried the original request.
The Act planner set requires_state_change: true and requires_submission: false.
Both fields were filled successfully and verified.
The Login button was not clicked and the form was not submitted.

An earlier run with a local OpenAI-compatible proxy and kimi-k2.6 did not show the button; the main Ask response remained correct and the fail-closed fallback applied. The Compass end-to-end flow is verified; provider-specific structured-output compatibility may warrant follow-up.

Scope

No changes were made to the main branch, no upstream branches were force-updated, and no submission action is introduced by the handoff button itself.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

vercel · 2026-09-13T13:38:04Z

@16Miku is attempting to deploy a commit to the esokullu's projects Team on Vercel.

A member of the Team first needs to authorize it.

Copilot

🟡 Changes recommended

Attachment-backed retries lose their files, and fallback parsing does not enforce the declared strict schema.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Adds a structured Ask-to-Act classifier and one-click Act-mode retry flow across both browser builds.

Changes:

Adds handoff schema, prompt, parser, and post-response classification.
Adds the localized handoff button, styling, and retry integration.
Adds mirrored regression coverage and test isolation.

File summaries

File	Description
`test/run.js`	Adds classifier and integration regression coverage.
`src/chrome/src/agent/agent.js`	Runs and emits Ask-mode handoff classification.
`src/chrome/src/agent/planner.js`	Defines the handoff schema, prompt, builder, and parser.
`src/chrome/src/ui/sidepanel.js`	Renders and binds the Act retry action.
`src/chrome/styles/sidepanel.css`	Styles the handoff button.
`src/chrome/src/ui/locales/en.js`	Adds English handoff strings.
`src/chrome/src/ui/locales/ar.js`	Adds Arabic locale keys.
`src/chrome/src/ui/locales/bn.js`	Adds Bengali locale keys.
`src/chrome/src/ui/locales/de.js`	Adds German locale keys.
`src/chrome/src/ui/locales/es.js`	Adds Spanish locale keys.
`src/chrome/src/ui/locales/fa.js`	Adds Persian locale keys.
`src/chrome/src/ui/locales/fr.js`	Adds French locale keys.
`src/chrome/src/ui/locales/he.js`	Adds Hebrew locale keys.
`src/chrome/src/ui/locales/hi.js`	Adds Hindi locale keys.
`src/chrome/src/ui/locales/id.js`	Adds Indonesian locale keys.
`src/chrome/src/ui/locales/ja.js`	Adds Japanese locale keys.
`src/chrome/src/ui/locales/ko.js`	Adds Korean locale keys.
`src/chrome/src/ui/locales/ms.js`	Adds Malay locale keys.
`src/chrome/src/ui/locales/nl.js`	Adds Dutch locale keys.
`src/chrome/src/ui/locales/pl.js`	Adds Polish locale keys.
`src/chrome/src/ui/locales/pt.js`	Adds Portuguese locale keys.
`src/chrome/src/ui/locales/ru.js`	Adds Russian locale keys.
`src/chrome/src/ui/locales/th.js`	Adds Thai locale keys.
`src/chrome/src/ui/locales/tl.js`	Adds Tagalog locale keys.
`src/chrome/src/ui/locales/tr.js`	Adds Turkish locale keys.
`src/chrome/src/ui/locales/uk.js`	Adds Ukrainian locale keys.
`src/chrome/src/ui/locales/vi.js`	Adds Vietnamese locale keys.
`src/chrome/src/ui/locales/zh.js`	Adds Chinese locale keys.
`src/firefox/src/agent/agent.js`	Mirrors handoff classification and emission.
`src/firefox/src/agent/planner.js`	Mirrors handoff schema and parsing.
`src/firefox/src/ui/sidepanel.js`	Mirrors the Act retry action.
`src/firefox/styles/sidepanel.css`	Mirrors handoff button styling.
`src/firefox/src/ui/locales/en.js`	Adds English handoff strings.
`src/firefox/src/ui/locales/ar.js`	Adds Arabic locale keys.
`src/firefox/src/ui/locales/bn.js`	Adds Bengali locale keys.
`src/firefox/src/ui/locales/de.js`	Adds German locale keys.
`src/firefox/src/ui/locales/es.js`	Adds Spanish locale keys.
`src/firefox/src/ui/locales/fa.js`	Adds Persian locale keys.
`src/firefox/src/ui/locales/fr.js`	Adds French locale keys.
`src/firefox/src/ui/locales/he.js`	Adds Hebrew locale keys.
`src/firefox/src/ui/locales/hi.js`	Adds Hindi locale keys.
`src/firefox/src/ui/locales/id.js`	Adds Indonesian locale keys.
`src/firefox/src/ui/locales/ja.js`	Adds Japanese locale keys.
`src/firefox/src/ui/locales/ko.js`	Adds Korean locale keys.
`src/firefox/src/ui/locales/ms.js`	Adds Malay locale keys.
`src/firefox/src/ui/locales/nl.js`	Adds Dutch locale keys.
`src/firefox/src/ui/locales/pl.js`	Adds Polish locale keys.
`src/firefox/src/ui/locales/pt.js`	Adds Portuguese locale keys.
`src/firefox/src/ui/locales/ru.js`	Adds Russian locale keys.
`src/firefox/src/ui/locales/th.js`	Adds Thai locale keys.
`src/firefox/src/ui/locales/tl.js`	Adds Tagalog locale keys.
`src/firefox/src/ui/locales/tr.js`	Adds Turkish locale keys.
`src/firefox/src/ui/locales/uk.js`	Adds Ukrainian locale keys.
`src/firefox/src/ui/locales/vi.js`	Adds Vietnamese locale keys.
`src/firefox/src/ui/locales/zh.js`	Adds Chinese locale keys.

Review details

Files not reviewed (5)

src/chrome/src/ui/locales/bn.js: Generated file
src/chrome/src/ui/locales/hi.js: Generated file
src/chrome/src/ui/locales/ru.js: Generated file
src/chrome/src/ui/locales/th.js: Generated file
src/chrome/src/ui/locales/uk.js: Generated file

Files reviewed: 50/55 changed files
Comments generated: 4
Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

16Miku · 2026-09-13T14:02:12Z

Addressed the review findings in b1878271:

Preserved the complete retry payload, including attachments, for Ask-to-Act handoffs. The active request payload is preferred, with an assistant-message fallback, and handoff retry attachment cleanup is included.
Made the Chrome and Firefox fallback parsers enforce the declared strict schema: exactly one mode_handoff property whose value is act or none.
Updated the mirrored regression coverage and the isolated sidepanel test fixture.

Validation:

node test/run.js: 2307 passed, 0 failed
Modified JavaScript files pass node --check
git diff --check: clean
Official smoke check: passed

The Vercel check remains blocked by external deployment authorization, unchanged from the earlier PR run.

Copilot

🟡 Changes recommended

The classifier can delay completed Ask turns, and attachment payloads are retained without a bounded lifetime.

Get a fresh assessment by requesting another Copilot review.

Review details

Files not reviewed (5)

src/chrome/src/ui/locales/bn.js: Generated file
src/chrome/src/ui/locales/hi.js: Generated file
src/chrome/src/ui/locales/ru.js: Generated file
src/chrome/src/ui/locales/th.js: Generated file
src/chrome/src/ui/locales/uk.js: Generated file

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

src/chrome/src/agent/agent.js:19980

The classifier call has no short, classifier-specific deadline, while both processMessage paths await this method before returning. A slow or unsupported provider can therefore keep the completed Ask turn in the processing state for the normal LLM request timeout (or longer while reading a response body), preventing the user from sending another message and delaying non-streamed results. Please either run this best-effort classification outside the original request's critical path or cap it with a small abortable timeout.
src/firefox/src/agent/agent.js:17764
The classifier call has no short, classifier-specific deadline, while both processMessage paths await this method before returning. A slow or unsupported provider can therefore keep the completed Ask turn in the processing state for the normal LLM request timeout (or longer while reading a response body), preventing the user from sending another message and delaying non-streamed results. Please either run this best-effort classification outside the original request's critical path or cap it with a small abortable timeout.

Files reviewed: 50/55 changed files
Comments generated: 2
Review effort level: Balanced

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

16Miku · 2026-09-13T14:29:03Z

Addressed the latest review findings in db5130e:\n\n- Added a dedicated 5-second abortable deadline for the best-effort Ask-to-Act classifier in both Chrome and Firefox, so a slow provider cannot hold the completed Ask turn for the normal request timeout.\n- Added a 30-second bounded lifetime for assistant fallback retry payloads, including attachment copies, while preserving the existing button retry cleanup.\n- Added mirrored regression coverage for the deadline, timeout fail-closed behavior, attachment retention, and payload expiry.\n\nValidation:\n-
ode test/run.js: 2307 passed, 0 failed\n- Modified JavaScript files pass
ode --check\n- git diff --check: clean\n- Official smoke check for db5130e: passed\n- Vercel remains blocked by the existing external team deployment authorization requirement.

Copilot

🟡 Changes recommended

The handoff blocks completion, mishandles inherited selection scope, and bypasses Chrome streaming cost state.

Get a fresh assessment by requesting another Copilot review.

Review details

Files not reviewed (5)

src/chrome/src/ui/locales/bn.js: Generated file
src/chrome/src/ui/locales/hi.js: Generated file
src/chrome/src/ui/locales/ru.js: Generated file
src/chrome/src/ui/locales/th.js: Generated file
src/chrome/src/ui/locales/uk.js: Generated file

Suppressed comments (5)

Previously missed (3) — in code that hasn't changed since the last review.

src/chrome/src/agent/agent.js:19969

Inherited selection-grounding is missed here. Follow-up turns intentionally omit runOptions.sourceGrounding; _selectionGroundedRunOptions reconstructs it only inside the completed inner call, while the durable scope remains in selectionGroundingScopes. Such a follow-up can therefore render this Act handoff even though rejectSelectionScopedMode will refuse the button when clicked. Guard the durable scope as well.
src/firefox/src/agent/agent.js:17753
Inherited selection-grounding is missed here. Follow-up turns intentionally omit runOptions.sourceGrounding; _selectionGroundedRunOptions reconstructs it only inside the completed inner call, while the durable scope remains in selectionGroundingScopes. Such a follow-up can therefore render this Act handoff even though rejectSelectionScopedMode will refuse the button when clicked. Guard the durable scope as well.
src/chrome/src/ui/locales/es.js:178
These new strings remain English in every non-English locale in both browser trees, so localized users get an English CTA and accessible hint amid translated mode controls. The repository expects locale dictionaries to contain translated copy (see test/run.js:41555-41574); provide translations for these keys across the mirrored locale files.

src/firefox/src/agent/agent.js:34658

The streaming path also awaits the best-effort classifier before releasing the run and returning the answer, so classifier latency or the 5-second timeout keeps the completed Ask turn busy. This should be decoupled from foreground completion rather than making the optional handoff part of the critical response path.
src/chrome/src/agent/agent.js:41648
The streaming path also awaits the best-effort classifier before releasing the run and returning the answer, so classifier latency or the 5-second timeout keeps the completed Ask turn busy. This should be decoupled from foreground completion rather than making the optional handoff part of the critical response path.

Files reviewed: 50/55 changed files
Comments generated: 3
Review effort level: Balanced

Decouple best-effort handoff classification from completion, preserve cost accounting and selection-scope guards, and localize the new handoff strings across browser builds. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

16Miku · 2026-09-13T14:58:24Z

Implemented the latest review fixes in commit b7a02e19:

Detached Ask-to-Act classification from both streaming and non-streaming completion paths so optional classification cannot delay the completed Ask response.
Captured the active provider cost state before the background classifier awaits, preserving Chrome streaming cost accounting.
Added a fail-closed guard for durable inherited selection-grounding scopes.
Localized the handoff CTA and accessible hint across all existing non-English Chrome and Firefox locales.
Added regression coverage for cost-state propagation, inherited selection scope, and non-blocking completion.

Validation: node test/run.js — 2308 passed, 0 failed; git diff --check and JavaScript syntax checks passed. The Vercel authorization check remains an external deployment-permission issue; the repository smoke check previously passed.

Copilot

🟡 Changes recommended

Unescaped classifier boundaries permit prompt-structure injection, and valid short replies or off-screen actions can miss the handoff.

Get a fresh assessment by requesting another Copilot review.

Review details

Files not reviewed (5)

src/chrome/src/ui/locales/bn.js: Generated file
src/chrome/src/ui/locales/hi.js: Generated file
src/chrome/src/ui/locales/ru.js: Generated file
src/chrome/src/ui/locales/th.js: Generated file
src/chrome/src/ui/locales/uk.js: Generated file

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

src/chrome/src/agent/planner.js:817

These interpolated values can contain newlines and literal <user_request>/<assistant_answer> tags, so an attacker-controlled page title or page-derived answer can break the classifier's data boundaries. sanitizeText only strips control characters and truncates; unlike sanitizePlannerPageField above, it does not neutralize boundary markup. Encode the fields as structured JSON or strip/escape every reserved delimiter before building this classifier message.
src/firefox/src/agent/planner.js:817
These interpolated values can contain newlines and literal <user_request>/<assistant_answer> tags, so an attacker-controlled page title or page-derived answer can break the classifier's data boundaries. sanitizeText only strips control characters and truncates; unlike sanitizePlannerPageField above, it does not neutralize boundary markup. Encode the fields as structured JSON or strip/escape every reserved delimiter before building this classifier message.

Files reviewed: 50/55 changed files
Comments generated: 4
Review effort level: Balanced

+        || typeof finalResponse !== 'string'
+        || finalResponse.trim().length < 8


+  configureRetryButton(btn, retryPayload);
+  content.appendChild(btn);


+        || typeof finalResponse !== 'string'
+        || finalResponse.trim().length < 8


+  configureRetryButton(btn, retryPayload);
+  content.appendChild(btn);


Encode classifier inputs as untrusted JSON data, allow short non-empty answers, and keep asynchronously rendered handoff actions visible. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

16Miku · 2026-09-13T15:25:19Z

Implemented the latest review fixes in commit 3db17d30:

Replaced interpolated XML-like classifier sections with a structured JSON data envelope in Chrome and Firefox. The system prompt now explicitly treats arbitrary markup, delimiters, and newlines inside those values as untrusted data.
Allow non-empty short Ask answers to reach the classifier; only empty/whitespace answers are skipped.
Scroll to the live edge after asynchronously inserting the handoff button, without forcing the user away from an intentional reading position.
Added mirrored regression coverage for delimiter-containing classifier inputs, short answers, and post-insertion scrolling.

Validation: node test/run.js — 2308 passed, 0 failed; modified JavaScript passed node --check; git diff --check passed.

16Miku · 2026-09-13T16:29:05Z

Manual validation completed after the latest changes:

Basic flow: In Ask mode on https://the-internet.herokuapp.com/login, the assistant returned the handoff CTA. Clicking Switch to Act and retry changed to Act mode and filled both Username and Password. Login was not clicked and the form was not submitted.
Attachment retry: Uploaded a non-sensitive TXT attachment containing test USERNAME and PASSWORD values. The attachment was present in both the original Ask turn and the retried Act turn; Act used it to fill both fields successfully. Login was not clicked and the form was not submitted.

Automated validation remains green: node test/run.js — 2308 passed, 0 failed. The Vercel check remains an external deployment-authorization issue, not a test failure.

16Miku · 2026-09-13T17:10:42Z

Additional manual validation completed with the OpenRouter channel:

Provider/model: openrouter/free
Test page: https://the-internet.herokuapp.com/login
In Ask mode, WebBrain identified that filling the form required Act mode and offered the handoff.
After switching to Act and retrying, both set_field operations returned success: true and verified: true for Username and Password.
The Login button was not clicked and the form was not submitted.

This confirms the Ask-to-Act handoff works end-to-end with an OpenRouter model as well as the previously documented WebBrain Compass flow. The local OpenAI-compatible Proxy/Kimi observation remains a provider-specific compatibility/timeout follow-up, not a regression in the core handoff flow.

Add structured Ask-to-Act handoff signal

bd1f223

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot AI balanced review requested due to automatic review settings September 13, 2026 13:38

Copilot started reviewing on behalf of 16Miku September 13, 2026 13:38 View session

Copilot AI reviewed Sep 13, 2026

View reviewed changes

Comment thread src/chrome/src/agent/planner.js

Comment thread src/chrome/src/ui/sidepanel.js Outdated

Comment thread src/firefox/src/agent/planner.js

Comment thread src/firefox/src/ui/sidepanel.js Outdated

Fix Ask-to-Act retry payload handling

b187827

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

16Miku requested a balanced review from Copilot September 13, 2026 14:10

Copilot started reviewing on behalf of 16Miku September 13, 2026 14:10 View session

Copilot AI reviewed Sep 13, 2026

View reviewed changes

Comment thread src/chrome/src/ui/sidepanel.js Outdated

Comment thread src/firefox/src/ui/sidepanel.js Outdated

Bound Ask-to-Act classifier and retry payloads

db5130e

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

16Miku requested a balanced review from Copilot September 13, 2026 14:34

Copilot started reviewing on behalf of 16Miku September 13, 2026 14:35 View session

Copilot AI reviewed Sep 13, 2026

View reviewed changes

Comment thread src/chrome/src/agent/agent.js Outdated

Comment thread src/chrome/src/agent/agent.js Outdated

Comment thread src/firefox/src/agent/agent.js Outdated

Fix Ask-to-Act handoff lifecycle

b7a02e1

Decouple best-effort handoff classification from completion, preserve cost accounting and selection-scope guards, and localize the new handoff strings across browser builds. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

16Miku requested a balanced review from Copilot September 13, 2026 14:58

Copilot started reviewing on behalf of 16Miku September 13, 2026 14:59 View session

Copilot AI reviewed Sep 13, 2026

View reviewed changes

Harden Ask-to-Act handoff classification

3db17d3

Encode classifier inputs as untrusted JSON data, allow short non-empty answers, and keep asynchronously rendered handoff actions visible. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

		\|\| typeof finalResponse !== 'string'
		\|\| finalResponse.trim().length < 8

		configureRetryButton(btn, retryPayload);
		content.appendChild(btn);

Uh oh!

Conversation

16Miku commented Sep 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What changed

Validation

Manual verification (WebBrain Compass)

Scope

Uh oh!

vercel Bot commented Sep 13, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Files not reviewed (5)

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

16Miku commented Sep 13, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Files not reviewed (5)

Suppressed comments (2)

Uh oh!

Uh oh!

Uh oh!

16Miku commented Sep 13, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Files not reviewed (5)

Suppressed comments (5)

Uh oh!

Uh oh!

Uh oh!

Uh oh!

16Miku commented Sep 13, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Files not reviewed (5)

Suppressed comments (2)

Uh oh!

16Miku commented Sep 13, 2026

Uh oh!

16Miku commented Sep 13, 2026

Uh oh!

16Miku commented Sep 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

16Miku commented Sep 13, 2026 •

edited

Loading