Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/release-browser-loop.yml
Original file line number Diff line number Diff line change
Expand Up @@ -74,11 +74,11 @@ jobs:
npm init -y > /dev/null
npm install "$RUNNER_TEMP"/onkernel-browser-loop-*.tgz
cat > smoke.mjs <<'NODE'
import { compileLoopToolCatalog, formatBrowserActResult, loop } from "@onkernel/browser-loop";
import { compileLoopToolCatalog, executeBrowserRepl, formatBrowserActResult, loop } from "@onkernel/browser-loop";
import * as loopPi from "@onkernel/browser-loop/pi";

const { attach, createLoopModels, getLoopModel } = loopPi;
for (const [name, value] of Object.entries({ compileLoopToolCatalog, formatBrowserActResult, attach, createLoopModels, getLoopModel })) {
for (const [name, value] of Object.entries({ compileLoopToolCatalog, executeBrowserRepl, formatBrowserActResult, attach, createLoopModels, getLoopModel })) {
if (typeof value !== "function") {
throw new Error(`expected ${name} to be a function, got ${typeof value}`);
}
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -153,7 +153,7 @@ per-tool verdict.

1. **Execution layer** — `@onkernel/browser-loop` materializes the caller's exact
catalog over one shared resource pool and executes canonical actions through
Kernel's computer API or a raw-CDP browser executor.
Kernel's computer, Playwright, or Browser REPL APIs, or a raw-CDP browser executor.
2. **Model layer** — `@onkernel/browser-loop/pi` opens pi-ai's whole model catalog and
composes provider declarations, headers, and payload transforms around that
core. Catalog compilation is declaration-only: it never sees an executor.
Expand All @@ -163,7 +163,7 @@ per-tool verdict.
4. **Browser** — a Kernel cloud browser with optional profile and proxy. The
model requests screenshots explicitly when it needs visual feedback.

See [`docs/architecture.md`](docs/architecture.md) for the full end-to-end flow.
See [`packages/browser-loop/docs/architecture.md`](packages/browser-loop/docs/architecture.md) for the full end-to-end flow.

---

Expand Down
88 changes: 0 additions & 88 deletions docs/npm-releases.md

This file was deleted.

5 changes: 5 additions & 0 deletions packages/browser-loop/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
# Changelog

## Unreleased

- Add `loop.tools.repl()` (`kloop.repl.v1`) as a wrapper around Kernel browsers' persistent JavaScript API, including ordered text/image output and explicit REPL lifecycle metadata.
- Add the framework-neutral `executeBrowserRepl()` client and the pi extension's `repl` selector.

## 0.12.1 - 2026-09-04

- Relax the `@onkernel/sdk` dependency from an exact pin to `>=0.49.0 <1.0.0`.
Expand Down
28 changes: 24 additions & 4 deletions packages/browser-loop/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ binding; Eve and AI SDK are anticipated next.

| import | what it is |
| --- | --- |
| `@onkernel/browser-loop` | The framework-neutral core: canonical actions, the tool namespace, catalog compilation, the tool menu, and Kernel-browser execution. Core declarations (`LoopToolDeclaration`) and executables (`LoopExecutableTool`) import nothing from pi — schemas come from `typebox` directly — and a unit test enforces the boundary. |
| `@onkernel/browser-loop` | The framework-neutral core: canonical actions, the tool namespace, catalog compilation, Kernel-browser execution, and the persistent Browser REPL API wrapper. Core declarations (`LoopToolDeclaration`) and executables (`LoopExecutableTool`) import nothing from pi, and a unit test enforces the boundary. |
| `@onkernel/browser-loop/pi` | The pi binding: `attach()`, model resolution, transport derivation, provider adapters, and provider retry. |

Installing the package into pi (`pi install npm:@onkernel/browser-loop`) registers the
Expand Down Expand Up @@ -302,10 +302,28 @@ loop.tools.computer.batch({ actions: ["click", "keypress", "screenshot"] });
loop.tools.browser.batch({ actions: ["snapshot", "click", "wait_for", "text"] });

loop.tools.playwright();
loop.tools.repl();
```

A toolset carries that surface's primitives, not every tool it has: `browser_act`,
`computer_zoom`, and the batch forms are selected explicitly.
`computer_zoom`, the persistent Browser REPL, and the batch forms are selected explicitly.

`loop.tools.repl()` wraps the Kernel browser VM's `POST /repl` API. Top-level bindings
survive across calls, output remains explicitly ordered, and the VM runtime exposes
raw CDP/browser helpers, accessibility snapshots, WebMCP, `repl.help()`, and optional dynamic
imports of Patchright or Playwright:

```ts
const compiled = kb.compile({
model: "anthropic:claude-opus-5",
tools: [loop.tools.repl()],
});

// compiled.agentTools is directly usable by pi's Agent.
```

See [Architecture: Browser REPL API wrapper](docs/architecture.md#browser-repl-api-wrapper)
for the ownership and result-mapping boundary.

Batches are mechanical primitive lists. They have no branching, saved values,
references, or workflow DSL.
Expand Down Expand Up @@ -495,7 +513,7 @@ pi -p --provider openai --model gpt-5.6-sol \

### The menu

Eight entries, one per capability. Availability is per model, and `/browser-tools`
Nine entries, one per capability. Availability is per model, and `/browser-tools`
tells you which apply to the one you selected.

| entry | tools | works on |
Expand All @@ -504,6 +522,7 @@ tells you which apply to the one you selected.
| `computer` | canonical computer primitives plus `computer_batch` | every provider |
| `browser-act` | `browser_act`, the verified-plan tool | every provider except Moonshot, which rejects its schema size |
| `playwright` | `playwright_execute` | every provider |
| `repl` | persistent `browser_repl` JavaScript cells | every provider |
| `anthropic-computer` | Anthropic's native computer tool | Anthropic models with that native surface |
| `anthropic-browser` | Anthropic's native browser tool | Anthropic models with that native surface |
| `openai-computer` | OpenAI's native computer tool | OpenAI models with that native surface |
Expand Down Expand Up @@ -570,7 +589,8 @@ npm run example:harness --workspace @onkernel/browser-loop -- \
```

See [`examples/`](examples) for direct catalog/model usage, these parameterized
agent and harness examples, and the Anthropic-native composition.
agent and harness examples, and the Anthropic-native composition. Release steps
are documented in [`docs/npm-releases.md`](docs/npm-releases.md).

## License

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,8 @@ The main groups are:
- `loop.tools.browser.*`: CDP/page tools, using element refs and viewport pixels.
- `loop.tools.computer.*`: Kernel OS input/read tools, using pixel coordinates by
default.
- `loop.tools.playwright()`: a Playwright code execution tool.
- `loop.tools.playwright()`: a wrapper around Kernel's Playwright execution API.
- `loop.tools.repl()`: a wrapper around Kernel's persistent Browser REPL API.
- `loop.toolsets.browser()`, `computer()`, and `mixed()`: ordinary convenience
arrays of Browser Loop-authored tools.
- `loop.providers.*`: only provider-native tools and predefined toolsets backed
Expand All @@ -118,8 +119,8 @@ await compiled.apply(harness);

Nothing mutates in place: a change compiles a new pair, and `compile()` throws
before anything reaches pi. Existing tool
identity with a changed schema, executor, or coordinates counts as a real
replacement. Additions made from inside a running tool are recorded in pi's
identity with a changed model-facing name, schema, or coordinate contract counts
as a real replacement. Additions made from inside a running tool are recorded in pi's
Anthropic-compatible `addedToolNames` marker only when that provider/model can
defer ordinary function tools. Additions outside a tool call are eager.
Provider-native tools are always eager.
Expand All @@ -136,12 +137,41 @@ catalog and model changes. It owns:
- one canonical computer translator;
- one lazily created raw-CDP `BrowserExecutor`;
- browser element-ref and frame state;
- screenshot and Playwright execution capabilities.
- screenshot, Playwright, and persistent Browser REPL execution capabilities.

Recompiling and applying a catalog preserves refs, tabs, browser state, and
caches. Tools are materialized as small adapters over that shared pool,
exactly once per spec object.

## Browser REPL API wrapper

`loop.tools.repl()` is the ordinary function tool `browser_repl`, with stable
identity `kloop.repl.v1`. It is framework-neutral: `LoopExecutionResources`
materializes it like every other Loop spec, the pi binding adapts that executable
to an `AgentTool`, and future framework bindings can adapt the same executable.
The root-exported `executeBrowserRepl()` is available when a binding needs the
HTTP client directly.

Browser Loop does not contain, publish, or start the REPL daemon. The browser VM
owns persistent JavaScript state and kernel-images owns `POST /repl`, daemon
packaging, process supervision, reset and destructive-timeout semantics, CDP and
browser helpers, WebMCP, and optional Patchright/Playwright installation.
Consequently this tool supports Kernel browser handles whose image implements
`POST /repl`; it has no local-Chrome execution mode.

The HTTP client requires the browser's `base_url` and `cdp_ws_url`. It copies the
existing `jwt` query parameter from the CDP URL onto `<base_url>/repl`, forwards
the execution's abort signal, and sends `{code, timeout_sec?, reset?}`. It
validates local input constraints before dispatch and rejects non-2xx or malformed
responses.

The API response keeps its exact `repl_id`, error and stack, duration,
`content_truncated`, and `repl_terminated` state. `LoopExecutionResources` maps
response items in order: `write` text remains plain, `stdout` and `stderr` keep a
channel prefix, and images retain their MIME type and base64 data. A failed REPL
execution becomes model-readable output with `details.isError`; transport and
protocol failures throw as tool failures.

## Action planes and result feedback

Canonical actions live under `packages/browser-loop/src/core/actions/`:
Expand All @@ -158,6 +188,7 @@ Tools return only the result requested by the model:
- Read actions return their requested text or structured data.
- Screenshot and zoom actions return images.
- `browser_act` returns causal outcomes and a bounded successor diff.
- `browser_repl` preserves ordered text/image output and process-lifecycle metadata.
- Failed batches replace images captured by earlier explicit screenshot steps
with textual markers.

Expand Down Expand Up @@ -215,8 +246,9 @@ transport.

### The tool menu

`loopToolMenu(model, selected)` in `packages/browser-loop/src/core/menu.ts` returns every tool
Browser Loop can offer for a model, each marked available or not. It decides availability
`loopToolMenu(model, selected)` in `packages/browser-loop/src/core/menu.ts`
returns every tool Browser Loop can offer for a model, each marked available or
not. It decides availability
by compiling the candidate catalog rather than by restating the compiler's
rules, so the menu cannot drift from what `compileLoopToolCatalog` accepts: an
entry is available exactly when selecting it compiles. Compilation is pure and
Expand Down Expand Up @@ -280,7 +312,7 @@ user prompt
-> provider stream
-> incoming native/function call normalization
-> shared LoopExecutionResources
-> Kernel computer API or raw-CDP BrowserExecutor
-> Kernel computer/Playwright/REPL API or raw-CDP BrowserExecutor
-> policy-specific action result
-> transcript + TUI/stdout/JSONL
```
Expand All @@ -289,7 +321,10 @@ user prompt

- `packages/browser-loop/test/tool-catalog.test.ts`: identities, collisions, provider
composition, compatibility, declarations, and coordinate contracts.
- `packages/browser-loop/test/resources.test.ts`: action feedback and batch boundaries.
- `packages/browser-loop/test/resources.test.ts`: action feedback, Browser REPL
result mapping, and batch boundaries.
- `packages/browser-loop/test/repl-client.test.ts` and `repl-tool.test.ts`:
Browser REPL authentication, validation, declaration, and catalog behavior.
- `packages/browser-loop/test/attach.test.ts` and `attach-session.test.ts`: compiled
pairs, applying one to a running harness, and the behaviors `activate()`
installs.
Expand Down
64 changes: 64 additions & 0 deletions packages/browser-loop/docs/npm-releases.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
# npm releases

`@onkernel/browser-loop` publishes from tags matching `browser-loop/v*` through
`.github/workflows/release-browser-loop.yml`.

## Release contract

Before pushing a tag:

1. Update `packages/browser-loop/package.json` and `package-lock.json` to the
release version.
2. Merge that version change to `main`.
3. Tag the merged commit as `browser-loop/v<version>`.

The workflow rejects a tag when its commit is not contained in `main` or when
the tag version differs from `packages/browser-loop/package.json`.

```sh
git checkout main
git pull --ff-only
git tag browser-loop/v0.13.0
git push origin browser-loop/v0.13.0
```

npm versions and Git tags are immutable. Never reuse a failed or previously
published version; fix the problem, bump again, and create a new tag.

## What the workflow validates

The release job uses Node 24 and npm 11, then runs:

```sh
npm run check:lockfile
npm ci
npm run build --workspace @onkernel/browser-loop
npm run typecheck --workspace @onkernel/browser-loop
npm test --workspace @onkernel/browser-loop
npm pack --workspace @onkernel/browser-loop
```

It installs the packed tarball into a fresh project and verifies both package
entry points before publishing the same package to npm.

## Trusted publishing

The npm package is configured for GitHub Actions trusted publishing:

| package | organization | repository | workflow filename | environment |
| --- | --- | --- | --- | --- |
| `@onkernel/browser-loop` | `kernel` | `browser-loop` | `release-browser-loop.yml` | none |

To restore that configuration if it is removed:

```sh
npm install -g npm@^11.17.0
npm trust github @onkernel/browser-loop \
--repo kernel/browser-loop \
--file release-browser-loop.yml \
--allow-publish
```

The workflow receives `id-token: write` and publishes with
`npm publish --workspace @onkernel/browser-loop --access public`; it does not use
a long-lived npm token.
27 changes: 27 additions & 0 deletions packages/browser-loop/examples/repl-agent.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
import { Agent } from "@earendil-works/pi-agent-core";
import Kernel from "@onkernel/sdk";
import { loop } from "../src/index";
import { attach } from "../src/pi/index";

const client = new Kernel({ apiKey: process.env.KERNEL_API_KEY! });
const browser = await client.browsers.create({ stealth: true, timeout_seconds: 600 });
const handle = attach({ client, browser });
const compiled = handle.compile({
model: "anthropic:claude-opus-5",
tools: [loop.tools.repl()],
});
const agent = new Agent({
streamFn: (model, context, options) => compiled.models.streamSimple(model, context, options),
initialState: {
model: compiled.model,
tools: [...compiled.agentTools],
systemPrompt: "Use browser_repl. Persist useful JavaScript helpers between calls and emit answers with repl.write().",
},
});

try {
await agent.prompt("Open example.com, save a helper that reads the page title, then call it and report the title.");
} finally {
await handle.dispose();
await client.browsers.deleteByID(browser.session_id);
}
Loading
Loading