Agentic UI Testing

A share-once demo: describe a workflow, let an agent discover it in your real application, save a deterministic recipe, and replay it with screenshots and video. When a recipe fails, the agent can investigate the code, prepare a repair, and write a PR description. You can explicitly publish the repair as a draft PR.

prompt → OpenCode + OpenRouter → OpenHands browser → JSON recipe
JSON recipe → OpenHands replay → result.json + screenshots + continuous MP4
failure → OpenCode investigation → source repair + PR.md → optional draft PR

OpenHands owns Chromium inside the Linux container. OpenCode defaults to openrouter/z-ai/glm-5.3-flash. Saved recipes replay without a model or an OpenRouter key. Your coding agent adapts this demo to your project; application- specific database checks and account provisioning are outside its scope.

Try the browser and video first

Install Docker Desktop/Engine and Node.js 24+, then run from this folder:

npm link
agentic-ui build
agentic-ui run --workspace ./demo --recipe .agentic-ui/recipes/example-homepage.json

This checks the real public Example Domain page; no credentials are required. Look in demo/.agentic-ui/artifacts/ for results, screenshots, and recording.mp4. The first build downloads the browser and agent runtimes. The CLI defaults to linux/arm64. To select an AMD/x86 image, set AGENTIC_UI_PLATFORM=linux/amd64 before build, discover, run, or repair. agentic-ui doctor shows the selected platform.

Let your agent set up your project

Give your coding agent this prompt, replacing the path and workflow:

Read this package's README.md and AGENTS.md. Set up Agentic UI Testing for /absolute/path/to/my-project. Inspect its instructions and startup commands, initialize configuration, and help me supply missing credentials in .env.agentic-ui. Start the real application and check container connectivity. Discover a scenario for: [the user outcome I want to test]. Replay the saved recipe and show me its result and video. Configure relevant source validation for repair analysis. Preserve existing secrets and project changes. Do not publish a PR unless I ask.

To initialize manually:

agentic-ui init --workspace /absolute/path/to/my-project

Edit agentic-ui.config.json: set targetUrl, test-account variable names in credentials ([] for public workflows), and source directories in repair.paths. Set repair.prepareCommand and repair.validationCommand to project-specific argv arrays, e.g. ["npm", "ci"] and ["npm", "test"], when appropriate. Use http://host.docker.internal:3000 with the actual port for a host app, which must accept Docker connections. Your project starts the app and its dependencies.

Create or edit the project's .env.agentic-ui, preserving existing files:

OPENROUTER_API_KEY=your-openrouter-key
TEST_EMAIL=your-dedicated-test-account
TEST_PASSWORD=your-test-account-password

Create a key at OpenRouter and fund that account for discovery/repair. Only set account variables named in your config. init adds a secret-file ignore entry. Shell variables take precedence, then the project's .env.agentic-ui, then this package's .env.agentic-ui. Ordinary .env files are not loaded. AGENTIC_UI_TARGET_URL overrides the configured URL. Run agentic-ui doctor --workspace /absolute/path/to/my-project to check setup.

Discover, replay, and record

agentic-ui discover --workspace /absolute/path/to/my-project \
  --prompt "Log in with the configured account, open Settings, and verify the profile heading"
agentic-ui run --workspace /absolute/path/to/my-project \
  --recipe .agentic-ui/recipes/login-and-open-settings.json

Use the actual recipe filename reported by discovery. The agent explores the live UI and saves recipes under .agentic-ui/recipes/; the controller independently replays new recipes and records its verdict in verification.json. Recipes support navigate, click, fill, press, wait_for, assert_text, assert_url, and screenshot. Credentials use ${TEST_EMAIL} references rather than values. Review and commit configuration, recipes, and generated ignore files.

Replay produces step results, screenshot checkpoints, failure evidence, and a continuous MP4. browser.videoQuality: "quick" records 720p at 10 fps; "demo" records 1080p at 60 fps with an 800 ms dwell between steps. Set browser.recordVideo: false to disable recording.

For extensions, build an unpacked extension and set extensionPath to its repository-relative directory containing manifest.json. It loads in OpenHands during discovery/replay. Use a stable manifest key if your scenario requires a stable extension ID. Profiles persist under .agentic-ui/profile/.

Repair analysis and GitHub

Repair starts from a clean Git checkout; commit config and recipes first:

agentic-ui repair --workspace /absolute/path/to/my-project \
  --failure .agentic-ui/artifacts/RUN_ID/result.json

The controller reproduces the failure. OpenCode investigates and may edit source directly in your checkout. The controller checks changed paths and independently runs repair.validationCommand. Under .agentic-ui/repairs/, inspect report.json, PR.md, the transcript, and validation output. There is no automatic rollback. The demo does not build/serve a repaired candidate: replaying the original URL does not validate source edits. The PR description records that limitation.

Review the changes and description, then optionally publish:

gh auth login
gh auth setup-git
agentic-ui publish-pr --workspace /absolute/path/to/my-project \
  --report .agentic-ui/repairs/REPAIR_ID/report.json

Publication requires a product-defect repair, passing controller-run source validation, and exactly the recorded changes and Git HEAD. It creates a branch, commits, pushes to origin, and opens a draft PR. GitHub CLI needs repository push and PR permissions. Remote jobs do not automatically publish.

Local and remote use the same runtime

Locally the CLI mounts your project into Docker. Remotely railway.json builds the same Dockerfile; the worker clones your target GitHub repository and invokes the same controller. Commit config and recipes to that target repository. Deploy this demo repository as a one-shot Railway job, attach a volume at /data, and set:

GITHUB_REPOSITORY=owner/target-repository
AGENTIC_UI_JOB=run
AGENTIC_UI_RECIPE=.agentic-ui/recipes/login-and-open-settings.json
AGENTIC_UI_TARGET_URL=https://your-deployed-app.example

Add test-account variables required by the target config. Public repositories need no token; private ones need GITHUB_TOKEN with repository Contents read access. GITHUB_REF optionally selects a branch/tag. Discovery uses job discover, AGENTIC_UI_PROMPT, and OPENROUTER_API_KEY. Repair uses job repair, the key, AGENTIC_UI_FAILURE, and AGENTIC_UI_RESUME_RUN (the previous run's directory name) to restore evidence. The target app must already be running and reachable.

Successful and failed jobs retain evidence under /data/runs/<id>/evidence. Remote repairs also retain repair.patch and new files. Scheduling belongs to the hosting platform. The local CLI defaults to linux/arm64, or uses the architecture selected through AGENTIC_UI_PLATFORM; Railway selects its Linux runtime architecture. Both run the same Dockerfile and controller. For closer deployment parity across Apple Silicon and x86 machines, publish a multi-platform image and set AGENTIC_UI_IMAGE to its immutable reference. Host Docker URLs are local only.

Trust and limitations

Use trusted repositories and dedicated test accounts. The project is mounted writable; OpenCode permissions and post-run checks are advisory controls, not a sandbox for hostile code. Repair commands can read mounted files. The browser does not receive OpenRouter/GitHub keys, but credentials, browser sessions, videos, screenshots, and transcripts can contain sensitive data. Review artifacts before sharing; remove old evidence/profiles yourself.

This small demo defaults to Linux/arm64 containers locally, with Linux/amd64 available through AGENTIC_UI_PLATFORM. It uses live providers and has no hosted dashboard, candidate deployment automation, or maintenance/support guarantee. See AGENTS.md for instructions to preserve when adapting it.