Skip to content

Observability pricing agent swarm demo - #12

Open
ChloeQuijano wants to merge 9 commits into
mainfrom
agent/observability-pricing-swarm
Open

Observability pricing agent swarm demo#12
ChloeQuijano wants to merge 9 commits into
mainfrom
agent/observability-pricing-swarm

Conversation

@ChloeQuijano

Copy link
Copy Markdown
Contributor

Summary

Adds the observability-pricing-agents demo: one script that fans out one Browserless /agent/run browser agent per observability vendor in parallel, polls a live terminal dashboard, and normalizes the structured results into a pricing comparison table.

On top of the original demo commit, this branch includes three fixes/improvements found by actually running it:

  • fix: unwrap the data.answer payload /agent/run returns — without this every field was undefined and the final comparison table rendered all dashes even on a fully successful run
  • feat: save each run's JSON results to a timestamped results/pricing-<timestamp>.json (gitignored) so closing the terminal doesn't lose the data
  • docs: README now points at the account dashboard for the API token and states the script reads it only from the BROWSERLESS_TOKEN env var (no .env support)

Test plan

  • Ran a 3-vendor swarm (Datadog, New Relic, Grafana Cloud): 3/3 succeeded, exposed the empty-table bug
  • Ran a 5-vendor swarm (…+ Dynatrace, Sentry) on the patched script: 5/5 succeeded, final table rendered with real values (e.g. New Relic $8,376/mo vs Sentry Team $76.50/mo for the same workload), results JSON saved under results/
  • node --check index.mjs passes; the save path was verified by replaying a captured run's payload through the identical logic

🤖 Generated with Claude Code

ChloeQuijano and others added 9 commits August 18, 2026 13:11
…nders

/agent/run returns the agent's output as a JSON string nested in
data.answer, but parseResult only handled a top-level string — so every
result field was undefined and the FINAL COMPARISON table rendered all
dashes even on a fully successful run.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The full results were only printed to the terminal, so closing it lost
the data. Each completed run now also writes the same JSON payload to
results/pricing-<timestamp>.json (gitignored).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ar-only handling

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@ashwinsingh2007

Copy link
Copy Markdown

Clean on the things that actually matter for a public repo: no secrets, no internal hosts, no committed .env, and zero dependencies so there's nothing to pin. Three things worth changing.

The terminal-status list is hardcoded and misses one. Both the poll loop and the "N/M runs finished" counter test against ['succeeded','failed','timed_out','stopped'], but the run-status union the API returns also includes capped. A capped run never matches, so the loop polls it forever every 2.5 seconds, Promise.all never settles, and the demo hangs with no comparison table and no saved JSON. capped is v2-only today so it isn't reachable yet, which is exactly what makes it the kind of thing that breaks later, on camera. Add it, and add a client-side wall-clock deadline so no unexpected status value can wedge the loop.

The prompt asks for JSON in prose when /agent/run has a parameter for it. Passing responseSchema makes the platform parse the answer into data directly instead of folding raw text into data.answer — which is the failure the PR description says it hit. The fence-stripping in parseResult is a workaround for not passing the param, and passing it makes the empty-table bug structurally impossible. Related: maxSteps defaults to 30 and hard-caps at 60, and "pricing page, then docs, then the calculator, then check SSO" is a lot of tool calls for 30 — worth setting explicitly rather than discovering it as a half-answered run.

The root README says "10 Browser Agents in Parallel". The demo ships 5 and the folder README says 5. That table is what people land on from a video description.

Minor, your call: nothing tells the reader that each run bills browser time, and 5 agents against a 15-minute per-run cap is real spend for someone who copy-pastes the command. One README line plus an explicit maxSteps would bound it. Also saveResults() only runs after every vendor has finished, so a Ctrl-C partway through a slow swarm throws away everything the agents that already succeeded returned — writing incrementally as each run terminalizes would be cheap.

What I verified: the token is read only from the environment and never printed or written anywhere, results/ is gitignored, the README's 15-minute cap claim is exactly right (the API clamps timeout to that ceiling), data.answer really is the shape returned when no responseSchema is set, steps really is a string array so the dashboard's latest-step column renders properly, and each vendor's error is caught per-run so one failure doesn't abort the swarm.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants