Skip to content

feat: dashboard chat reply is the agent's own words, not a canned string - #102

Merged
venkateshsakamuri-lab merged 1 commit into
mainfrom
feat/agent-chat-responses
Sep 8, 2026
Merged

feat: dashboard chat reply is the agent's own words, not a canned string#102
venkateshsakamuri-lab merged 1 commit into
mainfrom
feat/agent-chat-responses

Conversation

@notSumit25

Copy link
Copy Markdown
Collaborator

Problem

The dashboard build chat replied with the same hardcoded string on every completed build — "Done — built and verified against your data. Saved as a draft — tell me what to change." — regardless of what the user said. It showed up identically against unrelated messages and against corrections. In the reported case a user said the dashboard showed ~3k active hotels when they expected 6k+, and the reply was still a blanket "built and verified": a confident claim of success over a change that may never have landed. That's worse than templated.

Root cause: the agent's task contract literally said "no prose", and both the backend (SavedDashboardService.java) and the frontend store (useDashboardChatStore.js) hardcoded the constant. There was no agent reply to show — the model was forbidden from speaking.

Change

  • Contract (DashboardAgentService): the final message now ends with a required ```dashboard-note fence — 1–3 sentences in plain business language: what this turn changed, anything it could not do/verify or deliberately skipped, and on a correction whether the disputed value actually moved. Bound by the existing "never show internals" security rule (no table/column/SQL/UUID in the reply). extractSummary reads it; stripNotes removes it before HTML extraction so prose can't be mistaken for the artifact. The note rides in config.summary.
  • Rendering: SavedDashboardService.buildReplyText (persistence) and the frontend store's buildReplyText (live tab) both render the note, falling back to a now-shared DEFAULT_BUILD_REPLY constant only when a turn produced no note — so an agent still on the old contract keeps working. The two helpers are kept textually in sync (documented in CLAUDE.md).

Verification (live stack, not code-reading)

Real build + a correction turn + chat-only path, driven through the browser (chrome-devtools MCP), with DB read-back and network confirmation that calls hit the local backend:

  • Happy path — reply is the agent's own note ("…I did not add richer breakdowns because the recent activity is very sparse…"), persisted and rendered; not the canned string.
  • Correction case (the original bug) — challenged "20"; reply: "…The earlier 20 was not a broken count; it was the full conversation total, and the updated headline now shows sent replies separately, which is 10…" — addresses the dispute directly.
  • Security — leakage scan across all replies: clean.
  • State consistency — full reload shows the persisted honest notes, no fallback.
  • Regression — dashboards still build fully (real widgets via deepsql.query); chat-only path still returns the agent's words and never appends the canned line.
  • Sync invariant — backend and frontend fallback strings byte-identical.

Full hands-on QA: 11/11 scenarios PASS, zero blocking issues.

Notes

  • Backend recompiled (BUILD SUCCESS); frontend lint clean.
  • Not a code change but required to run the feature locally: the agent handoff needs AGENT_WEBUI_URL=http://deepsql-agent:8787 (the compose default). A local .env with http://localhost:8787 makes every build fail as "agent unavailable" — worth aligning .env.example/docs, but outside this PR's committed files.
  • This makes replies honest, not the underlying data more correct — a genuinely wrong figure is now surfaced in words, but fixing the SQL/grounding behind it is a separate follow-up.

🤖 Generated with Claude Code

The dashboard build contract forbade prose ("no prose"), and both the
backend and the frontend hardcoded the same reply on every completed
build: "Done — built and verified against your data." Every turn showed
the identical line regardless of what the user asked — including turns
that were corrections ("you're showing 3k, I expected 6k+"), where a
blanket "built and verified" claimed success over a change that may not
have landed. Worse than templated: a confident false claim.

The contract now ends with a required ```dashboard-note``` fence — 1-3
sentences in plain business language stating what THIS turn changed,
what it could not do or verify, and (on a correction) whether the
disputed value actually moved. It is bound by the existing
"never show internals" security rule, so no table/column/SQL/UUID leaks
into the reply.

- DashboardAgentService: NOTE_FENCE + extractSummary/stripNotes; the
  note is stripped BEFORE HTML extraction so prose can't be mistaken for
  the artifact; summary rides in config.summary.
- SavedDashboardService.buildReplyText and the frontend store's
  buildReplyText both render the note, falling back to the (now shared)
  DEFAULT_BUILD_REPLY constant only when a turn produced no note — so an
  agent still on the old contract keeps working.

Verified end-to-end against the live stack (real build + a correction
turn + chat-only path), DB read-back, and browser: the reply now differs
per turn, addresses corrections directly, leaks no internals, and
survives reload.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@notSumit25
notSumit25 requested a review from a team as a code owner September 8, 2026 09:39
@notSumit25

Copy link
Copy Markdown
Collaborator Author

Screenshots — the reply is now the agent's own words

The chat thread (one build turn, then a correction turn) — the two cases that show the fix:

chat thread

  • Build turn: "This turn added a read-only activity dashboard… I verified the underlying counts for the recent window before emitting the widgets; I did not add any extra breakdowns because … the available recent activity is very small." — describes what it did and volunteers a limitation, instead of the old constant "Done — built and verified against your data."
  • Correction turn (the original bug from the report screenshot): user says "Messages sent shows 20 but I expected far more. Is that number right?""The reported 20 is no longer what this dashboard shows: for the current default range it now shows 0 sent messages and 3 conversations started, and I did not add any broader 'all activity' metric because I could not verify a larger like-for-like sent-message source without changing its meaning." — it names the disputed figure, states the new value, and says what it couldn't verify.

Full workspace — the reply and the dashboard it produced side by side (note the KPI actually changed from 20 → 0 to match the words):

build reply

correction reply

Captured live through the browser (chrome-devtools MCP) against the local stack; no internals (table/column/SQL/UUID) appear in any reply.

Regression suite status

Ran the test:local-regression steps against the running stack. The host has no JDK, so the backend smoke test ran in an isolated maven:3.9-eclipse-temurin-25 container on the compose network (backend stopped first per CLAUDE.md's ddl-auto deadlock warning), with ENCRYPTION_KEYS pinned to key id local-2025-01.

Step Result
Backend health probe ✅ PASS (/actuator/health → 200, status UP)
Frontend health probe ✅ PASS (200)
Frontend production build ✅ PASS (npm run build exit 0, dist/index.html + 4 JS bundles produced)
Backend API smoke test (ApiSmokeTest) ✅ PASS — Tests run: 3, Failures: 0, Errors: 0, BUILD SUCCESS (full Spring context booted in 96.9s against real Postgres + valkey)

All green. Backend recompiled cleanly (BUILD SUCCESS) with the final diff.

@venkateshsakamuri-lab
venkateshsakamuri-lab merged commit 1412034 into main Sep 8, 2026
9 checks passed
@venkateshsakamuri-lab
venkateshsakamuri-lab deleted the feat/agent-chat-responses branch September 8, 2026 16:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants