Improve agent feedback signal quality

[masnwilliams]

summary

• classify capability requests by owner, product area, operation, and task outcome before recording demand
• route existing-tool failures, transient capacity failures, and client permission restrictions away from capability analytics
• add normalized task outcomes, KERNEL tool ownership, routing destinations, and deduplication keys to feedback events
• redact domains, IP addresses, and filesystem paths in addition to existing URL, email, and token redaction
• preserve the existing bot-detection and config-registry outcome contracts

testing

• bun test (539 passing)
• bun x tsc --noEmit
• Prettier check for all changed files
• bun run build with the required OAuth client IDs set to build-only placeholders
• git diff --check

The repository-wide Prettier check still flags the pre-existing formatting of AGENTS.md; this change does not modify that file.

---

Note

Medium Risk
Changes MCP agent-facing tool contracts and PostHog event shapes for feedback and capability demand; behavior is heavily tested but alters how client reports are classified and recorded.

Overview
Replaces the PostHog SDK’s unstructured get_more_tools path with a first-class MCP tool that validates structured gap reports and emits mcp_capability_requested only for true KERNEL or external-integration demand. Existing-tool failures, transient capacity issues, and client permission blocks are rejected from demand analytics and steered toward submit_feedback instead; SDK reportMissing is disabled so the real tool schema always runs.

submit_feedback gains normalized task_outcome, optional affected_tool (canonical KERNEL names with namespace normalization), richer feedback_destination routing, dedupe keys, and server-side redaction across free-form fields. MCP feedback must name a KERNEL-owned tool; capability gaps are directed to get_more_tools.

Analytics redaction now strips domains, IPv4/IPv6, and filesystem paths in addition to URLs, emails, and tokens. A KERNEL_MCP_TOOL_NAMES registry plus registration test keeps advertised tools aligned with the ownership checks. README standalone-tool docs reflect the split between get_more_tools and submit_feedback.

^{Reviewed by Cursor Bugbot for commit e8e7135. Bugbot is set up for automated code reviews on this repo. Configure here.}

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
mcp	Ready	Preview	Sep 15, 2026 10:32pm UTC

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 33b3133. Configure here.}

[rgarcia]

reviewed — the structured routing is a solid direction, but several cases can still produce incorrect or unsafe analytics signals:

bugs

• src/lib/mcp/analytics.ts:235 — the IPv6 redaction misses common valid forms including 2001:4860:4860::8888, fe80::1, and ::1, leaving IPs in captured free-form text. please cover compressed and digit-only IPv6 addresses.
• src/lib/mcp/tool-names.ts:40-42 — namespace normalization discards every __ prefix, so names like mcp__slack__manage_browsers and external__manage_apps are accepted as KERNEL-owned. please only strip recognized KERNEL namespace forms.
• src/lib/mcp/analytics.ts:647-651 — positive product feedback without product_area routes to product_praise, making product_unclassified unreachable for that case. check for the missing area before sentiment.
• src/lib/mcp/analytics.ts:671 — dedupe normalization removes all non-ASCII letters, causing unrelated Chinese, Japanese, and accented summaries to share the same input. consider hashing the sanitized summary directly or preserving Unicode letters and numbers.
• src/lib/mcp/tools/missing-capability.ts:127-133 — idempotentHint: true conflicts with emitting a new PostHog event on every call. the custom dedupe property does not enforce capture idempotency, so this should be false unless a stable $insert_id is added.

questions

• src/lib/mcp/analytics.ts:653-658 — lookup reports without analysis_id share one dedupe key per domain even when the applied configuration or outcome changes. should the key include the applied-config key and observed outcome?
• src/lib/mcp/tools/feedback.ts:256-268,440-464 — descriptions say affected_tool and product_area are required conditionally, but runtime accepts MCP feedback without a tool and product feedback without an area. if this is for legacy compatibility, could the schema/docs say “preferred”; otherwise, should these be enforced?

tests

• src/lib/mcp/register.test.ts:56-67 — this only proves registered tools appear in the allowlist; it cannot detect stale entries that remain accepted after tool removal. please check both set differences, accounting for get_more_tools and submit_feedback being registered elsewhere.
• src/lib/mcp/analytics.test.ts:1066-1111 — the classifier’s external-integration, owner-mismatch, existing-tool-failure, and capture-failure branches are untested. these are core routing paths and worth covering alongside the fixes above.