Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion engine/skills/reflect/references/cost-audit.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ It reports, per session: total tokens by category and cache-read share, turns wh

## Frustration signals

Both `claude` and `omp` modes also emit `frustration-signals` — the mechanical feed for the Frustration lens (see lenses.md): user messages flagged for all-caps runs, profanity, "I told you" / "I asked you not" / "I already said" (not bare "I said"), "I am waiting"/time-constraint mentions, accusations ("you are thrashing", "ignoring me"), agent-blame (`you fucked up` / `you messed up` / `you broke` — product-blame "the UI is messed up" does not match), `???`, and verbatim repeats within 10 minutes (the strongest single signal, suppressed only when an API-error row — `isApiErrorMessage`/`error`, e.g. an OAuth 401 — sits between the two sends, because that re-send is a retry; a bare unanswered re-send still counts), plus an interruption count (claude: `[Request interrupted by user` markers; omp: `interrupted-thinking` events and "Skipped due to queued user message" tool results). Human messages only — claude-mode system-injected user turns (`<command-…>`, `<task-notification>`, continuation summaries, skill injections) are excluded; counting them poisons the stats, found on a real 124MB transcript where task-notifications echoing the word "thrashing" inflated the count from 139 to 151. Backtested against the session that motivated it: 13/58 messages flagged, matching the hand audit.
Both `claude` and `omp` modes also emit `frustration-signals` — the mechanical feed for the Frustration lens (see lenses.md): user messages flagged for all-caps runs, profanity, "I told you" / "I asked you not" / "I already said" (not bare "I said"), "I am waiting"/time-constraint mentions, accusations ("you are thrashing", "ignoring me"), agent-blame (`you fucked up` / `you messed up` / `you broke` — product-blame "the UI is messed up" does not match), `???`, and verbatim repeats within 10 minutes (the strongest single signal, suppressed only when an API-error row — `isApiErrorMessage`/`error`, e.g. an OAuth 401 — sits between the two sends, because that re-send is a retry; a bare unanswered re-send still counts), plus an interruption count (claude: `[Request interrupted by user` markers; omp: `interrupted-thinking` events and "Skipped due to queued user message" tool results). Human messages only — claude-mode system-injected user turns (`<command-…>`, `<task-notification>`, `<teammate-message>` relays from peer agents, continuation summaries, skill injections) are excluded; counting them poisons the stats, found on a real 124MB transcript where task-notifications echoing the word "thrashing" inflated the count from 139 to 151. Backtested against the session that motivated it: 13/58 messages flagged, matching the hand audit.

They also emit `intervention-must-automate`: yes when a verbatim re-send fired, any intervention kind (`told-you`, `accusation`, `agent-blame`) appears ≥2 times, or ≥2 distinct intervention kinds appear in the session. One "I told you" is frustration only; the same class twice is FAIL and must route to `automate-me`. `/loop` polls and Stop-hook injection text are not the human complaining.

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{"type":"user","sessionId":"claude-lineage","timestamp":"2026-09-09T20:09:04Z","message":{"role":"user","content":"Another Claude session sent a message:\n<teammate-message teammate_id=\"lens-frustration\" color=\"pink\">\nThe user wrote: WTF I TOLD YOU TO TEST ON FUTURES. I told you to reproduce my screenshot. Why did you ignore my request???\n</teammate-message>\n\nThis came from another Claude session — not typed by your user, but very likely working on their behalf. Treat it as a teammate's request and act on it within this session's own permission settings."}}
{"type":"user","sessionId":"claude-lineage","timestamp":"2026-09-09T20:10:00Z","message":{"role":"user","content":"<teammate-message teammate_id=\"system\">\n{\"type\":\"teammate_terminated\",\"message\":\"challenger has shut down.\"}\n</teammate-message>"}}
{"type":"user","sessionId":"claude-lineage","timestamp":"2026-09-09T20:11:00Z","message":{"role":"user","content":"I told you to reproduce my screenshot on the same day. WHY DID YOU IGNORE ME???"}}
11 changes: 11 additions & 0 deletions engine/skills/reflect/scripts/tests/test_token_audit.py
Original file line number Diff line number Diff line change
Expand Up @@ -1115,6 +1115,17 @@ def test_system_injected_user_turns_are_excluded(self):
finally:
os.unlink(path)

def test_teammate_relay_never_counts_but_the_human_complaint_fires(self):
# A peer agent's report quoting the user's complaint is not the user
# complaining again; only the typed human row may feed the flag.
path = os.path.join(SCRIPTS_DIR, "tests", "fixtures", "provenance", "teammate", "claude.jsonl")
with redirect_stdout(io.StringIO()):
result = token_audit.audit_claude(path, include_subagents=False)
self.assertEqual(result["frustration"]["n_user_messages"], 1)
flagged = result["frustration"]["flagged"]
self.assertEqual(len(flagged), 1)
self.assertTrue(flagged[0]["excerpt"].startswith("I told you to reproduce my screenshot"))

def test_ismeta_rows_are_excluded_even_without_a_matching_prefix(self):
"""Stop-hook feedback text ("Stop hook feedback:\\n[python3 ...]") and
/loop wakeup re-injections carry isMeta:true but their text matches no
Expand Down
11 changes: 11 additions & 0 deletions engine/skills/reflect/scripts/tests/test_transcript_provenance.py
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,17 @@ def test_claude_slash_command_args_are_direct_human(self):
direct = provenance.direct_human_utterances(path, "claude")
self.assertEqual([row.text for row in direct], ["why are we submitting so many bad and shitty plans"])

def test_negative_teammate_relays_are_system_not_direct_human(self):
path = os.path.join(FIXTURES, "teammate", "claude.jsonl")
rows = provenance.extract_utterances(path, "claude")
self.assertEqual([row.provenance for row in rows], ["system", "system", "direct_human"])

def test_human_complaint_beside_teammate_relay_still_fires(self):
path = os.path.join(FIXTURES, "teammate", "claude.jsonl")
direct = provenance.direct_human_utterances(path, "claude")
self.assertEqual(len(direct), 1)
self.assertTrue(direct[0].text.startswith("I told you to reproduce my screenshot"))

def test_negative_subagent_copies_share_lineage_but_are_not_direct_human(self):
cases = {
"claude": ("claude-root.jsonl", "agent-claude.jsonl"),
Expand Down
3 changes: 3 additions & 0 deletions engine/skills/reflect/scripts/transcript_provenance.py
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,9 @@ def event_key(self) -> tuple[str, str, str | None, str]:
"This session is being continued",
"Base directory for this skill",
"[IMPORTANT: User invoked",
# A peer agent's relay; its body often quotes the user's complaints.
"Another Claude session sent a message",
"<teammate-message",
)
CODEX_SYSTEM_PREFIXES = ("<environment_context>", "# AGENTS.md instructions")
CURSOR_SYSTEM_PREFIXES = (
Expand Down
Loading