compaction prompt improvement ideas (including md analysis)

[brdloush]

Motivation

Having started to use autonomous agentic flow in last couple of days, I have a feeling that eca is curently not exactly strong when it comes to context compaction.

I know that I can rewrite the prompt myself (am about to try that soon)  but perhaps the "out of the box experience" (usable for most people) can be improved.

What

I had some basic (AI-driven) research done about how different opensource harnesses approach this compaction problem, what's their interesting ideas as well as limitations.

Slack conversation for context: https://clojurians.slack.com/archives/C093426FPUG/p1789151504300249

How & why

I believe some ideas might be good adopt "in the default". I've read the whole document and a lot of things clicked for me which might be good ideas, such as

• structure of the compaction output that must be filled (IMO this is very important to keep to keep at least some minimal quality of the compaction handout output)
• concrete fidelity rules, nudging agent to mention specific things, not generic prose
• distinguishing whether we are doing 1st compaction or Nth already (2nd and n-th compaction may be tailored more to avoid further degradation). I guess this might need some additional coding effort, so should be considered first. But the example mentioned in Merge Rules seems very usable (~ "be aware that there is already a heavily summarized content on the very beginning of this session, don't rely on it THAT MUCH, newer turns win over the summary, ..". So perhaps distinguishing between "first" and "nth" might be worth implementing.

I also believe that it might be beneficial to sacrifice a bunch of session-leading tokens (few tens of them) to better introduce what the purpose of the received compacted content really was:

;; current
"The conversation was compacted/summarized, consider this summary:\n"

;; suggested
(str "A previous agent worked on this task and ran out of context. "
     "Below is its handoff summary. Use it to avoid repeating work already done.\n\n"
     "Treat the working tree and command output as authoritative. The summary "
     "locates the relevant work; it is not proof that the work is correct or "
     "complete. Verify before you rely on it.\n\n"
     summary)

The actual analysis (ai comparison; some strong/weak/adopt ideas of other opensource harnesses)

Here's that mentioned full document for hammock-thinking on the compaction topic :-)

compaction-prompts-v2.md

[brdloush]

Btw with all those ideas in context, I asked my llm to derive me a "custom compaction prompt that we can already use today" (without any changes in ECA). And it managed to invent interesting prompting. Will try this prompt in upcoming days.

This one caught my eyes

First compaction, or a later one?

Look at the start of the conversation above. If it already contains an earlier
handoff summary, this is a later compaction — apply the Merge rules below.
If it does not, this is the first — skip them.

Basically it managed to fold quite a lot of ideas that "miss some proposed features" into prose-based instructions. Perhaps this might be useful strategy as well.

compact.md