Skip to content

Add agentic JSON output mode for AI agent integration - #536

Open
danielsuguimoto wants to merge 2 commits into
kool-dev:mainfrom
danielsuguimoto:feat/agentic-output
Open

Add agentic JSON output mode for AI agent integration#536
danielsuguimoto wants to merge 2 commits into
kool-dev:mainfrom
danielsuguimoto:feat/agentic-output

Conversation

@danielsuguimoto

Copy link
Copy Markdown
Contributor
Issue N/A
🪲 Bug Fix No
🧰 Improvement Yes
🏆 Feature Yes
📝 Refactor No
❌ Removed No
📖 Documentation No
⚠️ Break Change No

Description
Adds a global --output json flag that enables machine-readable JSON output across kool commands, making the CLI suitable for use by AI agents and automation tools. When JSON mode is active, commands emit structured payloads instead of human-readable tables or text, diagnostics route to stderr to keep stdout clean for data, and interactive prompts are automatically disabled.

Commands with JSON output:

  • kool status: emits {"services":[...],"count":N} with service state, ports, and running status
  • kool logs: emits JSON Lines format ({"service":"...","message":"..."}) with streaming support for --follow
  • kool info: emits structured payload with kool/docker versions, binary paths, and environment variables (KOOL_API_TOKEN is redacted)
  • kool run: script listing works in JSON mode; script-not-found errors emit structured JSON to stderr with suggestions field

The existing --json flag on kool run is now hidden but remains functional for backwards compatibility, unified under the global --output json flag.


Notes

  • --output json is a global flag that must be placed before the script name when using kool run
  • Script output from kool run <script> remains raw (unstructured) as it passes through the underlying command output
  • The legacy --json flag on kool run is hidden but still works for backwards compatibility
  • All 306 tests pass with go vet clean

@github-actions github-actions Bot added release-drafter:added Release drafter: Added release-drafter:changed Release drafter: Changed release-drafter:minor Release drafter: Minor labels Jul 11, 2026
@danielsuguimoto

Copy link
Copy Markdown
Contributor Author

CI failures on the last run are addressed in commit 3b0d29c:

Check Root cause Fix
lint commands/run.go:159 — unchecked fmt.Fprintln return value (errcheck) Explicitly discard return with _, _ = fmt.Fprintln(...)
test (ubuntu) TestStatusJSONOutputMultipleServices — data race on FakeShell.CalledExec map (concurrent writes from status goroutines) Added sync.Mutex to FakeShell; guarded Exec/Interactive/LookPath map operations
grype golang.org/x/crypto v0.50.0 (our dep, multiple critical CVEs) + libcurl 8.20.0-r1 (inherited from docker:29-cli base, 8 critical CVEs) Bumped x/cryptov0.52.0; added libcurl CVEs to .grype.yaml ignore list (same pattern as existing CVE-2026-27143 entry — upstream base image issue)

Validated locally: go vet clean, all tests pass with -race, grype scan passes (exit 0, no critical).

@fabriciojs fabriciojs left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the agentic JSON output mode. I applied the patch to a scratch worktree: it builds, go vet is clean and go test ./... passes. go test -race -count=3 ./commands/... only surfaces the pre-existing TestVersionFlagCommand flake, which reproduces on main too — so the new sync.Mutex in FakeShell is doing its job.

The overall shape is good, but there are a few issues worth fixing before merge. Grouped by impact:

JSON contract is breakable

  • kool run collapses every parse error into {"error":"script not found"}, so a malformed kool.yml is misreported.
  • kool status --output json emits nothing at all when there are no services.
  • emitJSONError writes JSON to stderr, then main.go appends a plain-text error: ... line to the same stream, leaving stderr unparseable.

Data loss / potential hang

  • streamLogsJSON uses a default bufio.Scanner (64KB cap) and discards scanner.Err(); one long log line silently ends streaming and leaves cmd.Wait() blocked on an undrained pipe.
  • The non-follow JSON path uses CombinedOutput, folding docker compose's own stderr warnings into the log stream as bogus entries.

Regressions to existing non-JSON behavior

  • The new useColor() inspects s.outStream, which DefaultKoolTask.Run replaces with an io.Pipe writer — so colored output inside long tasks (kool start etc.) silently goes plain.
  • Fprintln(w, out...) and color.Sprint(out...) differ in how they separate operands, so spacing changes depending on whether color is on.
  • Color is decided from outStream but JSON-mode diagnostics go to errStream, so --output json 2> file writes ANSI codes into the file.

Polish

  • JSON services array has no deterministic ordering.
  • --output silently ignores anything that isn't exactly json.

Details inline. Thanks for putting this together — the feature itself is a nice addition.

Comment thread commands/run.go
// we should just warn the user about multiple finds for the script
r.Shell().Warning("Attention: the script was found in more than one kool.yml file")
err = nil
} else if r.Shell().IsJSONOutput() {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This else if catches every error that isn't a typo-suggestion or a multiple-defined-script error, and rewrites it into ErrKoolScriptNotFound / {"error":"script not found"}.

So a malformed kool.yml, an unreadable file, or a YAML parse failure all get reported to the agent as a missing script — which is exactly the wrong signal, since an agent will then go looking for the script name rather than fixing the file.

Worth gating on the actual not-found case, e.g.:

} else if r.Shell().IsJSONOutput() {
    if parser.IsScriptNotFoundError(err) {
        r.emitJSONError("script not found", []string{})
        err = ErrKoolScriptNotFound
    } else {
        r.emitJSONError(err.Error(), []string{})
    }
    return
}

(or whatever the parser's not-found predicate is), so genuine parse errors keep their own message.

Comment thread commands/logs.go
return
}

scanner := bufio.NewScanner(stdout)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two problems with this scanner in the follow path:

  1. 64KB token limit. bufio.NewScanner defaults to bufio.MaxScanTokenSize. A single log line longer than that makes Scan() return false with bufio.ErrTooLong. That is not far-fetched for JSON-logging apps or stack traces.
  2. scanner.Err() is discarded. When the above happens the loop just exits, err = cmd.Wait() is called on a process whose stdout pipe is no longer being drained, and the command blocks once the pipe buffer fills. From the agent's point of view kool logs -f --output json silently stops emitting and hangs.

Suggest raising the buffer and checking the error:

scanner := bufio.NewScanner(stdout)
scanner.Buffer(make([]byte, 0, 64*1024), 1024*1024)
for scanner.Scan() {
    ...
}
if e := scanner.Err(); e != nil {
    _ = cmd.Wait()
    return e
}

Or drop the scanner entirely for a bufio.Reader + ReadString('\n') loop, which has no line-length ceiling.

Comment thread commands/status.go

s.table.SetWriter(s.Shell().OutStream())
s.table.AppendHeader("Service", "Running", "Ports", "State")
if !s.Shell().IsJSONOutput() {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The len(services) == 0 early return a few lines above (L83-86) fires before any JSON handling, so it calls Warning("No services found.") and returns with nothing written to stdout.

Result: kool status --output json on a project with no services emits an empty stdout. A consumer doing kool status --output json | jq gets a parse error rather than the perfectly valid {"services":[],"count":0}.

Since the JSON struct already initialises Services with make(..., 0, len(statuses)), emitting the empty document is the natural behavior. Worth moving the JSON branch above the early return, or special-casing it there.

Comment thread core/shell/shell.go
// Color is disabled when NO_COLOR env is set (handled by gookit/color)
// or when the output stream is not a terminal.
func (s *DefaultShell) useColor() bool {
return color.Enable && NewTerminalChecker().IsTerminal(s.outStream)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This checks s.outStream directly rather than going through a stream that reflects the current task context — and DefaultKoolTask.Run swaps outStream for an io.Pipe writer for the entire duration of the task.

An io.Pipe writer is never a terminal, so IsTerminal returns false and all colored Warning / Success / Info / Error output produced inside a long-running task (kool start, kool preset, etc.) loses its color, even on a fully interactive TTY. That's a visible regression for existing non-JSON users, not just a JSON-mode concern.

The pre-existing code colored unconditionally, which is why this didn't come up before. Probably needs to consult the original/underlying stream (or cache the TTY decision at shell construction time, before any task swaps the stream).

Comment thread commands/run.go
"suggestions": suggestions,
}
errPayload, _ := json.Marshal(payload)
_, _ = fmt.Fprintln(r.Shell().ErrStream(), string(errPayload))

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

emitJSONError writes the JSON payload to ErrStream(), but the error is still returned up the stack and main.go ends up calling Shell().Error(err), which — in JSON mode — also writes to stderr (via diagnosticStream()).

So stderr ends up as:

{"error":"script not found","suggestions":[]}
error: script not found

A consumer that reasonably assumes "stdout is data, stderr is JSON diagnostics" can't parse that trailing line. Either send the structured error to stdout (keeping stderr free-form), or suppress the plain-text Error() call when the JSON payload was already emitted.

Comment thread core/shell/shell.go
if s.useColor() {
out = []interface{}{color.New(color.Yellow).Sprint(out...)}
}
_, _ = fmt.Fprintln(s.diagnosticStream(), out...)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The colored and uncolored paths don't format identically. When useColor() is true, the variadic operands are collapsed by color.Sprint(out...) (which concatenates with no separator between non-string operands beyond Go's Sprint rules); when it's false, Fprintln(w, out...) inserts a space between operands.

Verified empirically: Info("\t", cmd) produces "\tmyscript" with color enabled and "\t myscript" without. Callers that pass an explicit indent/prefix as a separate operand get different spacing depending on whether stdout is a TTY.

Same pattern applies to Success, Info, and Warning. Cleanest fix is to normalize once before branching:

msg := fmt.Sprint(out...)
if s.useColor() {
    msg = color.New(color.Yellow).Sprint(msg)
}
_, _ = fmt.Fprintln(s.diagnosticStream(), msg)

Comment thread commands/status.go
Services: make([]statusServiceJSON, 0, len(statuses)),
Count: len(statuses),
}
for _, ss := range statuses {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

statuses is appended in the order results arrive on chStatus, which is goroutine-completion order — so the JSON services array ordering varies between runs on the same project.

The table path doesn't have this problem because the renderer applies SortBy(1). For a machine-readable format the nondeterminism is worse than for a table: it breaks golden-file tests, diffing two kool status --output json runs, and any agent caching keyed on the output.

Suggest sorting statuses by service name before building the payload.

Comment thread core/shell/shell.go
if s.IsJSONOutput() {
return s.errStream
}
return s.outStream

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

diagnosticStream() correctly routes diagnostics to errStream in JSON mode, but useColor() (just below) still decides colorization from outStream. The two disagree exactly when they matter most.

Concretely: kool start --output json 2> log.txt leaves stdout on a TTY, so useColor() returns true and ANSI escape codes get written into the redirected log.txt. The inverse also happens — piping stdout while stderr stays a TTY drops color from messages that could have had it.

Color should be decided from the stream actually being written to, i.e. IsTerminal(s.diagnosticStream()) for the diagnostic helpers.

Comment thread commands/logs.go
}

var output string
if output, err = l.Shell().Exec(l.logs, args...); err != nil {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The non-follow path goes through Shell().Exec, which uses CombinedOutput — so docker compose's own stderr chatter (WARN[0000] ..., orphan-container notices, deprecation warnings) is interleaved into output and then fed line-by-line through parseLogLine, producing bogus log entries in the JSON stream.

The --follow path gets this right by taking only StdoutPipe(). Worth making the two consistent, so the same command with and without -f doesn't yield structurally different data.

Comment thread commands/root.go
env.Set("KOOL_VERBOSE", verbose.Value.String())
}

if output := cmd.Flags().Lookup("output"); output != nil && output.Value.String() == "json" {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comparison is an exact match against "json", so anything else is silently ignored: --output JSON, --output jsonl, --output yaml, or a typo like --output jsno all fall through to normal human-readable output with no error.

For a flag whose entire purpose is machine consumption, silent fallback is a bad failure mode — the caller gets table output where it expected JSON and has to figure out why. Worth rejecting unknown values explicitly (and, if it's cheap, case-folding so --output JSON works).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

release-drafter:added Release drafter: Added release-drafter:changed Release drafter: Changed release-drafter:minor Release drafter: Minor

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants