Skip to content

feat(exec): run python inside containers - #441

Merged
jahvon merged 1 commit into
feat/mcp-run-pythonfrom
feat/python-container
Aug 27, 2026
Merged

feat(exec): run python inside containers#441
jahvon merged 1 commit into
feat/mcp-run-pythonfrom
feat/python-container

Conversation

@jahvon

@jahvon jahvon commented Aug 27, 2026

Copy link
Copy Markdown
Member

Part 3/5. Stacked on #440.

Summary

Combines interpreter: python with the existing container: block, so a pinned Python toolchain needs no local install.

exec:
  interpreter: python
  cmd: |
    import sys
    print(sys.version)
  container:
    image: python:3.13-alpine

Notable Changes

  • Inline code is bind-mounted read-only as a script and passed via spec.Script, never spec.Cmd. buildRunArgs maps Cmd to <entrypoint> -c <code>, which would put user code in the process table and cost traceback line numbers — the same trade-off Part 1 avoided on the host. The existing "script lives outside every mount" branch already did exactly this for files, so the generated temp script reuses it and internal/services/run/container.go needed no change at all.
  • The default entrypoint follows the interpretersh for a shell command, python3 for a Python one. Resolved in the runner rather than on ExecContainer, keeping the types package unaware of how flow launches containers. An explicit container.entrypoint still wins, including the empty form (which, with Python, only works if the image's own ENTRYPOINT is an interpreter — documented).
  • Host interpreter discovery stops at the container boundary. VIRTUAL_ENV, PYTHONPATH, PYTHONHOME, and FLOW_PYTHON_BIN are dropped from the container environment: those host paths either do not exist inside it or, worse, resolve to an unrelated mounted directory. PYTHONUNBUFFERED is set there too.
  • The temp script cleanup is registered as a context callback next to ForceRemoveContainer, so a run abandoned by the runner's timeout goroutine does not leak it.

Testing

  • Unit tests assert the built spec: Cmd empty, Script set to a .py, the script mount present and read-only, the python3 entrypoint default, explicit-entrypoint override, empty-entrypoint passthrough, that a shell command still gets sh, and that host python env vars are dropped while unrelated ones survive.
  • An e2e case runs Python in a python:3.13-alpine image, behind the suite's existing runtime guard.
  • flow validate passes; generate produces no diff.

Verification note: I have no container runtime on this machine, so the live container run was not executed locally — it is covered by the e2e case, which only executes where a runtime is available (CI). The spec and argv construction are verified by the unit tests above.

🤖 Generated with Claude Code

https://claude.ai/code/session_01R328pa3FUUfga4gYah1iQi

@codecov

codecov Bot commented Aug 27, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 91.17647% with 3 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
internal/runner/exec/container.go 89.65% 2 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

@jahvon
jahvon force-pushed the feat/python-container branch from dce32d4 to 60cd979 Compare August 27, 2026 04:28
@jahvon
jahvon force-pushed the feat/python-container branch from 60cd979 to c3b0edb Compare August 27, 2026 05:48
@jahvon
jahvon force-pushed the feat/python-container branch from c3b0edb to 56449b2 Compare August 27, 2026 06:24
@jahvon
jahvon force-pushed the feat/python-container branch from 56449b2 to 3782ca3 Compare August 27, 2026 06:44
Combines `interpreter: python` with the `container:` block so a pinned Python
toolchain needs no local install.

Inline code is bind-mounted read-only as a script and passed via spec.Script
rather than spec.Cmd: buildRunArgs maps Cmd to `<entrypoint> -c <code>`, which
would put user code in the process table and cost traceback line numbers. The
existing "script lives outside every mount" branch already did exactly this for
files, so the generated temp script reuses it.

The default entrypoint now follows the interpreter — sh for a shell command,
python3 for a python one — resolved in the runner rather than on ExecContainer,
keeping the types package unaware of how flow launches containers. An explicit
container.entrypoint still wins, including the empty form.

Host interpreter discovery deliberately stops at the container boundary:
VIRTUAL_ENV, PYTHONPATH, PYTHONHOME, and FLOW_PYTHON_BIN are dropped from the
container environment, since those host paths either do not exist inside it or
resolve to an unrelated mounted directory.

Verified through spec and argv unit tests; the live container run is covered by
the e2e case, which needs a runtime and so only executes in CI.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01R328pa3FUUfga4gYah1iQi
@jahvon
jahvon force-pushed the feat/python-container branch from 3782ca3 to ff6c97c Compare August 27, 2026 06:49
@jahvon
jahvon merged commit da47cdf into main Aug 27, 2026
23 checks passed
@jahvon
jahvon deleted the feat/python-container branch August 27, 2026 13:52
jahvon added a commit that referenced this pull request Aug 27, 2026
**Part 4/5.** Stacked on #441.

# Summary

Inline `cmd` steps in `serial`/`parallel` executables now take their own
`interpreter`, so one workflow can mix shell and Python without
splitting into separate executables.

```yaml
serial:
  execs:
    - cmd: ./fetch-data.sh
    - cmd: |
        import json
        print(json.load(open("data.json"))["total"])
      interpreter: python
```

**Notable Changes**

- Both step configs `$ref` the same `ExecInterpreter` definition the
`exec` type uses, so there is one enum rather than three parallel ones.
(Part 1 hoisted it to a top-level definition for exactly this.)
- A step that omits `interpreter` runs under the shell as before; a
`ref` step ignores the field — the referenced executable brings its own.
- **This lifts the interim restriction from #440**: `--interpreter` now
applies to every `--cmd` in an invocation, in serial and parallel mode
alike.
- `ExecutableForCmd` takes the interpreter as a new parameter (5 call
sites). Its unused `int` parameter is left alone to keep the diff to the
one concern.

# Testing

- E2E: a serial executable mixing a shell step and a Python step, each
running under its own interpreter; a step without an interpreter staying
on the shell (asserted via `echo`, a shell builtin, so it only passes if
the step really stayed there); and `--interpreter` applied across a
multi-command batch.
- The e2e case that previously asserted the multi-command rejection is
replaced with one asserting it now works.
- `flow validate` passes; `generate` produces no diff.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01R328pa3FUUfga4gYah1iQi

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant