Skip to content

feat(config): resolve extension versions from the extension's source tree - #192

Open
mobileoverlord wants to merge 2 commits into
mainfrom
jschneck/hermetic-ext-version-source
Open

feat(config): resolve extension versions from the extension's source tree#192
mobileoverlord wants to merge 2 commits into
mainfrom
jschneck/hermetic-ext-version-source

Conversation

@mobileoverlord

Copy link
Copy Markdown
Contributor

Problem

An extension's version is an identity field — it must resolve to the same string no matter who reads the config. {{ env.VAR }} can't do that: it binds the value to the caller's environment, and the job that packages an extension and the build that consumes it are by definition different environments.

The in-source program extensions (avocado-ext-cli, avocado-ext-connect, avocado-ext-tunnels) declare version: '{{ env.AVOCADO_EXT_VERSION }}', with CI reading Cargo.toml and exporting the variable. Point a project at one of them with source: { type: git | path } to test a local checkout and it breaks: the variable isn't set, env::resolve warns and substitutes "", and validation fails.

Error: Extension 'avocado-ext-cli' has invalid version ''. Version must be in
       semantic versioning format (e.g., '1.0.0', '2.1.3')

Three hard failure sites (ext/build.rs, ext/image.rs, ext/package.rs) plus one silent one: runtime/build.rs never validates and composes the artifact name avocado-ext-cli-.raw.

This also violates an invariant the code already states at utils/config.rs"switching between package/path/git doesn't change the composed config hash" — and re-introduces the exact failure class docs/features/extension-versioning-redesign.md was written to eliminate ("wildcards that can't resolve for git/path extensions").

Approach

Keep interpolation; move version off the ambient environment and onto a file inside the extension's own source tree, which is present in every consumption mode — the working copy for path, the clone for git, the RPM payload for package.

version:
  file: VERSION            # no key => whole file, trimmed

version:
  file: Cargo.toml         # parse and navigate
  key: package.version

key is the discriminator, so there's no format guessing for a plain VERSION file, and file: Cargo.toml with no key is a literal read rather than a surprise parse. format (toml/json/yaml) is inferred from the extension and can be set explicitly; it's rejected without a key. file must stay inside the extension — no .., no absolute paths.

Resolution runs during composition, before the final interpolation pass, so every existing consumer still sees version as a plain string and nothing downstream changes.

{{ avocado.* }} and {{ config.* }} remain fine elsewhere — those are consumer-context values that are supposed to differ per build. It's specifically env in an identity field that can't work.

Notable pieces

Extracted the extension-tree read ladder. Reading the version file needs the same access ladder that already existed for an extension's avocado.yaml — host source path, in-container includes dir, SDK volume mountpoint, throwaway container cat, dev fallback. That was open-coded in config.rs with a read/error/continue block per strategy; it's now utils::ext_source_reader, so both reads go by the same route. config.rs is a net ~340-line deletion. This also fixes an incidental bug: only type: path accepted avocado.yml, so a .yml extension shipped in an RPM was invisible once installed.

The payload must carry the version file. The published avocado.yaml keeps the provider rather than a baked literal, so ext package always appends the provider's file — including on the branch where an explicit package_files list replaces the defaults wholesale. Forgetting it would only surface when someone consumed the package.

The legacy bake is retained, and guarded. bake_extension_version still handles the {{ env.AVOCADO_EXT_VERSION }} form, which genuinely can't resolve downstream. It's skipped for provider-based extensions — baking one would strand the provider's file:/key: lines under a replaced version: scalar. test_bake_extension_version_would_corrupt_a_provider_block documents exactly that.

Also: config show --detail reports each extension's resolved version (release CI will read this instead of hand-parsing YAML, which can't see through a provider), and an invalid version now names the config file it came from.

Rollout — no extension migrates here

setup-avocado-cli installs latest, and a program-tracking extension's release workflow runs on the same tag that publishes the new CLI. So the provider has to be in a released CLI before any avocado.yaml can depend on it, or the first tag after merge installs a pre-feature CLI and fails to publish.

  1. This PR — the mechanism only. avocado-cli, avocado-conn, and avocado-rat stay on {{ env.AVOCADO_EXT_VERSION }} and keep working through the bake.
  2. After a release carrying this — migrate the three avocado.yaml files, drop AVOCADO_EXT_VERSION / the ext-version input from their workflows and from avocado-linux/actions, and delete the bake. Staged locally, not yet opened.

Step 2 is a payload format change: a CLI predating providers reads the mapping, coerces it to "0", and fails with invalid version '0'. There's no graceful degradation — cli_requirement is top-level and isn't merged from a remote extension's config — so publish to next first.

Verification

cargo fmt --check, clippy --all-targets --all-features -D warnings, and the full suite are clean (1163 lib + 1171 bin + 8 new integration tests).

Beyond tests, verified against real artifacts:

  • All three repos resolve from their own Cargo.toml with AVOCADO_EXT_VERSION unset and no warning — cli 1.0.0-rc.1, conn/rat 0.1.0.
  • Consuming avocado-cli via type: path — the reported bug — resolves 1.0.0-rc.1.
  • Built a real RPM from a scratch extension that omits its VERSION from package_files. Payload came out as VERSION, avocado.yaml, extra.txt; Version: 0.4.2; the shipped avocado.yaml kept the provider unbaked. A consumer then resolved 0.4.2 from that extracted payload. Deleting VERSION from it produces a four-level error chain naming the extension, the field, the file, and the root searched.
  • The legacy env path still bakes exactly as before.

Unrelated issue found while verifying

avocado ext package avocado-ext-cli fails with error: line 5: Illegal char '-' (0x2d) in: Version: 1.0.0-rc.1. This is pre-existing and unrelated — a scratch extension with a plain literal version: '1.0.0-rc.1' and no provider fails identically, and git show HEAD~1:src/commands/ext/package.rs has no sanitization either. CI exported that exact string, so the ext release workflow was already broken on RC tags. RPM wants 1.0.0~rc.1. Worth fixing separately before the next tag.


CI note: GitHub Actions was in a critical outage when this was opened, so checks may be delayed or spuriously failing.

Copilot AI lite review requested due to automatic review settings August 6, 2026 18:41

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds support for extension version values that are sourced from an extension’s own tree (e.g., Cargo.toml / VERSION) so that composed versions remain stable across path/git/package sources and packaging can reliably ship the required inputs.

Changes:

  • Introduces version: { file, key, format } providers and resolves them during config composition (and in the get_merged_section path used by ext package).
  • Factors extension-tree reading into a reusable ExtSourceReader (dir/volume/container/dev-fallback) so both config and version resolution use the same access strategy.
  • Updates ext package to include provider files in payloads and skip version “baking” when a provider is used; improves version validation errors to include origin path.

Reviewed changes

Copilot reviewed 13 out of 14 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
tests/ext_version_source.rs New integration coverage for provider resolution across source kinds and error reporting.
src/utils/version.rs Adds validate_ext_version to include the originating config path in semver errors.
src/utils/mod.rs Exposes new utility modules for extension source reading and version providers.
src/utils/ext_version_source.rs Implements provider parsing/validation and in-config resolution to literal strings.
src/utils/ext_source_reader.rs Centralizes reading extension files/configs from dir/container/volume/dev-fallback.
src/utils/config_edit.rs Adds a test demonstrating why baking must be skipped for provider-based versions.
src/utils/config.rs Integrates discovery/readers, resolves provider versions during composition, tracks sources.
src/commands/ext/package.rs Ensures provider file is packaged; skips baking when provider is present.
src/commands/ext/image.rs Uses validate_ext_version to report better version error context.
src/commands/ext/build.rs Uses validate_ext_version to report better version error context.
src/commands/config_show.rs Includes resolved extension version in config show --detail output.
docs/features/extension-versioning-redesign.md Documents provider syntax, rationale, rollout, and baking behavior.
Cargo.toml Adds toml dependency needed to extract versions by key from TOML files.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/utils/ext_source_reader.rs
Comment thread src/utils/ext_source_reader.rs
Comment thread src/utils/config.rs Outdated
Comment thread src/utils/version.rs
…tree

An extension's `version` is an identity field: it has to resolve to the
same string no matter who reads the config. `{{ env.VAR }}` cannot do
that — it binds the value to the caller's environment, and the job that
packages an extension and the build that consumes it are by definition
different environments.

That is why an extension declaring `version: '{{ env.AVOCADO_EXT_VERSION }}'`
(the in-source program extensions do) breaks the moment you point a
project at it with `source: { type: git | path }` to test a local
checkout: the variable isn't set, `env::resolve` warns and substitutes
"", and `ext build` / `ext image` / `ext package` then fail semver
validation with `invalid version ''`. `runtime build` doesn't even
validate — it composes the artifact name `<ext>-.raw`.

Add a mapping form for `version` that names a file inside the
extension's OWN source tree, which is present in every consumption mode
(working copy for `path`, clone for `git`, RPM payload for `package`):

    version:
      file: VERSION                # no key => whole file, trimmed

    version:
      file: Cargo.toml             # parse and navigate
      key: package.version

`key` is the discriminator, so there is no format guessing for a plain
VERSION file, and `file: Cargo.toml` with no `key` is a literal read
rather than a surprise parse. `file` must stay inside the extension —
no `..`, no absolute paths. Resolution happens during composition,
before the final interpolation pass, so every existing consumer still
sees `version` as a plain string and nothing downstream changes.

Reading the version file needs the same access ladder that already
existed to read an extension's avocado.yaml — host source path,
in-container includes dir, SDK volume mountpoint, throwaway container,
dev fallback. That ladder was open-coded in config.rs with a
read/error/continue block per strategy; extract it to
`utils::ext_source_reader` so both reads go by the same route. Doing so
also fixes an incidental bug: only `type: path` accepted `avocado.yml`,
so a `.yml` extension shipped in an RPM was invisible once installed.

`ext package` keeps the version file in the payload even when the
extension declares an explicit `package_files` list (which otherwise
replaces the defaults wholesale) — the published avocado.yaml keeps the
provider rather than a baked literal, so a payload missing that file
would be unresolvable for every consumer. The legacy env-template bake
is retained for the form that genuinely cannot resolve downstream, and
is skipped for provider-based extensions.

Also: `config show --detail` now reports each extension's resolved
version, and an invalid version names the config file it came from.

No extension migrates in this change — that needs a released CLI that
understands the provider, since `setup-avocado-cli` installs `latest`.
See the Rollout section in the design doc.
… versions

- `ExtSourceReader::read` resolves the joined path and requires it to stay
  under the extension root. `version.file` already rejects absolute paths
  and `..`, but a symlink inside a `type: git` tree could still point out.
- `get_merged_section_with_board` only resolves a provider for a direct
  `extensions.<name>` path; a deeper one was read as an extension named
  `<name>.<sub>`.
- The invalid-version error now shows a pre-release and a build-metadata
  example, both of which the provider flow accepts.
@mobileoverlord
mobileoverlord force-pushed the jschneck/hermetic-ext-version-source branch from 1fad6f0 to 1682022 Compare August 12, 2026 20:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants