fix: give a module its own API back, so it can call itself - #45
Open
TomChv wants to merge 5 commits into
Open
Conversation
A module's bindings carried core and its dependencies but not itself, so `dag.<self>()` did not exist and the self call was gone. The Go SDK has it: internal/dagger/<self>.gen.go, beside the dependency's. The schema a module's bindings are generated from is core plus dependencies -- at generation time the module is defined by source that has not been compiled, so its own API cannot be in it. The schema that does carry it is the module's own client-facing one, and reading that means loading the module, which needs the bindings that do not exist yet. So generation runs in two passes: bindings, entrypoint and config as before, then the bindings again with the module's own API folded in. Both passes run every time from the same inputs, so the result does not depend on what was on disk when the run started -- without that, a fresh module's first generation had no self bindings and the second did, and `dagger generate` was no longer idempotent. Only the bindings are redone; the entrypoint comes from scanning the user's source, which the self types do not change, and is the expensive half. Folding self needs its own path. Include selects a module's types by their type-level sourceMap directive, and a module's own types do not carry one in its own client schema -- only the Query field that reaches them does. So the fields come from Include and the types come from walking out of them. Also stop hiding generated files from git in the fixtures that are CLI 1.0 scopes. The engine cannot load a module whose committed files are ignored, so a dependent -- or the module itself, now -- fails with "committed generated file is missing" while the file sits right there. That is what the error means by "commit every generated file", and it made `dagger generate` fail on every run after the first. Signed-off-by: Tom Chauveau <tom@dagger.io>
A CLI 1.0 module is built from committed generated files, and the engine does not see files git ignores. So a fixture that ignored its own sdk/ could not be loaded once those files existed on disk: a dependent failed with "committed generated file sdk/client.gen.ts is missing" while the file sat right there, and so did the module itself, now that generating it reads its own schema. The failure only appears after a first successful run, which is why it looked like flaky state: on a clean tree the files are written into the workspace the generator threads along, and everything resolves. Only the ignore rules change. The generated output stays untracked, which is what the fixtures that never ignored it already do -- committing it would also pin the typescript version in the one fixture that has to start unpinned for generate-typescript-pin-check to mean anything. Signed-off-by: Tom Chauveau <tom@dagger.io>
Two things made a module's own bindings the odd one out. They lived in client.gen.ts while every other module's went to its own file. Nothing distinguishes them: a module reaches its own API through the same client it reaches a dependency's, so the one binding a reader goes looking for was the only one not where the others are. Every module in the schema now splits, the one being generated for included. Splitting it needs the types attributed to it, and a module's own types carry no sourceMap directive in its own client schema -- only the Query field that reaches them does. So the walk that collects them now stamps them, which is what lets the existing split, re-export and augmentation machinery treat them like a dependency's. The module-facing schema is what separates the module's own types from the core ones the walk passes through: core is already there. And the accessor was PascalCase. Schema fields are camelCase by convention, so this only showed on a module whose name starts with an underscore -- the leading underscore survives into the field name, and dag.TestSdkMaxDev() was the one member of the API not shaped like the rest. Member names are now lowered at the sites that render a schema field as a TypeScript member; the wire name is rendered from the raw field, so this changes what the caller writes, not what is selected. Signed-off-by: Tom Chauveau <tom@dagger.io>
A module's own API was folded into its bindings but left out of the package under clients/, so the one module a caller is most likely to want from that package -- the one it sits inside -- was the only one missing from it. It goes in as a local target like any other: the generated bootstrap serves it by resolving the scope against the workspace, so a caller outside the module reaches it the same way it reaches a target that was added on purpose. Still never a manifest dependency, because a module does not depend on itself. The self target is now resolved once, in generateScope between the two passes, rather than inside the first: it is not available until that pass has written the files that make the module loadable, and both halves need it. Signed-off-by: Tom Chauveau <tom@dagger.io>
A module writes its targets twice, and now its own API lands in both, so every module has a clients/ directory whether or not it has targets of its own. That reads like duplication worth removing until you know clients are being unified and the two directories merge into one -- so write it down, rather than leave the next reader to re-litigate it. Signed-off-by: Tom Chauveau <tom@dagger.io>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A module's generated bindings carried core and its dependencies but not itself,
so
dag.<self>()did not exist and the self call was gone. The Go SDK has it —internal/dagger/<self>.gen.go, sitting beside the dependency's.Why it was missing
The schema a module's bindings are generated from is core plus its dependencies.
At generation time the module is defined by source that has not been compiled
yet, so its own API cannot be in it. The schema that does carry it is the
module's own client-facing one — and reading that means loading the module,
which needs the bindings that do not exist yet.
Two passes
Generation runs bindings, entrypoint and config as before, then the bindings
again with the module's own API folded in. The self target is resolved once
between the passes: it is not available until the first has written the files
that make the module loadable, and both halves of the scope need it.
Both passes run every time, from the same inputs, so the result does not depend
on what was on disk when the run started. Without that, a fresh module's first
generation had no self bindings and its second did, and
dagger generatewas nolonger idempotent —
generate-idempotent-checkcatches exactly this.Only the bindings are redone. The entrypoint comes from scanning the user's
source, which the self types do not change, and it is the expensive half.
A target like any other
The module's own API is rendered wherever a target is: its own file in the
module's bindings, so its source can call itself, and the standalone package, so
callers outside it can too.
In the package it is a local target: the bootstrap serves it by resolving the
scope against the workspace, the same way it serves one that was added on
purpose. It is never a manifest dependency — a module does not depend on itself,
and the engine already serves it into its own session.
Every module therefore gets a
clients/directory, including one with no targetsof its own. That is an intermediate state: once clients are unified the two
directories merge into
clients/alone. Noted indesign/module-max.md.Splitting the module's own types out needs them attributed to it, and they carry
no
sourceMapdirective in its own client schema — only the Query field thatreaches them does. So the walk that collects them stamps them, which is what lets
the existing split, re-export and augmentation machinery treat them like a
dependency's. The module-facing schema separates the module's own types from the
core ones the walk passes through: core is already there.
The accessor was also PascalCase. Schema fields are camelCase by convention, so
this only surfaced on a module whose name starts with an underscore — the
underscore survives into the field name. Member names are now lowered at the
three sites that render a schema field as a TypeScript member;
FormatNameisleft alone because it also renders class and interface names. The wire name comes
from the raw field, so this changes what the caller writes, not what is selected.
Generated files must not be gitignored
Found while fixing the above, and it had been breaking
dagger generateon everyrun after the first. A CLI 1.0 module is built from committed generated files,
and the engine does not see files git ignores — so a fixture that ignored its own
sdk/could not be loaded once those files existed on disk. A dependent failedwith
committed generated file "sdk/client.gen.ts" is missingwhile the file satright there, and so did the module itself once generation started reading its own
schema.
It only shows up after a first successful run, which is why it read as flaky
state: on a clean tree the files are written into the workspace the generator
threads along and everything resolves.
Only the ignore rules change; the generated output stays untracked, which is what
the fixtures that never ignored it already do.
Verified
generate-self-call-checkcovers both halves and the accessor casing, plus theexisting 47 e2e checks and the
helpers/*Go tests, from a clean tree.dag.<self>(), a remoteclient target and a local one all return values from the module they address,
and
dagger generateis idempotent across apply-and-regenerate.