feat: federate image pull credentials to the cell - #248
Merged
Conversation
Deploying from a private registry is a basic expectation, but today a user who wires up an image pull Secret gets an Instance that cannot pull its image: the referenced-data collector ignores runtime.sandbox.imagePullSecrets entirely, so the credential never leaves the project control plane. Collect imagePullSecrets as a Secret reference source. Everything downstream — the SubjectAccessReview on the referencing user, the companion object on the Karmada hub, the expected-referenced-data annotation, the PropagationPolicy that carries companions to the cell, the Instance scheduling gate, and the ref-counted reclaim — keys off ObjectRef kind and is source-agnostic, so a pull credential now travels the same path as a mounted ConfigMap or Secret. A pull credential is never treated as an optional source. Optional sources are silently skipped when missing or oversized; for a credential that would trade a clear condition on the WorkloadDeployment for an opaque image-pull failure at the cell. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Chainsaw applies a merge patch, so omitting the key left the original entry in place and the reclaim assertion timed out against a still-referenced companion. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
scotwells
enabled auto-merge
August 21, 2026 18:04
savme
approved these changes
Aug 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Deploying from a private registry is a basic expectation, and today Datum compute does not meet it. A user can declare
spec.template.spec.runtime.sandbox.imagePullSecretson a Workload, the field validates, and then the Instance sits in image-pull failure forever — because the referenced-data collector ignoresimagePullSecretsentirely, so the credential never leaves the project control plane and does not exist in the cell namespace where the Instance actually runs.This carries the credential to the cell, using the exact machinery that already delivers ConfigMaps and Secrets referenced by
env,envFrom, andvolumes.What changed
The referenced-data collector now treats
runtime.sandbox.imagePullSecrets[]as a fourth Secret source, alongsideenv.valueFrom,envFrom, andvolumes. It uses the same ref-key construction, the same Workload-namespace handling, and the same dedup as the existing three paths, so a Secret referenced both as a pull credential and as a volume federates exactly once.Because every stage downstream of the collector keys off
ObjectRef.Kindrather than off where the reference came from, a pull credential now travels the whole path unchanged: the admission-time SubjectAccessReview (the referencing user must be able togetthe Secret), the companion Secret written intons-{project-uid}on the Karmada hub, theexpected-referenced-dataannotation, thecity-*PropagationPolicy (whose ConfigMap/Secret selectors are label-based and source-agnostic), the InstanceReferencedDatascheduling gate, and the ref-counted reclaim when the reference goes away.One behavioural decision worth calling out: a Secret used as a pull credential is never treated as an optional source, even if the same Secret is also referenced with
optional: truesomewhere else in the template. Optional sources are silently skipped when missing or oversized; for a pull credential that would swap a clearReferencedDataReady=Falsecondition on the WorkloadDeployment for an opaque image-pull failure at the cell.Deletion and reclaim
Removing an entry from
imagePullSecretsreleases that companion from the hub, and Karmada removes it from the cell. Reclaim is ref-counted per companion across every WorkloadDeployment that references it and across every reference source, so dropping the pull-credential reference to a Secret that is still mounted as a volume — or still referenced by another Workload in the same namespace — leaves the companion in place. Tests cover all three cases.RBAC and memory footprint
No RBAC change is needed. The hub-side compute-manager ClusterRole in
config/base/downstream-rbacalready grantsconfigmapsandsecrets, which is what made federated mounts work in the first place; a pull credential is a Secret and is covered by the same grant. No generated role file is touched.No new watch or informer is introduced either. The referenced-data controller already watches source Secrets on the project plane and the Instance controller already watches companion Secrets on the cell, both established in #129; this change only widens which Secrets those existing watches happen to match. Memory is therefore bounded exactly as before, with no new Secret cache.
Testing
make testandmake lintpass. Unit coverage: a realistic template (shaped from the existing e2e WorkloadDeployment fixture, with a realkubernetes.io/dockerconfigjsonSecret) producing the expected referenced-data entries; dedup and sort across multiple pull credentials; a pull credential collapsing with an identically-named env reference; the scheduling gate being stamped when a pull credential is the only referenced data; the credential landing on the hub with the referenced-data label and its docker-config type and payload intact; the reclaim cases above. Four of these fail onmainbefore the collector change.A new chainsaw suite,
test/e2e/referenced-data-pull-secret, asserts the full cross-plane path against a live Kind + Karmada environment: source credential in the project namespace, companion on the Karmada hub, companion on the pop-dfw cell, InstanceReferencedDatagate cleared, then the reference dropped and the companion reclaimed from both hub and cell while the still-referenced ConfigMap companion survives. It runs green in CI alongside the existing suites.Scope
Tracking issue: datum-cloud/unikraft-provider#26.
This is one of two hops. This PR gets the credential bytes to the cell namespace; the companion provider PR maps
imagePullSecretsonto the downstream Instance so the runtime actually authenticates with it. Neither PR alone delivers an end-to-end private pull.The runtime half is no longer a blocker — kraftlet 0.6.0-staging.28 ships pull-credential support (datum-cloud/unikraft-provider#70, datum-cloud/unikraft-provider#121), so the "vendor-blocked" framing in the image-pull-credentials RFC is stale and should be revised.
🤖 Generated with Claude Code