Skip to content

feat: federate image pull credentials to the cell - #248

Merged
scotwells merged 3 commits into
mainfrom
feat/image-pull-secrets-referenced-data
Aug 21, 2026
Merged

feat: federate image pull credentials to the cell#248
scotwells merged 3 commits into
mainfrom
feat/image-pull-secrets-referenced-data

Conversation

@scotwells

@scotwells scotwells commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Why

Deploying from a private registry is a basic expectation, and today Datum compute does not meet it. A user can declare spec.template.spec.runtime.sandbox.imagePullSecrets on a Workload, the field validates, and then the Instance sits in image-pull failure forever — because the referenced-data collector ignores imagePullSecrets entirely, so the credential never leaves the project control plane and does not exist in the cell namespace where the Instance actually runs.

This carries the credential to the cell, using the exact machinery that already delivers ConfigMaps and Secrets referenced by env, envFrom, and volumes.

What changed

The referenced-data collector now treats runtime.sandbox.imagePullSecrets[] as a fourth Secret source, alongside env.valueFrom, envFrom, and volumes. It uses the same ref-key construction, the same Workload-namespace handling, and the same dedup as the existing three paths, so a Secret referenced both as a pull credential and as a volume federates exactly once.

Because every stage downstream of the collector keys off ObjectRef.Kind rather than off where the reference came from, a pull credential now travels the whole path unchanged: the admission-time SubjectAccessReview (the referencing user must be able to get the Secret), the companion Secret written into ns-{project-uid} on the Karmada hub, the expected-referenced-data annotation, the city-* PropagationPolicy (whose ConfigMap/Secret selectors are label-based and source-agnostic), the Instance ReferencedData scheduling gate, and the ref-counted reclaim when the reference goes away.

One behavioural decision worth calling out: a Secret used as a pull credential is never treated as an optional source, even if the same Secret is also referenced with optional: true somewhere else in the template. Optional sources are silently skipped when missing or oversized; for a pull credential that would swap a clear ReferencedDataReady=False condition on the WorkloadDeployment for an opaque image-pull failure at the cell.

Deletion and reclaim

Removing an entry from imagePullSecrets releases that companion from the hub, and Karmada removes it from the cell. Reclaim is ref-counted per companion across every WorkloadDeployment that references it and across every reference source, so dropping the pull-credential reference to a Secret that is still mounted as a volume — or still referenced by another Workload in the same namespace — leaves the companion in place. Tests cover all three cases.

RBAC and memory footprint

No RBAC change is needed. The hub-side compute-manager ClusterRole in config/base/downstream-rbac already grants configmaps and secrets, which is what made federated mounts work in the first place; a pull credential is a Secret and is covered by the same grant. No generated role file is touched.

No new watch or informer is introduced either. The referenced-data controller already watches source Secrets on the project plane and the Instance controller already watches companion Secrets on the cell, both established in #129; this change only widens which Secrets those existing watches happen to match. Memory is therefore bounded exactly as before, with no new Secret cache.

Testing

make test and make lint pass. Unit coverage: a realistic template (shaped from the existing e2e WorkloadDeployment fixture, with a real kubernetes.io/dockerconfigjson Secret) producing the expected referenced-data entries; dedup and sort across multiple pull credentials; a pull credential collapsing with an identically-named env reference; the scheduling gate being stamped when a pull credential is the only referenced data; the credential landing on the hub with the referenced-data label and its docker-config type and payload intact; the reclaim cases above. Four of these fail on main before the collector change.

A new chainsaw suite, test/e2e/referenced-data-pull-secret, asserts the full cross-plane path against a live Kind + Karmada environment: source credential in the project namespace, companion on the Karmada hub, companion on the pop-dfw cell, Instance ReferencedData gate cleared, then the reference dropped and the companion reclaimed from both hub and cell while the still-referenced ConfigMap companion survives. It runs green in CI alongside the existing suites.

Scope

Tracking issue: datum-cloud/unikraft-provider#26.

This is one of two hops. This PR gets the credential bytes to the cell namespace; the companion provider PR maps imagePullSecrets onto the downstream Instance so the runtime actually authenticates with it. Neither PR alone delivers an end-to-end private pull.

The runtime half is no longer a blocker — kraftlet 0.6.0-staging.28 ships pull-credential support (datum-cloud/unikraft-provider#70, datum-cloud/unikraft-provider#121), so the "vendor-blocked" framing in the image-pull-credentials RFC is stale and should be revised.

🤖 Generated with Claude Code

scotwells and others added 2 commits August 21, 2026 12:16
Deploying from a private registry is a basic expectation, but today a user who
wires up an image pull Secret gets an Instance that cannot pull its image: the
referenced-data collector ignores runtime.sandbox.imagePullSecrets entirely, so
the credential never leaves the project control plane.

Collect imagePullSecrets as a Secret reference source. Everything downstream —
the SubjectAccessReview on the referencing user, the companion object on the
Karmada hub, the expected-referenced-data annotation, the PropagationPolicy that
carries companions to the cell, the Instance scheduling gate, and the ref-counted
reclaim — keys off ObjectRef kind and is source-agnostic, so a pull credential
now travels the same path as a mounted ConfigMap or Secret.

A pull credential is never treated as an optional source. Optional sources are
silently skipped when missing or oversized; for a credential that would trade a
clear condition on the WorkloadDeployment for an opaque image-pull failure at the
cell.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Chainsaw applies a merge patch, so omitting the key left the original entry in
place and the reclaim assertion timed out against a still-referenced companion.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@scotwells
scotwells enabled auto-merge August 21, 2026 18:04
@scotwells
scotwells merged commit bd34486 into main Aug 21, 2026
10 checks passed
@scotwells
scotwells deleted the feat/image-pull-secrets-referenced-data branch August 21, 2026 18:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants