Add design proposal: platform migration engine - #58
Conversation
Platform migrations are identified by a dense global integer that also serves as cluster state. Every new migration must claim the next number, so concurrent migration PRs conflict by construction; a single scalar cannot describe a branched history, so migrations are unbackportable; and the pending set comes from a number maintained in a different file than the migrations themselves, so silent skips are possible. The proposal replaces the integer with a content-addressed ID and an applied-set ledger, splits execution into a blocking pre-apply tier and an operator-driven background tier, and adds a machine-checkable contract around migration authoring: declared metadata, one shell dialect, a linter, and mandatory tests. Assisted-By: Claude <noreply@anthropic.com> Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
📝 WalkthroughWalkthroughThe proposal replaces integer-based platform migrations with content-addressed IDs, ledger-based tracking, explicit execution tiers, dependency ordering, package scoping, validation rules, and a phased legacy conversion. ChangesPlatform migration design
Estimated code review effort: 2 (Simple) | ~15 minutes 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 10
🧹 Nitpick comments (2)
design-proposals/platform-migrations/README.md (2)
61-70: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winAdd language tags to the fenced examples.
markdownlint-cli2reports MD040 for these three blocks. Addtext,sh, oryamlto each fence as appropriate.Also applies to: 102-106, 207-211
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@design-proposals/platform-migrations/README.md` around lines 61 - 70, Add language tags to all three fenced code blocks in the migration README, including the shown block and the blocks referenced at lines 102–106 and 207–211. Use the appropriate fence annotation for each block, such as text, sh, or yaml, so markdownlint MD040 passes.Source: Linters/SAST tools
120-135: 🩺 Stability & Availability | 🔵 TrivialAdd a ledger growth policy.
Retention deletes migration files but keeps every
m.*key. Package-scoped histories increase the key count. Define a size budget and a compaction or partitioning strategy before the ConfigMap becomes an availability limit.Also applies to: 244-248
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@design-proposals/platform-migrations/README.md` around lines 120 - 135, Update the Ledger design around the ConfigMap and its write mechanics to define a bounded size budget and an explicit growth-management strategy for m.* entries, such as compaction or partitioning. Ensure the policy accounts for retained package-scoped migration history and specifies how entries are handled when the budget is approached.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@design-proposals/platform-migrations/README.md`:
- Around line 59-69: Rename the documented identity scheme to a date/slug-based
identifier, removing any content-addressed implication. In the migration
identity documentation around the YYYYMMDD-slug example, define collision
handling for duplicate IDs and require verifying that an existing ID’s recorded
checksum matches its migration content; do not add content-derived ID generation
unless the scheme is intentionally changed.
- Around line 155-157: Update the migration immutability guard and ledger
checksum logic to include the transitive contents of migrations/lib/ alongside
each top-level migration script. Ensure changes to shared helpers alter the
recorded execution identity and cause mismatches to be rejected, or introduce an
equivalent versioned helper/image digest mechanism; apply the same behavior to
the related migration paths.
- Around line 110-118: Clarify the proposal’s requires semantics across
pre-apply and background execution tiers, including the referenced sections.
Either reject dependencies between tiers with explicit validation, or define a
shared barrier and deterministic topological ordering that prevents pre-apply
migrations from depending on pending background work and prevents background
jobs from starting before pre-apply dependencies are recorded.
- Around line 237-242: The migration seeding logic must populate every missing
legacy record below the cluster’s version, rather than requiring the ledger to
have no m.* keys. Update the step 3 runner behavior to iterate the legacy-map
entries below version and create only absent m.<id> records with outcome legacy,
preserving idempotency and existing records.
- Around line 174-191: Update the migration index generation around the
cozystack-migrations-index ConfigMap so its IDs are derived from the packaged
migration files rather than maintained separately. Add CI validation comparing
the chart’s packaged IDs, migration image IDs, and generated index IDs, and fail
on any mismatch to ensure every scheduled migration has a corresponding script
and background migration is not omitted.
- Around line 174-175: Update the pre-apply work-list gate described in the
migration proposal to include revoked IDs, triggering the runner when revoked
IDs are absent from the ledger so it can write the promised m.<id>: ... revoked
record. Apply the same revoked-minus-ledger handling to background IDs, using
either a revoked marker in the tier input or an equivalent gate condition.
- Around line 153-157: Update the migration proposal and hack/lint-migrations.sh
requirements to mandate an explicit fail-fast mechanism: each migration must use
set -e or the runner must invoke it with sh -e. Extend the linter/tests to
detect and enforce this requirement so a failed command cannot be followed by a
successful command and recorded as ok.
- Around line 135-136: Update the migration execution design to address crashes
between applying migration side effects and recording m.<id>: either make every
migration idempotent on retry or introduce a recoverable per-ID claim/lease that
prevents unsafe repetition. Document the selected behavior and ensure
reconciliation can safely resume pending migrations after a Job failure.
- Around line 176-193: Update the background Job specification described around
the migration index so each Job Pod explicitly uses a dedicated cozy-system
ServiceAccount rather than the namespace default. Define the ServiceAccount and
its cluster-admin binding, document their lifecycle with the migration
resources, and add a rendered identity test verifying
spec.template.spec.serviceAccountName.
- Around line 148-153: Update the migration configuration and runner logic
around the pre-apply/background tier handling and ledger outcome checks: enforce
that pre-apply migrations always use abort semantics, and make background non-ok
records consistently either retryable or terminal. Align the pending-selection
logic near the retry path and the terminal-outcome check so recorded non-ok
results follow that chosen behavior without contradicting the migration
contract.
---
Nitpick comments:
In `@design-proposals/platform-migrations/README.md`:
- Around line 61-70: Add language tags to all three fenced code blocks in the
migration README, including the shown block and the blocks referenced at lines
102–106 and 207–211. Use the appropriate fence annotation for each block, such
as text, sh, or yaml, so markdownlint MD040 passes.
- Around line 120-135: Update the Ledger design around the ConfigMap and its
write mechanics to define a bounded size budget and an explicit
growth-management strategy for m.* entries, such as compaction or partitioning.
Ensure the policy accounts for retained package-scoped migration history and
specifies how entries are handled when the budget is approached.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: b7515bb1-f3f1-4578-a580-74f3336782a2
📒 Files selected for processing (1)
design-proposals/platform-migrations/README.md
| Identity is `YYYYMMDD-slug`. Tier is the **directory**, so nothing is parsed at render time and there is no generated index to keep in sync: | ||
|
|
||
| ``` | ||
| packages/core/platform/images/migrations/migrations/ | ||
| 1 .. 53 # legacy integer set — frozen, never extended | ||
| lib/ | ||
| revoked # IDs that must not run (§8) | ||
| pre-apply/ # blocking, runs in the pre-upgrade hook Job | ||
| 20260812-redis-failover-group-label | ||
| background/ # non-blocking, run by cozystack-operator | ||
| 20260814-clickhouse-keeper-pvc-labels |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift
Align the identity name with the actual identity scheme.
YYYYMMDD[-NN]-slug is a sortable human-assigned identifier, not a content-addressed identifier. The checksum is recorded after execution and is not part of the ID. Two branches can assign the same ID to different content, and two authors can choose the same date and slug. Either derive IDs from content, or rename this to a date/slug scheme and define collision and ID-to-content verification rules.
🧰 Tools
🪛 markdownlint-cli2 (0.23.2)
[warning] 61-61: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@design-proposals/platform-migrations/README.md` around lines 59 - 69, Rename
the documented identity scheme to a date/slug-based identifier, removing any
content-addressed implication. In the migration identity documentation around
the YYYYMMDD-slug example, define collision handling for duplicate IDs and
require verifying that an existing ID’s recorded checksum matches its migration
content; do not add content-derived ID generation unless the scheme is
intentionally changed.
| `NN` covers intent and readability but enforces nothing. Where one migration genuinely depends on another — across any dates — it is declared and verified: | ||
|
|
||
| ```sh | ||
| # cozystack-migration: requires=20260801-etcd-crds | ||
| ``` | ||
|
|
||
| The runner topologically sorts on `requires` and fails loudly on a missing or cyclic dependency rather than guessing. | ||
|
|
||
| Merge order is deliberately not encoded and must not be relied on: a PR merged in June can carry a later date than one merged in July. What the scheme guarantees is that every cluster — fresh install or two years old — walks the same total order. |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift
Define requires behavior across execution tiers.
The proposal allows dependencies across dates, but the pre-apply runner processes only pre-apply/ while the operator processes background/ later. A pre-apply migration cannot wait for a pending background dependency. A background Job can also start before its pre-apply dependency is recorded. Reject cross-tier dependencies or define a shared dependency barrier and deterministic topological ordering.
Also applies to: 167-170, 191-193
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@design-proposals/platform-migrations/README.md` around lines 110 - 118,
Clarify the proposal’s requires semantics across pre-apply and background
execution tiers, including the referenced sections. Either reject dependencies
between tiers with explicit validation, or define a shared barrier and
deterministic topological ordering that prevents pre-apply migrations from
depending on pending background work and prevents background jobs from starting
before pre-apply dependencies are recorded.
| **Write mechanics.** Ledger keys are written with `kubectl patch --type merge`: atomic, no read-modify-write race between concurrent writers. The existing `stamp_cozystack_version` in `migrations/lib/cozystack-version.sh` uses `kubectl apply` and must keep touching **only** `version`. Apply's three-way merge prunes keys that were in last-applied-configuration but absent from the new manifest; patch-written `m.*` keys were never in last-applied, so they survive — subtle enough to deserve a dedicated test. It is the same class of bug migration `42` was written to fix for the `no-delete` label. | ||
|
|
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
rg -n -C 6 \
'record_migration|kubectl patch|ONLY=|generateName|ownerReferences|AlreadyExists' \
packages/core/platform internal/operator hackRepository: cozystack/community
Length of output: 335
🏁 Script executed:
set -euo pipefail
printf '%s\n' '--- candidate files ---'
git ls-files | rg '(^|/)(README\.md|.*migration.*|.*platform.*|.*cozystack-version.*|.*operator.*)$' | head -200
printf '%s\n' '--- relevant symbols ---'
rg -n -C 8 'record_migration|stamp_cozystack_version|kubectl patch|kubectl apply|generateName|ownerReferences|AlreadyExists|m\.' . --glob '!vendor/**' --glob '!node_modules/**' | head -400Repository: cozystack/community
Length of output: 50377
🏁 Script executed:
set -euo pipefail
printf '%s\n' '--- proposal sections ---'
sed -n '120,205p' design-proposals/platform-migrations/README.md
printf '%s\n' '--- failure and testing sections ---'
sed -n '282,307p' design-proposals/platform-migrations/README.md
printf '%s\n' '--- implementation files referenced by the proposal ---'
for path in migrations/lib/cozystack-version.sh packages/core/platform internal/operator hack; do
if [ -e "$path" ]; then
printf 'present: %s\n' "$path"
git ls-files "$path" | head -100
else
printf 'absent: %s\n' "$path"
fi
done
printf '%s\n' '--- all tracked migration/operator paths ---'
git ls-files | rg '(^|/)(migrations|hack|internal/operator|packages/core/platform)(/|$)' | head -200Repository: cozystack/community
Length of output: 12130
🏁 Script executed:
python3 - <<'PY'
from dataclasses import dataclass
`@dataclass`
class State:
ledger: set[str]
effects: int = 0
def run_once(state: State, migration_id: str, crash_after_effect: bool = False):
if migration_id in state.ledger:
return "skip"
state.effects += 1
if crash_after_effect:
return "crash"
state.ledger.add(migration_id)
return "recorded"
state = State(set())
first = run_once(state, "m.example", crash_after_effect=True)
second = run_once(state, "m.example")
print({
"first_attempt": first,
"second_attempt": second,
"effect_count": state.effects,
"ledger": sorted(state.ledger),
"rerun_occurred": state.effects == 2,
})
assert state.effects == 2
assert "m.example" in state.ledger
PYRepository: cozystack/community
Length of output: 282
Make migration execution idempotent.
The atomic ledger patch does not make migration side effects and ledger updates transactional. If a Job exits after applying changes but before recording m.<id>, reconciliation retries the pending ID and repeats those changes. Serial background Jobs do not remove this crash window. Require idempotent migrations, or add a recoverable per-ID claim/lease.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@design-proposals/platform-migrations/README.md` around lines 135 - 136,
Update the migration execution design to address crashes between applying
migration side effects and recording m.<id>: either make every migration
idempotent on retry or introduce a recoverable per-ID claim/lease that prevents
unsafe repetition. Document the selected behavior and ensure reconciliation can
safely resume pending migrations after a Job failure.
| # cozystack-migration: tier=pre-apply # pre-apply | background | ||
| # cozystack-migration: on-error=abort # abort | warn | ||
| # cozystack-migration: requires=20260801-etcd-crds | ||
| ``` | ||
|
|
||
| `on-error=warn` turns the "best-effort by design" paragraph into something the runner enforces. The script is then written plainly fail-fast, and the runner decides what a non-zero exit means — instead of `|| true` per command, which also swallows the failures the author wanted to see. |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift
Make on-error consistent with the tier and ledger state.
A pre-apply migration with on-error=warn can fail while the chart continues, which defeats the pre-apply guarantee. Under the stated set difference, a recorded non-ok result is also no longer pending, which conflicts with the retry behavior in Line 293 and the terminal-outcome check in Line 306. Enforce pre-apply => abort, and define whether background non-ok records are retried or terminal.
Also applies to: 168-170, 287-306
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@design-proposals/platform-migrations/README.md` around lines 148 - 153,
Update the migration configuration and runner logic around the
pre-apply/background tier handling and ledger outcome checks: enforce that
pre-apply migrations always use abort semantics, and make background non-ok
records consistently either retryable or terminal. Align the pending-selection
logic near the retry path and the terminal-outcome check so recorded non-ok
results follow that chosen behavior without contradicting the migration
contract.
| `on-error=warn` turns the "best-effort by design" paragraph into something the runner enforces. The script is then written plainly fail-fast, and the runner decides what a non-zero exit means — instead of `|| true` per command, which also swallows the failures the author wanted to see. | ||
|
|
||
| **A shared library, extended.** `migrations/lib/` already holds `cozystack-version.sh` and `seaweedfs-db-adopt.sh`, so the precedent exists. It grows helpers for operations that keep being re-implemented: a `kubectl` wrapper with retry on transient apiserver errors (what the `|| true` sites are really reaching for), a list helper that does not SIGPIPE under `pipefail` (migration `44` documents that trap in a comment), and the Helm-ownership adopt / `resource-policy: keep` pattern shared by `31`, `33`, `35`, `43`, `45` and `53` — by far the most repeated operation in the tree. | ||
|
|
||
| **A linter**, `hack/lint-migrations.sh`, wired into `make unit-tests`: filename matches the ID grammar; header present, parseable, and declaring `tier` and `on-error`; one shell dialect (`#!/bin/sh`, since the image is busybox — several migrations are currently `#!/bin/bash` for no stated reason); `shellcheck` clean; `requires` targets exist; no direct writes to the ledger. That last rule is where the architectural guard currently in `hack/cozystack-version-stamp.bats` moves to. |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
fd -i 'run-migrations.sh|lint-migrations.sh' |
while IFS= read -r file; do
rg -n -C 5 'set[[:space:]]+-e|sh[[:space:]]+-e|exec[[:space:]]+.*migration' "$file"
doneRepository: cozystack/community
Length of output: 157
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '%s\n' '--- migration-related files ---'
git ls-files | rg '(^|/)(run-migrations|lint-migrations|.*migration.*|migrations/|hack/)' | head -200
printf '%s\n' '--- README references to execution and on-error ---'
rg -n -C 4 'on-error|fail-fast|run-migrations|set -e|shell|runner|execute|non-zero|exit' design-proposals/platform-migrations/README.md
printf '%s\n' '--- migration script headers and error handling ---'
files="$(git ls-files | rg '(^|/)migrations/[^/]+\.sh$' || true)"
if [ -n "$files" ]; then
while IFS= read -r file; do
printf '\n### %s\n' "$file"
sed -n '1,24p' "$file"
done <<< "$files"
fi
printf '%s\n' '--- migration invocation sites ---'
rg -n -C 5 'migrations|run-migrations|on-error|sh -|bash ' --glob '*.sh' --glob '*.yaml' --glob '*.yml' --glob 'Makefile*' --glob '*.mk' . | head -300Repository: cozystack/community
Length of output: 21382
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '%s\n' '--- proposal execution requirements ---'
sed -n '137,173p' design-proposals/platform-migrations/README.md
sed -n '282,306p' design-proposals/platform-migrations/README.md
printf '%s\n' '--- referenced implementation files present in the tree ---'
for path in \
packages/core/platform/images/migrations \
packages/core/platform/templates/migration-hook.yaml \
hack/lint-migrations.sh \
hack/cozystack-version-stamp.bats
do
if git ls-files --error-unmatch "$path" >/dev/null 2>&1; then
printf 'present: %s\n' "$path"
else
printf 'absent: %s\n' "$path"
fi
done
printf '%s\n' '--- shell behavior without and with errexit ---'
sh -c 'false; printf "continued-after-failure\n"; exit 0'
printf 'plain status: %s\n' "$?"
sh -e -c 'false; printf "continued-after-failure\n"; exit 0' || printf 'errexit status: %s\n' "$?"
printf '%s\n' '--- explicit fail-fast wording in the proposal ---'
rg -n -i -C 2 'fail[- ]fast|errexit|set[[:space:]]+-e|sh[[:space:]]+-e|shellcheck|middle of|partial|non-zero' design-proposals/platform-migrations/README.mdRepository: cozystack/community
Length of output: 10137
Require an explicit fail-fast mechanism.
The proposal says migrations are “plainly fail-fast,” but it does not require set -e or invoke scripts with sh -e. Without either mechanism, a failed command can be followed by a successful command, and the runner can record ok. Define and test this requirement.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@design-proposals/platform-migrations/README.md` around lines 153 - 157,
Update the migration proposal and hack/lint-migrations.sh requirements to
mandate an explicit fail-fast mechanism: each migration must use set -e or the
runner must invoke it with sh -e. Extend the linter/tests to detect and enforce
this requirement so a failed command cannot be followed by a successful command
and recorded as ok.
| **A shared library, extended.** `migrations/lib/` already holds `cozystack-version.sh` and `seaweedfs-db-adopt.sh`, so the precedent exists. It grows helpers for operations that keep being re-implemented: a `kubectl` wrapper with retry on transient apiserver errors (what the `|| true` sites are really reaching for), a list helper that does not SIGPIPE under `pipefail` (migration `44` documents that trap in a comment), and the Helm-ownership adopt / `resource-policy: keep` pattern shared by `31`, `33`, `35`, `43`, `45` and `53` — by far the most repeated operation in the tree. | ||
|
|
||
| **A linter**, `hack/lint-migrations.sh`, wired into `make unit-tests`: filename matches the ID grammar; header present, parseable, and declaring `tier` and `on-error`; one shell dialect (`#!/bin/sh`, since the image is busybox — several migrations are currently `#!/bin/bash` for no stated reason); `shellcheck` clean; `requires` targets exist; no direct writes to the ledger. That last rule is where the architectural guard currently in `hack/cozystack-version-stamp.bats` moves to. |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | 🏗️ Heavy lift
Include shared helper code in the immutable execution identity.
migrations/lib/ is shared by multiple migration IDs, but the immutability guard and ledger checksum cover only the top-level migration file. Changing a helper can change migration behavior without changing the ID or recorded checksum. Version the helpers, or record an image or transitive-source digest and reject mismatches.
Also applies to: 215-215, 280-280
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@design-proposals/platform-migrations/README.md` around lines 155 - 157,
Update the migration immutability guard and ledger checksum logic to include the
transitive contents of migrations/lib/ alongside each top-level migration
script. Ensure changes to shared helpers alter the recorded execution identity
and cause mismatches to be rejected, or introduce an equivalent versioned
helper/image digest mechanism; apply the same behavior to the related migration
paths.
| **Pre-apply** stays the render-gated hook. The gate generalises from a scalar compare to a set difference: the chart already ships the scripts (there is no `.helmignore` in `packages/core/platform`), so `.Files.Glob "images/migrations/migrations/pre-apply/*"` yields the ID list at render with no content reads. The Job is only created when the difference is non-empty. | ||
|
|
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
Include revoked IDs in the work-list gate.
The gate computes IDs only from pre-apply/. If revocation moves an ID into revoked/, no Job starts and the proposal cannot write the promised m.<id>: ... revoked record. Keep a revoked marker in the tier input or trigger the runner for revoked − ledger. Apply the same rule to background IDs.
Also applies to: 203-204
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@design-proposals/platform-migrations/README.md` around lines 174 - 175,
Update the pre-apply work-list gate described in the migration proposal to
include revoked IDs, triggering the runner when revoked IDs are absent from the
ledger so it can write the promised m.<id>: ... revoked record. Apply the same
revoked-minus-ledger handling to background IDs, using either a revoked marker
in the tier input or an equivalent gate condition.
| **Pre-apply** stays the render-gated hook. The gate generalises from a scalar compare to a set difference: the chart already ships the scripts (there is no `.helmignore` in `packages/core/platform`), so `.Files.Glob "images/migrations/migrations/pre-apply/*"` yields the ID list at render with no content reads. The Job is only created when the difference is non-empty. | ||
|
|
||
| **Background** is orchestrated — not executed — by cozystack-operator, because the scripts live in the migrations image and the operator has neither them nor a Helm renderer. The chart therefore writes down the two things the operator cannot derive: which IDs are background, and which migrations image the current platform release pins. | ||
|
|
||
| ```yaml | ||
| apiVersion: v1 | ||
| kind: ConfigMap | ||
| metadata: | ||
| name: cozystack-migrations-index | ||
| namespace: cozy-system | ||
| data: | ||
| image: ghcr.io/cozystack/cozystack/platform-migrations:v1.7.0@sha256:… | ||
| background: | | ||
| 20260814-clickhouse-keeper-pvc-labels | ||
| 20260814-tenant-ancestor-labels | ||
| ``` | ||
|
|
||
| The operator reconciles `pending = background − ledger − revoked`, creates one Job per pending ID from `image` with `ONLY=<id>`, and patches the ledger on success. Because the ConfigMap is re-rendered on every platform upgrade, the image ref and the list cannot drift from the release that shipped them. The operator is already `cluster-admin`, so this needs no RBAC change. |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift
Make the packaged migration set a single verifiable contract.
The hook derives IDs from chart files, execution reads the migration image, and background execution reads the separate background list. A chart/image mismatch can create a Job without its script. An omitted background ID is never scheduled. Generate the index from the packaged files and add a CI check that compares the chart, image, and index ID sets.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@design-proposals/platform-migrations/README.md` around lines 174 - 191,
Update the migration index generation around the cozystack-migrations-index
ConfigMap so its IDs are derived from the packaged migration files rather than
maintained separately. Add CI validation comparing the chart’s packaged IDs,
migration image IDs, and generated index IDs, and fail on any mismatch to ensure
every scheduled migration has a corresponding script and background migration is
not omitted.
| **Background** is orchestrated — not executed — by cozystack-operator, because the scripts live in the migrations image and the operator has neither them nor a Helm renderer. The chart therefore writes down the two things the operator cannot derive: which IDs are background, and which migrations image the current platform release pins. | ||
|
|
||
| ```yaml | ||
| apiVersion: v1 | ||
| kind: ConfigMap | ||
| metadata: | ||
| name: cozystack-migrations-index | ||
| namespace: cozy-system | ||
| data: | ||
| image: ghcr.io/cozystack/cozystack/platform-migrations:v1.7.0@sha256:… | ||
| background: | | ||
| 20260814-clickhouse-keeper-pvc-labels | ||
| 20260814-tenant-ancestor-labels | ||
| ``` | ||
|
|
||
| The operator reconciles `pending = background − ledger − revoked`, creates one Job per pending ID from `image` with `ONLY=<id>`, and patches the ledger on success. Because the ConfigMap is re-rendered on every platform upgrade, the image ref and the list cannot drift from the release that shipped them. The operator is already `cluster-admin`, so this needs no RBAC change. | ||
|
|
||
| Background Jobs run **serially**, one at a time in ID order: it matches the current model, keeps failure attribution unambiguous, and avoids two migrations touching the same objects concurrently. |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | 🏗️ Heavy lift
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
rg -n -C 8 \
'cozystack-migrations-index|ONLY=|serviceAccountName|ClusterRoleBinding' \
packages/core/platform internal/operatorRepository: cozystack/community
Length of output: 286
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '%s\n' '--- candidate files ---'
git ls-files | sed -n '1,160p'
printf '%s\n' '--- migration-related paths ---'
git ls-files | rg -i '(^|/)(operator|platform|migration|chart|helm|workload|rbac|serviceaccount)' | sed -n '1,240p'
printf '%s\n' '--- relevant identifiers ---'
rg -n -C 5 \
'cozystack-migrations-index|ONLY=|serviceAccountName|ClusterRoleBinding|cluster-admin|background' \
. --glob '!design-proposals/platform-migrations/README.md' \
--glob '!vendor/**' \
--glob '!node_modules/**' \
| sed -n '1,320p'Repository: cozystack/community
Length of output: 16599
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '%s\n' '--- proposal context ---'
sed -n '1,260p' design-proposals/platform-migrations/README.md
printf '%s\n' '--- repository manifests and workflow files ---'
git ls-files | rg -i '\.(yaml|yml|json|tpl|go|ts|tsx|js|jsx)$' | sed -n '1,320p'
printf '%s\n' '--- service account and RBAC declarations ---'
rg -n -C 8 \
'kind:[[:space:]]*(ServiceAccount|ClusterRole|RoleBinding|ClusterRoleBinding)|serviceAccountName:|cluster-admin|rbac.authorization.k8s.io' \
. --glob '*.{yaml,yml,json,tpl,go,ts,tsx,js,jsx}' \
| sed -n '1,320p'Repository: cozystack/community
Length of output: 24853
🏁 Script executed:
#!/bin/bash
set -euo pipefail
python3 - <<'PY'
from pathlib import Path
p = Path("design-proposals/platform-migrations/README.md")
text = p.read_text()
for needle in ("cluster-admin", "ServiceAccount", "serviceAccountName", "Job", "operator"):
print(f"--- {needle} ---")
for i, line in enumerate(text.splitlines(), 1):
if needle.lower() in line.lower():
print(f"{i}:{line}")
PYRepository: cozystack/community
Length of output: 10960
🌐 Web query:
Kubernetes official documentation Job Pod serviceAccountName controller ServiceAccount namespace
💡 Result:
In Kubernetes, the serviceAccountName for a Job is defined within the pod template section of the Job specification [1][2]. The Job controller itself does not directly hold or manage the serviceAccountName; rather, it uses the pod template to create new Pods [3][4]. Key details regarding the relationship between Jobs, Pods, and ServiceAccounts include: 1. Pod Template Configuration: To specify a ServiceAccount for a Job, you must set the spec.template.spec.serviceAccountName field within the Job definition [1][5]. This configuration ensures that every Pod created by the Job controller is assigned the specified ServiceAccount [1]. 2. Admission Controller Role: When a Job controller creates a new Pod based on the template, the API server's ServiceAccount admission controller processes the Pod [6]. If the serviceAccountName is not explicitly provided in the Job's pod template, the admission controller automatically assigns the default ServiceAccount for the namespace where the Pod is created [1][6]. 3. ServiceAccount Controller: A separate ServiceAccount controller manages the existence of ServiceAccounts [6]. It ensures that a ServiceAccount named default exists in every namespace [6][7]. If a Pod (created by a Job or otherwise) references a non-existent ServiceAccount, the admission controller will reject the Pod creation [6]. 4. Namespace Scope: A ServiceAccount is namespaced [8]. A Pod created by a Job will use a ServiceAccount from the same namespace as the Pod itself [1][8]. While you can use RBAC to grant a ServiceAccount permissions to access resources in other namespaces, the ServiceAccount must exist within the namespace where the Pod is running [8]. You can update the serviceAccountName of a Job's template using the kubectl set serviceaccount command, which modifies the pod template of the specified resource [9]. Note that this change affects only future Pods created by the Job controller; existing Pods managed by the Job cannot have their serviceAccountName updated [1].
Citations:
- 1: https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/
- 2: https://kubernetes.io/docs/reference/kubernetes-api/batch/job-v1/
- 3: https://kubernetes.io/docs/concepts/workloads/controllers/job/
- 4: https://github.com/kubernetes/kubernetes/blob/master/pkg/controller/job/job_controller.go
- 5: https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/job
- 6: https://kubernetes.io/docs/reference/access-authn-authz/service-accounts-admin/
- 7: https://github.com/kubernetes/kubernetes/blob/master/pkg/controller/serviceaccount/serviceaccounts_controller.go
- 8: https://kubernetes.io/docs/concepts/security/service-accounts/
- 9: https://kubernetes.io/docs/reference/kubectl/generated/kubectl_set/kubectl_set_serviceaccount/
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '%s\n' '--- exact design claims around identity and background Jobs ---'
sed -n '168,202p' design-proposals/platform-migrations/README.md
sed -n '268,282p' design-proposals/platform-migrations/README.md
printf '%s\n' '--- all ServiceAccount references in the proposal ---'
rg -n -C 4 'ServiceAccount|ClusterRoleBinding|serviceAccountName|operator.*identity|identity' \
design-proposals/platform-migrations/README.mdRepository: cozystack/community
Length of output: 7326
🏁 Script executed:
#!/bin/bash
set -euo pipefail
python3 - <<'PY'
from pathlib import Path
p = Path("design-proposals/platform-migrations/README.md")
lines = p.read_text().splitlines()
for start, end in ((176, 193), (270, 278)):
print(f"--- lines {start}-{end} ---")
for n in range(start, end + 1):
print(f"{n}:{lines[n-1]}")
PYRepository: cozystack/community
Length of output: 2777
Specify the ServiceAccount for background Jobs.
Job Pods do not inherit the operator's cluster-admin identity. Without spec.template.spec.serviceAccountName, they use the namespace default ServiceAccount and may fail with Forbidden. Define a cozy-system ServiceAccount, its binding, lifecycle, and a rendered identity test.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@design-proposals/platform-migrations/README.md` around lines 176 - 193,
Update the background Job specification described around the migration index so
each Job Pod explicitly uses a dedicated cozy-system ServiceAccount rather than
the namespace default. Define the ServiceAccount and its cluster-admin binding,
document their lifecycle with the migration resources, and add a rendered
identity test verifying spec.template.spec.serviceAccountName.
| 1. Rename `migrations/1..53` to IDs derived from the release each shipped in, preserving order — `20250409-01-mariadb-operator-secrets` through `20260724-53-seaweedfs-db-adopt-repair`. The numeric suffix in the slug keeps the original sequence readable and guarantees the sort matches the old order exactly. | ||
| 2. Commit the integer → ID mapping as `migrations/legacy-map`, one `N <id>` pair per line. | ||
| 3. On first run against a cluster that has a `version` scalar but no `m.*` keys, the runner seeds the ledger from the map: every integer below `version` gets its `m.<id>` recorded with outcome `legacy` (no checksum — those files were not immutable when they ran). Idempotent, since the seeding condition is the absence of `m.*` keys. | ||
| 4. Delete the legacy pass, `targetVersion`, and `hack/check-migrations-target.sh`. | ||
|
|
||
| Only step 3 touches clusters, and it writes ledger keys rather than running anything. A cluster stamped `54` ends up with fifty-three `legacy` records and behaves identically. |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
Seed missing legacy records, not only an empty ledger.
A cluster can have version: "54" and one m.* key from Phase 1 or Phase 2. The current condition then skips seeding all legacy IDs. After the rename and legacy-pass removal, the 53 renamed migrations appear pending and can run again. Seed each absent m.<id> whose legacy number is below version.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@design-proposals/platform-migrations/README.md` around lines 237 - 242, The
migration seeding logic must populate every missing legacy record below the
cluster’s version, rather than requiring the ledger to have no m.* keys. Update
the step 3 runner behavior to iterate the legacy-map entries below version and
create only absent m.<id> records with outcome legacy, preserving idempotency
and existing records.
1 participant
This PR adds design proposal for platform migrations engine.
Current implementation uses one dense integer as both migration id and cluster state. Every new migration takes next number, so concurrent PRs fight for same slot - #3406, #3379 and #3315 all add
migrations/54right now and two of them will renumber on merge. Scalar high water mark cant describe branched history, so migrations are not backportable, #3534 is realised case of that. Pending set is also computed fromtargetVersionliving in another file than migrations, that is whyhack/check-migrations-target.shexists at all.Proposal replaces integer with
YYYYMMDD-slugid and applied set ledger in the same configmap, splits execution to blocking pre-apply tier and background tier run by cozystack-operator, and adds contract for writing migrations: declared metadata header, one shell dialect, linter and mandatory tests.Please look at open questions at the end, especially per package ledger vs single one when packages start splitting out.
Summary by CodeRabbit