diff --git a/docs/PROVENANCE.md b/docs/PROVENANCE.md index 908e108..4305654 100644 --- a/docs/PROVENANCE.md +++ b/docs/PROVENANCE.md @@ -42,7 +42,7 @@ Excluded, and why: both resolve exactly one upstream clone. Its provenance travels with the files instead, in that directory's own `NOTICE` — source repository, commit, file-by-file origin and licence — which is what a reader who finds the - directory will actually look at. `docs/public/logo.png` comes from the same + directory will actually look at. `common/docs/public/logo.png` comes from the same repository and the same `NOTICE` covers it. - **`docs/decisions/`** (this repository's own, at the root — not `common/docs/decisions/`, which ships and is covered below). This is this @@ -62,12 +62,12 @@ Excluded, and why: | `common/.editorconfig` | `.editorconfig` | verbatim | | | `common/renovate.json` | `renovate.json` | original | was `verbatim` (byte-identical) until this task. Replaced: the upstream file extends `local>immich-app/.github:renovate-config` (unresolvable outside that organisation), matches `machine-learning/**`, `mobile/**`, `ghcr.io/immich-app/*`, none of which exist in a generated project, and has a trailing comma that makes it invalid strict JSON — every generated project has shipped a Renovate config that fails on its first parse since Task 3. Replaced with a minimal, generic config (`$schema` plus the public `config:recommended` preset, no per-project rules). | | `common/.gitattributes` | `.gitattributes` | original | instructed as a byte-for-byte copy originally, then reversed: upstream's version is almost entirely `mobile/**` and `packages/sdk/**` linguist/generated rules that have no counterpart in a generated project. Kept as one line, `* text=auto eol=lf`. | -| `common/compose.yaml` | `docker/docker-compose.yml` | adapted | one application service instead of server/machine-learning/redis/postgres; digest-pinned postgres image kept; every variable defaults so the file validates before `.env` exists (ADR-0014). | -| `common/compose.dev.yaml` | `docker/docker-compose.dev.yml` (`database` service) | adapted | the rest of upstream's dev compose mounts the whole immich monorepo into containers, which has no counterpart here; only its `database` service (port `5432:5432` exposed, postgres env vars) is the real parallel, simplified to a pinned-digest image with hardcoded local-only credentials. | +| `common/compose.yaml` | `docker/docker-compose.yml` | adapted | one service per application plus the selected database and cache instead of server/machine-learning/redis/postgres, assembled by `scaffold` (ADR-0022, ADR-0019); each database or cache image is pinned by digest in `services//service.env`, and each application image is tagged `${IMAGE_TAG:-latest}`; every variable defaults so the file validates before `.env` exists (ADR-0014). | +| `common/compose.dev.yaml` | `docker/docker-compose.dev.yml` (`database` service) | adapted | the rest of upstream's dev compose mounts the whole immich monorepo into containers, which has no counterpart here; only its `database` service (a port exposed to the host) is the real parallel. The template ships `services: {}`; `scaffold` merges in each selected service's `compose.dev.fragment.yaml`, with ports bound to `127.0.0.1` and digest-pinned images. | | `common/compose.test.yaml` | — | original | no `docker-compose.test.yml` exists upstream. Same shape as `compose.dev.yaml` (its sibling in this repo, not an upstream file) with `tmpfs` storage so CI starts from an empty database every run. | | `common/example.env` | `docker/example.env` | adapted | one image's variables instead of immich's per-service set; same "copy to `.env` and edit" framing and a placeholder `DB_PASSWORD` an operator must change (ADR-0014). | | `common/install.sh` | `install.sh` | adapted | one image instead of several; never overwrites an existing `.env`; generates the database password from `/dev/urandom` instead of asking the operator to supply one. Rationale recorded inline in the file. | -| `common/.github/workflows/build.yml`, `ci.yml`, `docs.yml`, `release.yml`, `security.yml` | `.github/workflows/test.yml`, `docker.yml`, `docs-build.yml`, `codeql-analysis.yml`, `static_analysis.yml` | adapted | immich's per-repository workflows collapsed into five thin call sites that each delegate to `you/.github` (ADR-0005); `GITHUB_TOKEN` instead of a minted GitHub App token (ADR-0010); one docs workflow instead of three (ADR-0009); build and release split so an ordinary merge never waits on the standing release PR (ADR-0015). | +| `common/.github/workflows/build.yml`, `ci.yml`, `docs.yml`, `release.yml`, `security.yml` | `.github/workflows/test.yml`, `docker.yml`, `docs-build.yml`, `codeql-analysis.yml`, `static_analysis.yml` | adapted | immich's per-repository workflows collapsed into five thin call sites that each delegate to `you/.github` (ADR-0005); `GITHUB_TOKEN` instead of a minted GitHub App token, except Release Please's optional app token (ADR-0010); one docs workflow instead of three (ADR-0009); build and release split so an ordinary merge never waits on the standing release PR (ADR-0015). | | `common/docs/` | `docs/` | adapted | VitePress instead of Docusaurus; one docs workflow instead of three (ADR-0009). `common/docs/scripts/check-adrs.mjs` and `check-paths.mjs` have no upstream equivalent — immich has no ADR process — and are original, not adapted. | | `common/lefthook.yml` | — | original | immich runs no git hooks (ADR-0007). | | `common/.gitignore`, `common/CODEOWNERS`, `common/CONTRIBUTING.md`, `common/README.md`, `common/SECURITY.md`, `common/commitlint.config.js`, `common/.git-blame-ignore-revs`, `common/release-please-config.json`, `common/.release-please-manifest.json`, `common/mise.root.toml`, `common/pnpm-workspace.yaml`, `common/AGENTS.md`, `common/packages-types/` | (various — e.g. `.gitignore`, `CODEOWNERS`, `CONTRIBUTING.md`, `README.md`, `pnpm-workspace.yaml`) | original | conventional files any GitHub/pnpm project carries. Checked against immich's own copies of each — `diff` shows no shared content beyond the two both being, say, a `.gitignore` — so these were written for this project, not adapted from immich's. | diff --git a/docs/README.md b/docs/README.md index 960961f..5592943 100644 --- a/docs/README.md +++ b/docs/README.md @@ -97,6 +97,8 @@ Measured on `scaffold new demo --api nestjs --web nextjs --db postgres`: 101 tra | Day one | This page, then [01-toolchain](tour/01-toolchain.md) through [03-ci](tour/03-ci.md) | | First week | [04-guardrails](tour/04-guardrails.md) through [09-wizard](tour/09-wizard.md); ADR-0001, ADR-0003, ADR-0011 | | On demand | The [runbook](runbook/) that names the situation | +| Before a project's first push | [publish-a-project](runbook/publish-a-project.md) | +| When the toolbox changed under an existing project | [update-a-project](runbook/update-a-project.md) | ## Diagrams diff --git a/docs/decisions/0003-adapter-overlay-instead-of-vendored-presets.md b/docs/decisions/0003-adapter-overlay-instead-of-vendored-presets.md index c4e17fe..a790909 100644 --- a/docs/decisions/0003-adapter-overlay-instead-of-vendored-presets.md +++ b/docs/decisions/0003-adapter-overlay-instead-of-vendored-presets.md @@ -12,6 +12,11 @@ application per stack. Invoke each framework's own generator and overlay four files. +> **Amended 2026-09-17.** `copy_adapter_files` in `lib/adapter.sh` overlays +> every file in the adapter directory except `adapter.env` and +> `lefthook.fragment.yml` (`ADAPTER_INTERNAL_FILES`). Four is the minimum +> `scaffold lint` requires (`REQUIRED_ADAPTER_FILES` in `lib/contract.sh`). + **Amendment, 2026-09-06 (0021).** The boundary was "an adapter overlays configuration, it never writes application code." It narrows to: no application code except the health routes 0021's deploy gate requires — one diff --git a/docs/decisions/0005-share-ci-through-reusable-workflows.md b/docs/decisions/0005-share-ci-through-reusable-workflows.md index a4026a4..19e0b69 100644 --- a/docs/decisions/0005-share-ci-through-reusable-workflows.md +++ b/docs/decisions/0005-share-ci-through-reusable-workflows.md @@ -20,6 +20,10 @@ one per entrypoint — that only name the reusable workflow and pass its inputs. Projects reference `you/.github` by the moving tag `v1`, not a commit or a fixed minor version. +> **Amended 2026-09-17.** `you/.github` now has six `workflow_call` files. +> The sixth, `app-targets.yml`, resolves the image matrix for `app-build.yml` +> and `app-release.yml` (ADR-0022). No call site in `common/` names it. + ## Consequences - A fix to CI reaches every project that calls `v1` the next time it runs, diff --git a/docs/decisions/0010-github-token-over-a-github-app.md b/docs/decisions/0010-github-token-over-a-github-app.md index 016058f..27164e4 100644 --- a/docs/decisions/0010-github-token-over-a-github-app.md +++ b/docs/decisions/0010-github-token-over-a-github-app.md @@ -2,6 +2,7 @@ Status: Accepted Date: 2026-08-28 +Superseded in part by ADR-0024 (the Release Please token) ## Context @@ -23,6 +24,14 @@ Use the built-in `GITHUB_TOKEN` everywhere, scoped down per job with `permissions:` (every workflow starts `permissions: {}` and grants only what that job needs — `contents: read`, `packages: write`, and so on). +> **Amended 2026-09-17.** One exception exists. `app-release.yml`'s +> `release-please` job mints a GitHub App token when the `RELEASE_APP_ID` and +> `RELEASE_APP_PRIVATE_KEY` secrets are set, and falls back to `GITHUB_TOKEN` +> when they are not. `common/.github/workflows/release.yml` passes both, and +> `scaffold publish` sets them from the environment (ADR-0024). A pull request +> opened with `GITHUB_TOKEN` starts no workflow, so its checks wait for +> approval. Every other job still uses `GITHUB_TOKEN`. + ## Consequences - No secret to provision, rotate, or document per project — `GITHUB_TOKEN` diff --git a/docs/decisions/0012-tiered-adapter-support.md b/docs/decisions/0012-tiered-adapter-support.md index 02ac848..53c3a99 100644 --- a/docs/decisions/0012-tiered-adapter-support.md +++ b/docs/decisions/0012-tiered-adapter-support.md @@ -26,6 +26,10 @@ its own directory changes plus weekly — 25 minutes is far past what anyone will wait on before pushing. Tier C has no automated verification and is allowed to rot. +> **Amended 2026-09-17.** `flask` joined Tier A (`adapters/flask/adapter.env`). +> Tier B also runs on a manual `workflow_dispatch` of +> `.github/workflows/adapters.yml` (`scripts/adapter-matrix.sh`). + Tier membership lives in one place: each adapter's own `ADAPTER_TIER` (`adapter.env`). Workflows read it (`scripts/adapter-matrix.sh`, backed by `scaffold list`) rather than repeating the adapter list in YAML, so the tier diff --git a/docs/decisions/0014-deployment-deferred-with-seams.md b/docs/decisions/0014-deployment-deferred-with-seams.md index 79759a2..ef5f55f 100644 --- a/docs/decisions/0014-deployment-deferred-with-seams.md +++ b/docs/decisions/0014-deployment-deferred-with-seams.md @@ -2,7 +2,7 @@ Status: Accepted Date: 2026-08-27 -Superseded in part by 0021 (seams 1 and 4) +Superseded in part by 0021 (seams 1 and 4) and 0022 (seam 1's single `app` service) ## Context @@ -53,6 +53,13 @@ deploy target plugs into later without restructuring anything above it: reusable release workflow will carry a `deploy` job that does nothing until a client sets that variable. +> **Amended 2026-09-17.** Seams 1 and 4 name one `app` service and a +> postgres `database`. Since ADR-0022, `compose.yaml` runs one service per +> application, each on `ghcr.io//-`. Since ADR-0019, the +> `database` service and its healthcheck come from the selected +> service's `compose.fragment.yaml` (`pg_isready` for postgres), and only an +> `api` or `app` service waits on it. + **Update, 2026-09-06 (0021).** Seams 1 and 4 as written above are now false, and the record stays rather than being rewritten: seam 4's premise was that php-fpm speaks FastCGI and no HTTP check is possible, so no check @@ -83,6 +90,9 @@ adapter still fills in a body rather than restructuring anything, and by hand, on the target host, after cloning nothing more than the two files a release publishes (`compose.yaml`, `example.env`). +> **Amended 2026-09-17.** A release publishes three files: `app-release.yml`'s +> `assets` job uploads `compose.yaml`, `example.env` and `install.sh`. + **Update, 2026-09-07.** "The one deploy mechanism that exists today" held only for a public project. Measured against a real private repository, a release asset's browser download URL 404s even with a Bearer token — the diff --git a/docs/decisions/0015-continuous-builds-separate-from-cut-releases.md b/docs/decisions/0015-continuous-builds-separate-from-cut-releases.md index 20afa3f..a8daef0 100644 --- a/docs/decisions/0015-continuous-builds-separate-from-cut-releases.md +++ b/docs/decisions/0015-continuous-builds-separate-from-cut-releases.md @@ -22,6 +22,11 @@ Split publishing into two reusable workflows that never gate each other: 'true'` — i.e. only on the merge that closes a release PR — and publish semver tags (`1.4.0`, `1.4`) plus `latest`. +> **Amended 2026-09-17.** `app-build.yml`'s `build` job and `app-release.yml`'s +> `image` job also require `needs.targets.outputs.matrix != '[]'`: a project +> with no application builds nothing. The release `image` job also publishes +> `sha-`. Each job builds one image per application (ADR-0022). + Both call sites (`common/.github/workflows/build.yml` and `release.yml`) are separate files with separate triggers, not two jobs in one workflow, so one failing does not block the other from being invoked at all. diff --git a/docs/decisions/0017-supply-chain-defaults-in-generated-projects.md b/docs/decisions/0017-supply-chain-defaults-in-generated-projects.md index 57613ac..1799f33 100644 --- a/docs/decisions/0017-supply-chain-defaults-in-generated-projects.md +++ b/docs/decisions/0017-supply-chain-defaults-in-generated-projects.md @@ -76,6 +76,12 @@ whether it clears the immediate failure: `allowBuilds: { esbuild: true }` entry for the identical reason, kept in sync by hand since the two files serve different projects and are never both present at once. + + > **Amended 2026-09-17.** `common/pnpm-workspace.yaml` is no longer deleted. + > For a project that is not all-typescript, `keep_apps_standalone` in + > `lib/pnpm.sh` removes only its `packages:` key and keeps `allowBuilds`. + > `join_typescript_workspace` deletes the `docs` pair instead. + - **`confirmModulesPurge` does not ship in `common/pnpm-workspace.yaml`.** It is set instead as `env = { npm_config_confirm_modules_purge = "false" }` on the `install` and `ci-unit` mise tasks, in every adapter's @@ -90,6 +96,11 @@ whether it clears the immediate failure: ambient environment variables to the tool it launches — a fact this decision had to design around, not rely on, for `record_release_age_exceptions` below). + + > **Amended 2026-09-17.** Only the pnpm-based files set it: + > `adapters/nestjs/mise.toml`, `adapters/nextjs/mise.toml`, + > `common/packages-types/mise.toml` and `common/docs/mise.toml`. + - **`minimumReleaseAge: 0` does not ship anywhere, ever.** Lowering it permanently, silently, for every client project this toolbox will ever generate is exactly the outcome this decision exists to prevent — a @@ -179,6 +190,13 @@ whether it clears the immediate failure: relaxation ships to the client on every *successful* `scaffold add`; without `cmd_add_cleanup`'s strip, it survives every *failed* one. + > **Amended 2026-09-17.** `relax_pnpm_workspace` in `lib/pnpm.sh` appends + > three lines, `PNPM_RELAXATIONS`: `confirmModulesPurge: false`, + > `frozenLockfile: false` and `minimumReleaseAge: 0`. Both sites strip them + > with `restore_pnpm_workspace`: on success from `settle_added_app`, before + > `trap - EXIT`; on failure from `cmd_add_cleanup`. The known gap below + > applies to all three lines. + **Known gap, not fixed as part of this decision**: both removal sites match the line blindly (`sed -i '/^confirmModulesPurge: false$/d'`), with no way to tell "the line @@ -205,6 +223,11 @@ whether it clears the immediate failure: minimum-release-age recording the first round of apps got from `scaffold new`. + > **Amended 2026-09-17.** Only an app joining a shared workspace + > (`joins_shared_workspace`) gets `sync_workspace_lockfile`. A standalone + > app gets `record_release_age_exceptions` alone + > (`reconcile_standalone_app`). + ## Alternatives considered - **Ship `minimumReleaseAge: 0` in `common/pnpm-workspace.yaml`.** Rejected diff --git a/docs/decisions/0018-add-does-not-recompute-the-typescript-workspace.md b/docs/decisions/0018-add-does-not-recompute-the-typescript-workspace.md index 5a3a1b7..e273188 100644 --- a/docs/decisions/0018-add-does-not-recompute-the-typescript-workspace.md +++ b/docs/decisions/0018-add-does-not-recompute-the-typescript-workspace.md @@ -13,6 +13,11 @@ every requested adapter's `ADAPTER_LANGUAGE` is `typescript`, (`rm -rf "${target}/packages-types" "${target}/pnpm-workspace.yaml"`) — see `scaffold`'s `cmd_new`. +> **Amended 2026-09-17.** The decision now lives in `settle_workspace_shape` +> in `scaffold`. A mixed project loses `packages-types`, but keeps +> `pnpm-workspace.yaml` without its `packages:` key (`keep_apps_standalone` +> in `lib/pnpm.sh`). + `scaffold add` installs one more adapter into a project that already made that decision. Its choice of adapter can contradict the inputs the decision was originally made from in either direction: @@ -87,6 +92,11 @@ retroactively when an added adapter happens to be typescript. the next time someone runs `pnpm install`, with no extra step from `scaffold add`. + > **Amended 2026-09-17.** `scaffold add` does run the install. A typescript + > app joining a shared workspace gets `reconcile_shared_workspace` + > (`sync_workspace_lockfile`, then `record_release_age_exceptions`), and the + > file gains and loses `PNPM_RELAXATIONS` during the call (ADR-0017). + ## Alternatives considered - **Recompute and remove `packages/types` when the project becomes diff --git a/docs/decisions/0023-a-project-records-what-generated-it.md b/docs/decisions/0023-a-project-records-what-generated-it.md index cc5fbe5..b35a4e3 100644 --- a/docs/decisions/0023-a-project-records-what-generated-it.md +++ b/docs/decisions/0023-a-project-records-what-generated-it.md @@ -68,6 +68,11 @@ comment change. After the patch, `sync_ci_roots` runs and the build targets are rebuilt from the manifest — only when the array actually came back empty, so a project whose targets survived is untouched. +> **Amended 2026-09-17.** `common/.github/workflows/ci.yml` carries +> `roots: '["docs"]'`, not `roots: '[]'`, so a verbatim patch checks only +> `docs`. `resync_derived_files` in `lib/update.sh` runs `sync_ci_roots` on +> every update and rebuilds the image targets only when `images` is `"[]"`. + **It refuses a dirty working tree**, because `git diff` afterwards is the only review this gets and it has to show one run's changes alone. It never commits. diff --git a/docs/decisions/0024-publishing-a-project-is-part-of-generating-it.md b/docs/decisions/0024-publishing-a-project-is-part-of-generating-it.md index 832d296..4a204fe 100644 --- a/docs/decisions/0024-publishing-a-project-is-part-of-generating-it.md +++ b/docs/decisions/0024-publishing-a-project-is-part-of-generating-it.md @@ -32,6 +32,10 @@ guardrails that is a repository setting rather than a file, and the one most likely to be skipped because nothing in the project records whether it was applied. +> **Amended 2026-09-17.** ADR-0004 lists no guardrails. The four layers — git +> hooks, CI, Renovate and the ruleset — are listed in +> `docs/tour/04-guardrails.md`. + A runbook step that is required, has no in-repo trace, and fails somewhere else is a step that should not be a runbook step. diff --git a/docs/runbook/add-an-adapter.md b/docs/runbook/add-an-adapter.md index a844a0a..f830b1e 100644 --- a/docs/runbook/add-an-adapter.md +++ b/docs/runbook/add-an-adapter.md @@ -1,116 +1,88 @@ # Add an adapter -Time: about one session. Tier B when the smoke test lands; Tier A when a -real project depends on it. +When: a framework needs to be generated that `scaffold list --adapters` does not show. -## 1. Create the directory +## Steps -```bash -mkdir -p adapters/ -``` - -## 2. Write `adapter.env` - -```bash -ADAPTER_NAME="" -ADAPTER_ROLE="api" # web, api, or app -ADAPTER_TIER="B" -ADAPTER_LANGUAGE="go" # "typescript" opts into packages/types -ADAPTER_GENERATOR='' - # no generator? the package manager's project - # init: adapters/flask uses `uv init --bare` -# ADAPTER_POST_GENERATE='' -``` +1. Create the directory. -`ADAPTER_POST_GENERATE` is optional — most adapters omit it. It's a -one-time shell command run right after the generator, for fixups the -generator itself gets wrong: `adapters/nestjs/adapter.env` sets it to -un-await `bootstrap()` and run prettier once. Only reach for it once -you've hit a real generator bug — it's a patch, not a default step. + ```bash + mkdir -p adapters/ + ``` -The exception is a stack whose generator writes only a manifest: there the -dependencies are ordinary setup rather than a fixup, and `adapters/flask` -uses this field to `uv add` them. Either way it ends in a `grep` that fails -loudly, so a generator that reports success while writing nothing becomes a -build failure rather than an `ImportError` at container start. +2. Write `adapters//adapter.env`. `adapters/flask/adapter.env` is a complete example. -## 3. Write `mise.toml` + ```bash + ADAPTER_NAME="" + ADAPTER_ROLE="api" # web, api or app + ADAPTER_TIER="B" # A, B or C (ADR-0012) + ADAPTER_LANGUAGE="go" # "typescript" opts into the shared pnpm workspace + ADAPTER_FAMILY="" # which service driver wires it (ADR-0019) + ADAPTER_GENERATOR='' + ADAPTER_LIVENESS_PATH="/health/live" + ADAPTER_READINESS_PATH="/health/ready" # required for api and app, omitted for web + # ADAPTER_POST_GENERATE='' + ``` -All nine contract tasks. Declare the language in a local `[tools]` block so -it never reaches the project root — or, where the language's own tooling -owns the pin, declare the tool and let it read that pin (`adapters/flask` -pins `uv` and leaves python to `.python-version`). `format`, `lint`, and -`check` must not write. + | Field | Rule | + | --- | --- | + | Required | `ADAPTER_NAME`, `ADAPTER_ROLE`, `ADAPTER_FAMILY`, `ADAPTER_GENERATOR`, `ADAPTER_LIVENESS_PATH` (`REQUIRED_ADAPTER_VARS` in `lib/contract.sh`) | + | No generator | Use the package manager's init: `adapters/flask` runs `uv init --bare` | + | `ADAPTER_POST_GENERATE` | Optional. For a generator bug (`adapters/nestjs`) or a manifest-only generator (`adapters/flask` runs `uv add`). End it with a `grep` that fails when nothing was written | -## 4. Write `Dockerfile`, `.env.example`, `lefthook.fragment.yml` +3. Write `adapters//mise.toml` with all nine contract tasks (`CONTRACT_TASKS` in `lib/contract.sh`). + - Pin the language in a local `[tools]` block, never at the project root. Where the language's own tool owns the pin, pin that tool: `adapters/flask` pins `uv`, and `adapters/flask/.python-version` pins python. + - `format`, `lint` and `check` must not write; `scaffold lint` rejects `--write`, `--fix` and similar flags in them. -The Dockerfile is multi-stage. Add a `HEALTHCHECK` only if you can make it -fail on a real broken state — a check that always passes is worse than no -check (see docs/tour/07). It must never copy a `.env` file. An adapter with -no extra git hook ships `{}` as its fragment. +4. Write the overlay files. Every file in the directory is copied into the app except `adapter.env` and `lefthook.fragment.yml` (ADR-0003). -## 5. Verify - -```bash -./scaffold lint -./scaffold new /tmp/probe --api -cd /tmp/probe && mise run "//apps/api:checklist" -``` + | File | Rule | + | --- | --- | + | `Dockerfile` | Multi-stage, base images pinned by digest, `EXPOSE 8080`, a `HEALTHCHECK` that probes `ADAPTER_LIVENESS_PATH`, never copies a `.env` (`tests/compose.bats`) | + | `Dockerfile.workspace` | TypeScript only: the variant built from the workspace root | + | `.dockerignore` | Required by `tests/compose.bats` | + | `.env.example` | The app's own variables | + | `lefthook.fragment.yml` | Hooks merged into the project's `lefthook.yml`; `{}` for none | -(the project root's own `mise run checklist` also runs this once -`register_config_root` picks it up — see `lib/manifest.sh` — but the -`//apps/api:` prefix runs only the new app, without waiting on every other -config root along with it.) +5. For an `api` or `app` role, add `services//drivers/.sh` to every service, unless the family already has drivers. -If `ADAPTER_LANGUAGE="typescript"`, that alone doesn't prove the thing -that has actually broken before: a *second* typescript app sharing the -same workspace. That happens on two different code paths, and only one -still carries a known, open gap. +6. Copy `tests/new-laravel-api.bats` to `tests/new-.bats` and change the adapter name. Keep the assertion that the language never reaches the project's root `mise.toml`. -**Via `scaffold new`** (both requested together — no known gap): +## Verify ```bash -./scaffold new /tmp/probe2 --api --web nextjs +./scaffold lint +./scaffold new ../probe --api +(cd ../probe && mise run //apps/api:checklist) +bats tests/new-.bats +./scaffold list --adapters ``` -**Via `scaffold add`** (joining a workspace that's already installed — -`cmd_add` relaxes `confirmModulesPurge` in `pnpm-workspace.yaml` then -strips the line back out by blind text match; if the caller had already -added that exact line themselves, on purpose, this silently deletes it -too — known, not fixed, see `restore_pnpm_workspace` in `lib/pnpm.sh` -and ADR-0017's Consequences): - -```bash -./scaffold new /tmp/probe3 --api nestjs -cd /tmp/probe3 && scaffold add apps/ --adapter -``` +- `scaffold lint` prints nothing and exits 0. +- `scaffold list --adapters` shows the adapter with its tier. -Either way, confirm the workspace itself, not just a green checklist — -this is the part that broke: +For a TypeScript adapter, also check a second TypeScript app in the same workspace, both ways: ```bash -find /tmp/probe3 -name pnpm-lock.yaml -not -path '*/node_modules/*' -test -f /tmp/probe3/packages/types/package.json && echo "types package: ok" -grep confirmModulesPurge /tmp/probe3/pnpm-workspace.yaml # expect no match +./scaffold new ../probe2 --api --web nextjs +./scaffold new ../probe3 --api nestjs +toolbox="$PWD" +(cd ../probe3 && "$toolbox/scaffold" add apps/ --adapter ) +find ../probe3 -name pnpm-lock.yaml -not -path '*/node_modules/*' # exactly one, at the root +test -f ../probe3/packages/types/package.json && echo "types package: ok" +grep confirmModulesPurge ../probe3/pnpm-workspace.yaml # no match ``` -The first command must list exactly one lockfile, at the project root. - -## 6. Add the smoke test - -Copy `tests/new-laravel-api.bats`, change the adapter name, and keep the -assertion that the language never appears in the project's root -`mise.toml`. +Delete the probe projects afterwards. -## 7. Promote to Tier A +## Promote to Tier A -Change `ADAPTER_TIER` to `A` in `adapter.env`. Nothing in -`.github/workflows/adapters.yml` names an adapter directly — it reads tiers -from `scripts/adapter-matrix.sh`, which reads `ADAPTER_TIER` for every -adapter via `scaffold list` — so this one-line edit is the whole promotion -(ADR-0012). +Set `ADAPTER_TIER="A"` in `adapter.env`. `scripts/adapter-matrix.sh` reads tiers through `scaffold list`, so `.github/workflows/adapters.yml` needs no edit (ADR-0012). -## Done when +## If it fails -`scaffold lint` is silent, `bats tests/new-.bats` passes, and -`scaffold list` shows the adapter with the intended tier. +| Symptom | Fix | +| --- | --- | +| `scaffold lint` names a missing driver | Step 5: add `drivers/.sh` to that service | +| `scaffold lint` says a read-only task writes | Remove the writing flag from `format`, `lint` or `check` | +| A caller's own `confirmModulesPurge: false` line vanished after `scaffold add` | Known gap: `restore_pnpm_workspace` in `lib/pnpm.sh` strips it by text match (ADR-0017). Add it back | diff --git a/docs/runbook/bump-a-toolchain-version.md b/docs/runbook/bump-a-toolchain-version.md index 79aee82..85a330c 100644 --- a/docs/runbook/bump-a-toolchain-version.md +++ b/docs/runbook/bump-a-toolchain-version.md @@ -1,58 +1,51 @@ # Bump a toolchain version -## 1. Find where the version is pinned +When: a pinned tool needs a new version, in the toolbox or in what it generates. -- This toolbox's own tools (`bats`, `shellcheck`, `yq`, `jq`): `mise.toml` - at the repository root. -- What ships into every generated project (`node`, `pnpm`, `lefthook`, - `gitleaks`): `common/mise.root.toml`. -- One adapter's own language (`php`'s Composer, or the pinned Node used by - `laravel-inertia`'s build step): that adapter's own `mise.toml`, e.g. - `adapters/laravel-api/mise.toml`. -- `flask`'s python is the exception, and searching the `[tools]` blocks for - it finds nothing: `adapters/flask/mise.toml` pins `uv` only, and the - interpreter is pinned in `adapters/flask/.python-version`, the file uv - reads. Bumping it means editing that file and the `--python` argument in - `adapters/flask/adapter.env`, which sets `requires-python` at generation. +## Steps -## 2. Edit the version +1. Find the pin. -Change the version string for the tool in question. Leave everything else -in the `[tools]` block alone. + | Tool | Pinned in | + | --- | --- | + | This toolbox's tools: `bats`, `shellcheck`, `shfmt`, `yq`, `jq`, `zizmor`, `rush`, `lefthook`, `gitleaks` | `mise.toml` | + | Every generated project: `node`, `pnpm`, `lefthook`, `gitleaks` | `common/mise.root.toml` | + | One adapter's toolchain, e.g. composer for `laravel-api`, node for `laravel-inertia` | `adapters//mise.toml` | + | `flask`'s python | `adapters/flask/.python-version` and the `--python` argument in `adapters/flask/adapter.env` | + | php | Not pinned: system php, checked by each Laravel adapter's `install` task (ADR-0016) | -## 3. Re-resolve the lock +2. Edit the version string. Leave the rest of the `[tools]` block alone. -```bash -mise install -``` +3. `lefthook` and `gitleaks` are pinned twice. Change `mise.toml` and `common/mise.root.toml` together. + +4. Re-resolve this toolbox's lock. -This rewrites `mise.lock` (for this toolbox's own tools) with the newly -resolved version and checksum. `mise.lock` itself is tracked, here and in -every generated project (docs/tour/01-toolchain.md) — what a `common/`- -level bump has *no* local file for is a pre-built one: `common/` ships no -`mise.lock` template, because a generated project doesn't have one until -its own first `mise install` creates it. From that point on it's a normal -tracked file, same as this toolbox's — there's just nothing sitting in -`common/` for this step to re-lock right now. + ```bash + mise install + mise lock + ``` -## 4. Verify + `common/` ships no `mise.lock`. `scaffold new` runs `mise lock` in each new project (`lock_toolchains` in `lib/project.sh`). + +## Verify ```bash +git diff mise.lock mise run checklist ``` -For a `common/`-level bump specifically, also generate a throwaway project -and run its own checklist, since nothing in this repository's own suite -exercises `common/mise.root.toml` end to end the way a real generation -does: +For a `common/` or adapter bump, also generate a project and run its checklist: ```bash -./scaffold new /tmp/probe --api nestjs && cd /tmp/probe && mise run checklist +./scaffold new ../probe --api nestjs +(cd ../probe && mise run checklist) ``` -## Done when +Delete the probe project afterwards. + +## If it fails -`mise run checklist` passes here, the probe project's own checklist -passes, and `mise.lock` (if this toolbox's own tools changed) reflects the -new version — check with `git diff mise.lock`. It's tracked, same as a -generated project's own (see docs/tour/01-toolchain.md). +| Symptom | Fix | +| --- | --- | +| A generated project warns `could not lock the toolchain` | Run `mise lock` in that project, then commit `mise.lock` | +| `laravel-api` fails `install` with `requires system php >= 8.3.0` | Install php 8.3 or newer on the host; mise cannot (ADR-0016) | diff --git a/docs/runbook/ci-is-red.md b/docs/runbook/ci-is-red.md index 52b9238..c02d593 100644 --- a/docs/runbook/ci-is-red.md +++ b/docs/runbook/ci-is-red.md @@ -1,50 +1,48 @@ # CI is red -CI calls exactly one of the nine contract tasks per failing step (ADR-0011). -Find the task name in the failed step's log, then look here. +When: a check is red on a generated project or on this toolbox. -| Task | What failing usually means | Where to look | +## Steps + +1. Read the failing job's name and find it in the tables below. +2. Reproduce with the command in the table, on a clean clone. +3. Fix the cause in the file the task names: the app's `mise.toml`, the source file, the test. Do not edit the workflow to hide what the task caught. +4. Before calling it done, ask what else reaches the code you changed. A fix checked only against the one repro that prompted it has missed adjacent paths four times in this project's history. + +### A generated project + +`ci ()` runs `mise run //:ci-unit` in each config root the change touched; `ci-unit` runs `install`, `format`, `lint`, `check` and `test` in that order (ADR-0011). + +| Job, or `ci-unit` step | Usually means | Reproduce | +| --- | --- | --- | +| `install` | Lockfile out of sync, or a supply-chain guard: a too-fresh dependency, an unapproved native build (ADR-0017) | `mise run //:install` | +| `format` | Unformatted code was committed | `mise run //:format-fix`, then commit the diff | +| `lint` | A real lint violation, or the lint config changed | `mise run //:lint` | +| `check` | A type error, or `phpstan` on Laravel | `mise run //:check`; it never writes, so it repeats what CI saw | +| `test` | A failing test, or a test that needs a file a fresh checkout lacks | Move aside what `.gitignore` excludes, then `mise run //:test` | +| `changes` | The path filter failed; it runs only on a pull request | The job log | +| `commitlint` | A commit in the pull request is not a Conventional Commit; runs only on a pull request | Reword the commit | +| `codeql`, `zizmor`, `gitleaks` | A security finding; `gitleaks` runs `mise run secrets` | `mise run secrets` for gitleaks; the job log for the others | +| `build` (Build workflow), `image` (Release workflow) | The Docker image does not build | `docker build -f `, with the `context` and `dockerfile` from `images:` in the project's `build.yml` | + +`build` is not part of `ci-unit`. Locally it runs inside `checklist`, which `pre-push` runs. + +### This toolbox + +| Job | Usually means | Reproduce | | --- | --- | --- | -| `install` | Lockfile out of sync with the manifest, or a supply-chain guard tripped (a fresh `minimumReleaseAge` violation, an unapproved native build). | The dependency you (or a dependency's dependency) just bumped; ADR-0017. | -| `format` | Someone committed unformatted code, or ran `format-fix` locally without committing the result. | `git diff` after running the adapter's own `format-fix` task. | -| `format-fix` | Should never run in CI — it writes. If you see it here, the pipeline is miswired. | `common/.github/workflows/ci.yml` / the reusable `app-ci.yml` call. | -| `lint` | A real lint violation, or the lint config itself changed underneath the code. | The step's own output names the file and rule. | -| `check` | A type error, or (Laravel) `phpstan` found something real. | The step's own output; `check` never modifies files, so re-running it locally reproduces exactly what CI saw. | -| `test` | A real test failure, or a test that depends on state a fresh checkout doesn't have. | Re-run the same task locally against a clean clone before assuming CI is wrong. | -| `build` | The app doesn't compile, or a build-time dependency (an env var, a generated file) is missing in CI that exists locally. | Diff what CI's environment provides against your own. | -| `ci-unit` | An aggregate of `install`/`format`/`lint`/`check`/`test` — read which sub-step actually failed; the aggregate name alone doesn't say. | The step's full log, not just its final line. | -| `checklist` | Same as `ci-unit`, plus `build` — this is what `pre-push` runs locally, so a red `checklist` in CI after a clean local run usually means an environment difference, not a code difference. | Compare the CI runner's toolchain (`mise ls` in the job log, if captured) against local `mise ls`. | - -Outside the nine tasks: a red `zizmor` job means a workflow file itself has -a static-analysis finding (untrusted input interpolated into a `run:` -block is the common one — see `.github/workflows/adapters.yml`'s own -`env:`-first pattern for the fix). A red `provenance` job's `self-test` -means `scripts/check-provenance.sh` itself regressed, not that upstream -drifted; its `check` job means upstream actually moved — see -`docs/runbook/sync-with-upstream-immich.md`. - -- `tests/contract.bats` fails with " runs an adapter generator to completion", or the toolbox's `unit` job hits its 5-minute timeout: that `test-unit` suite runs an adapter generator. Move it to `test-integration` in `mise.toml`. - -## Before you call a fix done - -This project's own history has four separate fixes that each stopped the -exact failure that motivated them and each still missed an adjacent path -the same input space allowed: a cleanup trap in `common/install.sh` that -covered a signal landing mid-function but missed two plain `return` paths -out of the same function; a workflow `if:` in `provenance.yml` that -covered the outcome it checked but missed GitHub's implicit `success()` -gate; a tier default in `lib/adapter.sh`'s `load_adapter` that covered a -stripped `ADAPTER_TIER` field but missed a missing `adapter.env` file -entirely; and a job condition in `provenance.yml` that covered a pull -request but missed a manual `workflow_dispatch` run. Four unrelated -mechanisms, the same shape every time: verified against the one repro that -motivated the fix, not against the full input space the fix now lives -inside. Before marking a red-CI fix done, ask what else can reach the code -you just changed — not just whether today's repro now passes. - -## Done when - -The failing step's log names a real cause you can point at, not just a -red X — and the fix lands as a task-scoped change (the adapter's `mise.toml`, -the file `lint`/`check` complained about), never as a workflow edit that -papers over what the task actually caught. +| `unit` | `lint` or `test-unit` failed | `mise run ci-unit` | +| `unit`, 5-minute timeout, or `tests/contract.bats` fails with " runs an adapter generator to completion" | A `test-unit` suite runs an adapter generator | Move that suite to `test-integration` in `mise.toml` | +| `integration` | A suite that generates a real project failed | `mise run test-runner`, which sets `CI` as a runner does | +| `zizmor` | A workflow finding; the common one is untrusted input interpolated into `run:` | `mise exec -- zizmor .github/workflows/`; pass the value through `env:` as `.github/workflows/adapters.yml` does | +| `pull-request-body` | The pull request body lost a `##` heading of `.github/pull_request_template.md` | Restore the template's headings | +| `self-test` (Provenance) | `scripts/check-provenance.sh` itself regressed | `bats tests/provenance.bats` | +| `check` (Provenance) | A `verbatim` file no longer matches the commit pinned in `UPSTREAM` | `docs/runbook/sync-with-upstream-immich.md` | +| `smoke`, `smoke-tier-b`, `deploy`, `deploy-tier-b`, `deploy-multi-app` (Adapters) | An adapter's generator or image broke | `bats tests/new-.bats`, or `./scripts/deploy-check.sh ` | +| `compose` (Adapters) | A compose file, Dockerfile or `install.sh` invariant broke | `bats tests/compose.bats` | +| `services` (Adapters) | One adapter, database and cache combination does not generate or pass its checklist | `./scaffold new ../demo --api --db --cache `, then `mise run //apps/api:checklist` in it | + +## Verify + +- The failing job is green on a re-run of the fixed commit. +- The fix is in a task, a source file or a test, not in a workflow that skips the failing step. diff --git a/docs/runbook/cut-a-release.md b/docs/runbook/cut-a-release.md index 39b59c6..2bc7a75 100644 --- a/docs/runbook/cut-a-release.md +++ b/docs/runbook/cut-a-release.md @@ -1,56 +1,52 @@ # Cut a release -This applies to a generated project (Release Please, `common/release-please-config.json`), -not to this toolbox itself, which has no release process of its own. +When: a generated project should ship a version. This toolbox has no release process; its tags are manual (`CONTRIBUTING.md`). -## 0. Once per repository: let Actions open pull requests +![Release flow](../diagrams/release-flow.svg) -Release Please works by opening a pull request, and a new repository -forbids that by default — the run fails with `GitHub Actions is not -permitted to create or approve pull requests` after it has already -pushed its branch, so the symptom appears late and looks like a -permissions bug in the workflow. It is a repository setting: +## Steps -```bash -gh api -X PUT "repos///actions/permissions/workflow" \ - -f default_workflow_permissions=read \ - -F can_approve_pull_request_reviews=true -``` - -Or Settings → Actions → General → Workflow permissions → *Allow GitHub -Actions to create and approve pull requests*. +1. Once per repository, run `scaffold publish`. It allows Actions to open pull requests, which Release Please needs, and sets the release app secrets when they are in the environment ([publish-a-project](publish-a-project.md)). -## 1. Merge conventional commits to `main` +2. Merge Conventional Commits to `main`. `feat:` and `fix:` move the version; `docs:` and `chore:` do not (`common/release-please-config.json`). -Every commit must already be a Conventional Commit — enforced at -`commit-msg` by lefthook and commitlint, and again in CI. `feat:` and -`fix:` commits are what move the version; `chore:`/`docs:` do not. +3. Wait for Release Please. `release.yml` runs on every push to `main` and keeps one pull request, `chore(main): release `, with the changelog. -## 2. Let Release Please open (or update) its release PR + ```bash + gh pr list --search "release in:title" + ``` -`common/.github/workflows/release.yml` runs on every push to `main` and -maintains one standing pull request with the next version's changelog, -computed from the commits merged since the last release. +4. Review the changelog against what shipped, then merge the pull request. -## 3. Review the release PR + ```bash + gh pr merge --squash + ``` -Check the generated changelog against what actually shipped. This is the -only manual step in the whole flow. + Merging it is the release. Only on that merge, `app-release.yml`'s jobs: -## 4. Merge it + | Job | Publishes | + | --- | --- | + | `image` | Every application's image, tagged ``, `.`, `latest`, `sha-` | + | `assets` | `compose.yaml`, `example.env`, `install.sh` on the GitHub Release | + | `deploy` | Nothing: it runs only when `vars.DEPLOY_TARGET` is set, and then fails, since no deploy adapter exists (ADR-0014) | -Merging the release PR is the release. `release.yml`'s image and asset -jobs only run `if: needs.release-please.outputs.released == 'true'` — -i.e. only on this specific merge — and publish the version tags -(`1.4.0` and `1.4`, for example) plus `latest`. +## Verify -## 5. Confirm the image published +```bash +gh release list --limit 1 +gh release view --json assets --jq '.assets[].name' +gh run list --workflow release.yml --limit 1 +``` -Check the workflow run for `release.yml` succeeded, then confirm the new -tag exists in the registry a client's `compose.yaml` points at. +- The release lists `compose.yaml`, `example.env` and `install.sh`. +- `ghcr.io//-:` exists for every application. +- A host that runs `install.sh` pulls the new version ([first-project-walkthrough](first-project-walkthrough.md)). -## Done when +## If it fails -The GitHub Release exists, the semver and `latest` image tags are -published, and a client pinning `IMAGE_TAG` to the new version (or to -`latest`) can pull it. +| Symptom | Fix | +| --- | --- | +| `GitHub Actions is not permitted to create or approve pull requests` | Run `scaffold publish` in the project | +| Checks on the release pull request sit at `Action required` | No `RELEASE_APP_ID`/`RELEASE_APP_PRIVATE_KEY`. Merge anyway, or set them with `scaffold publish` | +| No release pull request after a merge | The merged commits are only `docs:` or `chore:` | +| `image` skipped | `images:` in the project's `release.yml` is `"[]"`. Copy the entries from `build.yml`, which `scaffold new` and `scaffold add` write to both | diff --git a/docs/runbook/first-project-walkthrough.md b/docs/runbook/first-project-walkthrough.md index af9cdeb..a221054 100644 --- a/docs/runbook/first-project-walkthrough.md +++ b/docs/runbook/first-project-walkthrough.md @@ -1,351 +1,162 @@ # Walk through a first project -A scripted run of everything a new engineer does between cloning this toolbox -and shipping a release from a project it generated. Follow it in order on a -machine that has never run the toolbox, and record where it goes wrong — the -purpose is to find the rough steps, not to prove they are smooth. - -Each step states what to expect. A step that does something other than what is -written here is a finding, even when it still works. - -## 0. Prerequisites - -`git` and `mise` installed. A GitHub account, and `gh auth login` completed. -Docker only matters for step 10. - -## 1. Clone and install - -```sh -git clone https://github.com/ttncode/scaffold.git -cd scaffold -mise install -``` - -Expect: mise installs jq, yq, bats, shellcheck, zizmor and rush, and prints no -prompt. A prompt about trusting the config means step 1 is a finding — the -README does not mention one. - -## 2. Prove the toolbox runs - -```sh -./scaffold list -./scaffold lint -``` - -Expect: `list` prints eight rows now, not four — every adapter and every -service, as name, second column, tier. The second column is the adapter's -role (`api`, `app`, `web`) or the service's kind (`database`, `cache`); a -service carries no tier, so its third column reads `-` — tiers (ADR-0012) -measure verification cost for adapters, and a service's own manifest names -no such thing (ADR-0019). `list --adapters` or `list --services` narrows to -one half. `lint` still prints nothing and exits 0. Anything else stops the -walkthrough here. - -## 3. Try the wizard - -```sh -./scaffold -``` - -Run this in an actual terminal. Expect an interactive wizard: a name prompt, -then shape (`web+api`, `app`, `api`, `web`), then one to four more screens -depending on the shape. Type `demo-app` at the name prompt; at each menu -after that, typing a letter jumps the highlight to the first option that -starts with it — arrows work too, and either way Enter takes the highlighted -option. Reach `web+api`, `nextjs`, `laravel-api`, `postgres`, `redis` that -way, then `n` at "Generate this project?" to stop at the summary without -generating anything. Expect the line above the prompt to read: - -``` -scaffold new demo-app --web nextjs --api laravel-api --db postgres --cache redis -``` - -That is the command step 5 runs, argument order aside — reaching it by menu -first is how a first-time user is meant to find it. See -docs/tour/09-wizard.md for the question logic and its known limits. - -Piped, redirected, or run from a script — a closed stdin, not a terminal — -`scaffold` with no arguments takes none of this and prints usage and exits 1, -same as before: - -```sh -printf '' | ./scaffold -``` - -## 4. Make it callable from anywhere - -```sh -ln -s "$PWD/scaffold" ~/.local/bin/scaffold -cd ~/some/other/directory -scaffold list -``` - -Expect: the same output as step 2. A report of a missing tool means -`load_toolchain_env` could not read this toolbox's mise environment — check -`mise env -C ` by hand. A later `scaffold new relative-name` that -lands inside the toolbox rather than in the current directory is a finding: -nothing in `scaffold` may change directory before resolving the target. - -## 5. Generate a project - -```sh -cd ~/playground -scaffold new demo-app --api laravel-api --web nextjs --db postgres --cache redis -``` - -Expect: two generators run, then `created …/demo-app`. Expect a warning naming -the GitHub account it detected. Expect it to take several minutes. - -Then read what it made before doing anything else: - -```sh -cd demo-app -git log --oneline # one commit, "feat: scaffold project" -cat mise.toml # [monorepo] config_roots = apps/web, apps/api, docs - # [vars] database = "postgres", cache = "redis" -ls .github/workflows # five call sites -grep -l database compose*.yaml # all three: compose.yaml, .dev.yaml, .test.yaml -cat example.env # DB_PASSWORD and REDIS_PASSWORD, appended for the services chosen -cat apps/api/.env.example # DB_CONNECTION=pgsql and REDIS_* — the driver's own variables -``` - -## 6. Run what CI will run, before pushing - -```sh -mise run //docs:ci-unit -mise run //apps/web:ci-unit -mise run //apps/api:ci-unit -``` - -Expect: all three pass. This is the same command CI issues per config root. - -Now check the harder thing — that they pass on a machine that has none of your -local state: - -```sh -mv apps/api/.env /tmp/env-aside -mv apps/web/.next /tmp/next-aside 2>/dev/null -mise run //apps/api:ci-unit && mise run //apps/web:ci-unit -mv /tmp/env-aside apps/api/.env -``` - -Expect: still pass. A failure here is a real finding — it means a check depends -on a file that is not committed, and CI will fail where you succeeded. - -Which files those are depends on the adapter, and this is the step where -guessing costs you a red pull request. Move aside everything the app's -`.gitignore` excludes that some task writes, not just the two named above. -For `laravel-inertia` that is `apps/app/resources/js/actions`, -`apps/app/resources/js/routes` and `apps/app/public/build` — the first two -are written by the vite build's wayfinder plugin and compiled against by -`types:check`, the third is the vite manifest, without which every test that -renders an inertia page returns 500. Running the checks with those in place -is not the same experiment as CI runs. - -A `nestjs` app's `check` and `build` also depend on a `prisma` task now -(`prisma generate` first, when the project has a database). Moving its `.env` -aside the same way still passes — `prisma generate` only parses the schema, -it does not need `DATABASE_URL` to resolve. Nothing here uses `nestjs` yet; -this matters again once step 11 adds one. - -## 7. Install the hooks and make a commit - -```sh -mise exec -- lefthook install -git checkout -b feat/health -``` - -`lefthook` is pinned in the generated project's own `mise.toml`, not on your -PATH — bare `lefthook install` works only if your shell profile already runs -`mise activate`, which step 0's prerequisites do not ask for. - -Add a small feature with a test — for `laravel-api`, a `/health` route in -`apps/api/routes/web.php` and a `HealthTest` beside the other feature tests. - -```sh -mise run //apps/api:ci-unit -git add -A -git commit -m "feat(api): add a health endpoint" -``` - -Expect: pre-commit runs pint and gitleaks; commit-msg runs commitlint. Then -prove the gate works: - -```sh -git commit --allow-empty -m "added a health thing" -``` - -Expect: rejected, naming the convention. - -## 8. Push and open a pull request - -```sh -git checkout main -scaffold publish -``` - -Expect: it creates the ttncode/demo-app repository — the one the project's own -`compose.yaml`, `install.sh` and workflows already name — pushes `main`, -allows Actions to open pull requests, and protects `main`. It says what it did -and what it skipped. `scaffold publish --dry-run` prints that list without -doing any of it. See -[ADR-0024](../decisions/0024-publishing-a-project-is-part-of-generating-it.md). - -Expect this step to take about ninety seconds. The push runs the pre-push -hook, which runs the whole checklist — every config root's `ci-unit` and -`build`. It looks like a hang and is not. - -Expect a warning about `RELEASE_APP_ID`/`RELEASE_APP_PRIVATE_KEY` unless both -are in your environment, and, on a free account with a private repository, a -warning that `main` could not be protected — rulesets need GitHub Pro there. -Both are findings to act on, not failures. - -```sh -git checkout feat/health -git push -u origin feat/health -gh pr create --fill -gh pr checks --watch -``` - -The first `gh pr checks --watch` usually exits 1 straight away with `no checks -reported on the 'feat/health' branch`. Nothing is wrong: GitHub has not -registered the check runs yet, and `--watch` does not wait for a first one to -appear. Give it twenty seconds and issue it again. - -Expect: `CI / ci / ci (apps/api)` runs, because `apps/api` changed. Expect -`CI / ci / changes` to skip roots that did not. Expect `commitlint` to run — -it only ever runs on a pull request. Expect every check green, including `security / codeql`, -`security / gitleaks` and `security / zizmor` — on a private repository those -three each need something the workflow grants them explicitly, so a failure -there is a finding, not the normal state. How many checks there are depends on -the project: six named jobs plus one `ci ()` for each config root the -commit touched, so this project has seven and an api-only one has six. - -Allowing Actions to open pull requests is required, not optional: without it -Release Please cannot open its pull request later, and the failure appears -several steps away — as "GitHub Actions is not permitted to create or approve -pull requests", on the release job. `scaffold publish` sets it every time it -runs, including against a repository that already exists. - -## 9. Merge, and let the release happen - -```sh -gh pr merge --squash --delete-branch -gh run list --limit 5 -``` - -Expect: five workflows on `main`, all green. Expect Release Please to open -`chore(main): release 0.2.0` — `feat:` moves the minor version. - -If that pull request's checks sit at `Action required`, the repository has no -`RELEASE_APP_ID`/`RELEASE_APP_PRIVATE_KEY`; a pull request opened with -`GITHUB_TOKEN` starts no workflow. Merging it directly still releases. - -Those runs do not stay amber. Once the approval window passes, or once the -squash merge deletes the head branch, they end as `failure`, and `gh run view` -explains them with `This run likely failed because of a workflow file issue`. -There is no workflow file issue. Expect to be left with three red runs in the -history that the release did not depend on and that say nothing true about -your project. - -```sh -gh pr merge --squash -gh release list -``` - -Expect: `v0.2.0`, and the image tagged `0.2.0`, `0.2`, `latest`, `sha-…`. - -## 10. Run it - -```sh -git checkout main -git pull -./install.sh -``` - -Expect: it downloads the release assets and starts the stack. `RepoUrl` and -`compose.yaml`'s image already name the ttncode/demo-app repository step 8 -created: `scaffold new` wrote both from the GitHub owner it resolved and this -project's own directory name, the same pair it wrote into `build.yml` and -`release.yml`. - -Check that they agree, because a repository renamed after generation breaks -the assumption and this is where it would show: - -```sh -grep -n 'image: ghcr' compose.yaml .github/workflows/build.yml -grep -n '^RepoUrl=' install.sh -``` - -Expect: one owner and one project name across all three. If the repository -was renamed, edit them here and cut another release — `install.sh` -re-downloads `compose.yaml` on every run, so editing the deployed copy is -undone the next time it runs. - -This project is private (step 8's `--private`), so `install.sh` needs one -more thing: a personal access token scoped `repo` and `read:packages`, and -`jq` on the host to parse the release JSON only the token path reads. A -private release's browser download URL returns 404 even with a token -attached, so `install.sh` switches to the GitHub API endpoint instead of -only adding a header — an operator who tries the old URL with a token and -still sees 404 would otherwise conclude the token is wrong. - -```sh -GITHUB_TOKEN=ghp_... bash install.sh -curl -fsS http://localhost:8080/api/health/live -``` - -Expect: `install.sh` downloads `compose.yaml` and `example.env` from -`v0.2.1` through that API endpoint, generates passwords, signs in to -`ghcr.io`, starts the stack, runs the migration task, and prints one line per -application — `web is running on http://localhost:8080`, `api is running on -http://localhost:8081`. The curl returns `200`. - -Every application in the project is published and running, each on its own -port (ADR-0022) — `WEB_PORT` and `API_PORT` in `.env`, printed one per line -when `install.sh` finishes. `nextjs` ships no readiness route, because the -`web` role takes no database driver and there is nothing for one to query; -curl the api's instead, `curl -fsS http://localhost:8081/health/ready`, -which returns `200`. See ADR-0021 for both routes. - -```sh -docker compose -f app/compose.yaml down -v -``` - -## 11. Add a second application to the existing project - -```sh -cd ~/playground/demo-app -scaffold add apps/worker --adapter nestjs -git status -``` - -Expect: `apps/worker` exists, `mise.toml` gained a config root, `ci.yml`'s -`roots:` gained an entry, and `lefthook.yml` and `pnpm-lock.yaml` also -changed. Expect the changes to be left uncommitted for review. - -Two of those are conditional, and a diff that does not show them is not a -finding. `pnpm-workspace.yaml` changes only when the new app needs an -`allowBuilds` or `minimumReleaseAgeExclude` entry the project does not -already carry — adding a second `nestjs` app to a project that already has -one leaves it untouched, because `apps/*` already matched and prisma was -already decided. And `lefthook.yml` changes without gaining a hook: `nestjs` -contributes none, since prettier in the common layer already covers -typescript sources. - -The new application is wired to the database and cache this project already -recorded, not asked again: `apps/worker/.env.example` names a `DATABASE_URL` -and `REDIS_URL` for the same services `mise.toml`'s `[vars]` already records, -not whatever nestjs would default to on its own. `scaffold` cannot change a -project's database after generation (ADR-0019) — reading it back on `add` is -how a later application still ends up on the one already running. +When: an engineer is new to the toolbox, or a change to it needs an end-to-end run. Follow the steps in order on a clean machine. + +A step that does something other than its **Expect** is a finding, even when it still works. Record the step number, what was expected and what happened. + +## Steps + +1. **Prerequisites.** `git`, `mise`, `gh` with `gh auth login` done. Docker for step 10. + +2. **Clone and install.** + + ```sh + git clone https://github.com/ttncode/scaffold.git + cd scaffold + mise install + export PATH="$PWD:$PATH" + ``` + + Expect: mise installs `bats`, `shellcheck`, `shfmt`, `yq`, `jq`, `zizmor`, `rush`, `lefthook` and `gitleaks` without a prompt. Put the clone on `PATH`; a symlink to `scaffold` does not work. + +3. **Prove the toolbox runs.** + + ```sh + scaffold list + scaffold lint + printf '' | scaffold; echo "exit $?" + ``` + + Expect: `list` prints one row per adapter and service: name, role or kind, tier (`-` for a service). `lint` prints nothing. With no terminal, `scaffold` prints usage and exits 1. + +4. **Try the wizard.** In a real terminal: + + ```sh + scaffold + ``` + + Type `demo-app`, then pick `web+api`, `nextjs`, `laravel-api`, `postgres`, `redis`. Typing a letter jumps to the first option starting with it; Enter takes it. Answer `n` at "Generate this project?". + Expect: `scaffold new demo-app --web nextjs --api laravel-api --db postgres --cache redis` above the prompt, and nothing generated ([09-wizard](../tour/09-wizard.md)). + +5. **Generate a project.** Outside the toolbox: + + ```sh + cd ~/playground + scaffold new demo-app --web nextjs --api laravel-api --db postgres --cache redis + cd demo-app + ``` + + Expect: a warning naming the detected GitHub owner, several minutes of generators, then `created …/demo-app` and the next steps. Then: + + | Command | Expect | + | --- | --- | + | `git log --oneline` | One commit, `feat: scaffold project` | + | `cat mise.toml` | `config_roots` with `apps/api`, `apps/web`, `docs`; `[vars]` `database = "postgres"`, `cache = "redis"`, `image` | + | `cat .scaffold.toml` | The toolbox version and `"apps/web" = "nextjs"`, `"apps/api" = "laravel-api"` | + | `ls .github/workflows` | `build.yml`, `ci.yml`, `docs.yml`, `release.yml`, `security.yml` | + | `grep -l database compose*.yaml` | `compose.yaml`, `compose.dev.yaml`, `compose.test.yaml` | + | `cat example.env` | `WEB_PORT=8080`, `API_PORT=8081`, `DB_PASSWORD`, `REDIS_PASSWORD` | + | `cat apps/api/.env.example` | `DB_CONNECTION=pgsql` and `REDIS_*` | + +6. **Run what CI runs.** + + ```sh + mise install && mise exec -- lefthook install + mise run //docs:ci-unit + mise run //apps/web:ci-unit + mise run //apps/api:ci-unit + ``` + + Expect: all pass. `lefthook` is pinned in the project, not on `PATH`, hence `mise exec`. + + Then repeat with local state moved aside: everything the app's `.gitignore` excludes that a task writes. A failure here fails CI too. + + | Adapter | Move aside | + | --- | --- | + | `laravel-api` | `apps/api/.env` | + | `nextjs` | `apps/web/.next` | + | `laravel-inertia` | `apps/app/.env`, `apps/app/resources/js/actions`, `apps/app/resources/js/routes`, `apps/app/public/build` | + +7. **Commit through the hooks.** + + ```sh + git checkout -b feat/health + # add a /health route in apps/api/routes/web.php and a HealthTest beside the other feature tests + mise run //apps/api:ci-unit + git add -A && git commit -m "feat(api): add a health endpoint" + git commit --allow-empty -m "added a health thing" + ``` + + Expect: `pre-commit` runs prettier, pint and gitleaks; `commit-msg` runs commitlint. The second commit is rejected. + +8. **Publish and open a pull request.** + + ```sh + git checkout main + scaffold publish + git checkout feat/health + git push -u origin feat/health + gh pr create --fill + gh pr checks --watch + ``` + + Expect: `scaffold publish` creates the private repository, pushes `main` and applies the settings ([publish-a-project](publish-a-project.md)). The push runs `pre-push`, the whole `checklist`, so it takes minutes. + + Expect these checks: `changes`, `ci (apps/api)`, `commitlint`, `codeql`, `zizmor`, `gitleaks`, and the docs `build`. A root the commit did not touch gets no `ci ()`. If `gh pr checks --watch` exits at once with `no checks reported`, run it again after a few seconds. + +9. **Merge and release.** + + ```sh + gh pr merge --squash --delete-branch + gh run list --limit 5 + ``` + + Expect: every workflow on `main` green, and a Release Please pull request. Merge it and follow [cut-a-release](cut-a-release.md). + + Without `RELEASE_APP_ID`/`RELEASE_APP_PRIVATE_KEY`, that pull request's checks sit at `Action required`, then expire red; `gh run view` reports `This run likely failed because of a workflow file issue`. There is no workflow file issue, and merging it still releases. + +10. **Run the release.** The repository is private, so `install.sh` needs a token with `repo` and `read:packages`, and `jq` on the host. + + ```sh + GITHUB_TOKEN= bash install.sh + curl -fsS http://localhost:8080/api/health/live + curl -fsS http://localhost:8081/health/ready + docker compose -f app/compose.yaml down -v + ``` + + Expect: `install.sh` creates an `app` directory, downloads `compose.yaml` and `example.env` from the latest release, generates passwords, signs in to `ghcr.io`, starts the stack, runs `migrate`, then prints `web is running on http://localhost:8080` and `api is running on http://localhost:8081`. Both curls succeed. `nextjs` has no readiness route (ADR-0021). + + The owner and name must agree in three places; a repository renamed after generation breaks them: + + ```sh + grep -n '^image' mise.toml + grep -n 'ghcr.io' compose.yaml + grep -n '^RepoUrl=' install.sh + ``` + + Expect: one owner and one project name across all three. If the repository was renamed, edit these files in the project and cut another release: `install.sh` re-downloads `compose.yaml` on every run, overwriting any edit to the deployed copy. + +11. **Add an application.** + + ```sh + scaffold add apps/worker --adapter nestjs + git status + ``` + + Expect: `apps/worker` staged. Unstaged edits to `mise.toml` (a config root), `ci.yml` (`roots:`), `build.yml` and `release.yml` (`images:`), `compose.yaml` (a `worker` service), `example.env` (`WORKER_PORT`), `.scaffold.toml` and `lefthook.yml`. `pnpm-workspace.yaml` changes only when it gains a new `allowBuilds` or `minimumReleaseAgeExclude` entry. + + Expect `apps/worker/.env.example` to name `DATABASE_URL` and `REDIS_URL`: `scaffold add` reads `[vars]` instead of asking (ADR-0019). + +12. **Clean up.** Delete `~/playground/demo-app`, and the GitHub repository if it was a trial. + +## Verify + +- Every step matched its Expect, or each difference is recorded as a finding. ## What counts as a finding - A step that needs a command this page does not give - An error message that does not say what to do next - A check that passes locally and fails in CI, or the reverse -- Anything that required reading the source to get past -- Any wait longer than the step led you to expect - -Record each one with the step number, what was expected, and what happened. +- Anything that needed reading the source to get past +- A wait longer than the step said diff --git a/docs/runbook/publish-a-project.md b/docs/runbook/publish-a-project.md new file mode 100644 index 0000000..7e7a7ea --- /dev/null +++ b/docs/runbook/publish-a-project.md @@ -0,0 +1,74 @@ +# Publish a project + +When: a project from `scaffold new` has no GitHub repository yet, or its repository is missing settings. + +`scaffold publish` creates the repository the project already names and applies the settings no file records (ADR-0024). It is idempotent: against an existing repository it applies only the settings. + +## Steps + +1. Sign in to GitHub. + + ```bash + gh auth login + ``` + +2. Optional: export the release app secrets, so Release Please opens its pull request as the app. + + ```bash + export RELEASE_APP_ID= + export RELEASE_APP_PRIVATE_KEY="$(cat .private-key.pem)" + ``` + +3. From a clean `main` in the project, preview. + + ```bash + git checkout main + scaffold publish --dry-run + ``` + +4. Publish. + + ```bash + scaffold publish # private, main protected + ``` + + | Flag | Effect | + | --- | --- | + | `--private` | Default visibility | + | `--public` | Public repository | + | `--no-protect` | Skip the `main` ruleset | + | `--dry-run` | Print the plan; change nothing | + | `[dir]` | The project directory; default is the git root of the current directory | + +## What it does + +| Step | Does it (`lib/publish.sh`) | Skips or warns (`scaffold`) | +| --- | --- | --- | +| Create `/` and push `main` | `create_repo` | `cmd_publish`: skipped when the repository exists | +| Allow Actions to open pull requests | `allow_actions_to_open_pull_requests` | Never skipped | +| Secret scanning and push protection | `enable_secret_scanning` | `apply_repo_settings`: warns when the plan lacks it | +| Ruleset `main`: pull request required, no force-push, no deletion | `protect_main` | `protect_main_branch`: skipped on `--no-protect` or an existing ruleset named `main` (`main_is_protected`); warns when the plan lacks it | +| Set `RELEASE_APP_ID` and `RELEASE_APP_PRIVATE_KEY` | `set_release_secrets` | `apply_repo_settings`: warns when either variable is unset | + +`/` is read from `[vars] image` in the project's `mise.toml` (`repo_slug`). There is no flag to change it. + +## Verify + +```bash +gh repo view / +gh api repos///rulesets --jq '.[].name' # main +scaffold publish --dry-run # "would leave … alone" +``` + +## If it fails + +| Symptom | Fix | +| --- | --- | +| `publish needs the GitHub CLI` or `not signed in to GitHub` | Install `gh`, run `gh auth login` | +| `check out main first` | `git checkout main`; `gh repo create --push` makes the current branch the default | +| `has uncommitted changes` | Commit, then run again | +| `already pushes to , but it names /` | Point `[vars] image`, `compose.yaml`, `install.sh` and the build workflows at the existing repository, or `git remote remove origin` | +| `could not finish creating` | The repository may exist half-made. Check it on GitHub, then run again | +| Warning `main is unprotected` | A free account's private repository cannot use rulesets. Make it public or upgrade, then run again | +| Warning `no secret scanning` | A private repository needs Advanced Security. The CI gitleaks scan still runs | +| Warning `no RELEASE_APP_ID/RELEASE_APP_PRIVATE_KEY` | Checks on the release pull request sit at `Action required`, then expire red, and `gh run view` reports `This run likely failed because of a workflow file issue`. There is no workflow file issue. Merging it still releases. To fix, export both and run again | diff --git a/docs/runbook/rotate-a-leaked-secret.md b/docs/runbook/rotate-a-leaked-secret.md index c210cb9..5bcad6b 100644 --- a/docs/runbook/rotate-a-leaked-secret.md +++ b/docs/runbook/rotate-a-leaked-secret.md @@ -1,46 +1,47 @@ # Rotate a leaked secret -gitleaks runs at `pre-commit` locally (`common/lefthook.yml`) and again in -CI, so most leaks never reach a pushed commit. This runbook is for the one -that does anyway — a hook bypassed with `--no-verify`, or a secret that -predates the hook being installed. +When: a secret reached a pushed commit, past gitleaks at `pre-commit` (`common/lefthook.yml`) and in CI. -Rotate first, rewrite history second. A rotated secret makes every leaked -copy worthless immediately, including ones already cloned or cached -somewhere history rewriting can't reach; rewriting history first, with the -old secret still valid, leaves a window where anyone who already has the -commit still has a working credential. +Rotate first, rewrite history second: a rotated secret makes every copy worthless, including clones history rewriting cannot reach. -## 1. Rotate the credential +## Steps -The database password is the one this toolbox generates for you -(`install.sh`'s `generate_service_passwords`): change `DB_PASSWORD` in the -affected host's `.env` to a new value and restart the stack — -`docker compose up -d` picks up the new value without touching the -`database` volume's existing data. For any other leaked credential (a -third-party API key, a registry token), rotate it at the provider first. +1. Rotate the credential. + - A third-party key or token: revoke and reissue it at the provider. + - A service password from `install.sh` (`DB_PASSWORD`, `REDIS_PASSWORD`): change it inside the running service first. The database images read `DB_PASSWORD` only when their volume is empty, so editing `.env` alone locks the apps out. -## 2. Confirm the new credential works + ```bash + cd app + docker compose exec database psql -U app -d app -c "ALTER USER app PASSWORD ''" # postgres + docker compose exec database mysql -uroot -p -e "ALTER USER 'app'@'%' IDENTIFIED BY ''; ALTER USER 'root'@'%' IDENTIFIED BY ''; ALTER USER 'root'@'localhost' IDENTIFIED BY ''" # mysql + docker compose exec database mongosh -u app -p --authenticationDatabase admin --eval "db.getSiblingDB('admin').changeUserPassword('app', '')" # mongodb + ``` -Restart the affected service and verify it comes up healthy against the -new value before touching git history — there's no reason to rewrite -history for a secret that turned out not to matter yet, or to discover the -new credential is wrong only after history is already rewritten. + `redis` reads `REDIS_PASSWORD` on every start and needs no command. -## 3. Remove the secret from history +2. Put the new value in `.env` and restart. -Only now: `git filter-repo` (or the BFG Repo-Cleaner) to strip the commit -that introduced it, then force-push the rewritten history and have every -collaborator re-clone rather than merge. + ```bash + docker compose up -d + docker compose ps # every service healthy before step 3 rewrites history + ``` -## 4. Add a gitleaks rule if this shape wasn't caught +3. Remove the secret from history with `git filter-repo` or BFG Repo-Cleaner, force-push, and have every collaborator re-clone. -If gitleaks didn't flag the leak (a secret shape it doesn't recognize by -default), that's the real gap — a rotated secret fixes this one incident, -but the same shape leaks again the next time someone bypasses the hook. +4. If gitleaks did not flag the leak, add a rule for that secret's shape, or the next bypassed hook leaks it again. -## Done when +## Verify -The old credential no longer works anywhere, the new one is confirmed -live, and the leaked commit is gone from every remaining clone of the -repository. +```bash +docker compose ps # every service healthy +mise run secrets # in the project: gitleaks over the whole history +``` + +- The old credential is refused wherever it was used. + +## If it fails + +| Symptom | Fix | +| --- | --- | +| Apps fail to connect after the restart | The password inside the database was not changed. Run the step 1 command with the old password, or restore the old `.env` value and repeat step 1 | +| `mise run secrets` still reports the secret | History still holds it. Repeat step 3 | diff --git a/docs/runbook/sync-with-upstream-immich.md b/docs/runbook/sync-with-upstream-immich.md index dbeba44..76ab17d 100644 --- a/docs/runbook/sync-with-upstream-immich.md +++ b/docs/runbook/sync-with-upstream-immich.md @@ -1,49 +1,49 @@ # Sync with upstream immich -`.github/workflows/provenance.yml` runs `scripts/check-provenance.sh` -monthly against the commit pinned in `UPSTREAM`, and opens an issue titled -"upstream drift detected" when a `verbatim` row in `docs/PROVENANCE.md` no -longer matches. A `DRIFTED` row is not itself a failure — leaving it -`DRIFTED` is. +When: moving the pinned immich commit forward, or after `.github/workflows/provenance.yml` opens an issue titled "upstream drift detected". -## 1. Read the diff +`scripts/check-provenance.sh` diffs every `verbatim` row of `docs/PROVENANCE.md` against the commit pinned in `UPSTREAM`, read from a local clone. It never fetches. `DRIFTED` means the file here differs from that commit. -The issue body is `check-provenance.sh`'s own output: which file, and the -diff against the pinned commit. Decide which of three responses applies — -there is no fourth. +## Steps -## 2. Pull the change in +1. Update the local clone. The default path is `~/workspace/playground/immich`; set `SCAFFOLD_UPSTREAM_CLONE` for another. -If upstream's edit is one this project should have too (a real bug fix, a -security update), apply it here and leave the row `verbatim`. Re-run the -check to confirm it now reports `ok`. + ```bash + git -C ~/workspace/playground/immich fetch origin + ``` -```bash -SCAFFOLD_UPSTREAM_CLONE=/path/to/immich ./scripts/check-provenance.sh -``` +2. To move the pin, write the new commit into `UPSTREAM` as `immich-app/immich@`. Skip this step when answering a drift issue. + +3. Run the check. -## 3. Accept the divergence + ```bash + ./scripts/check-provenance.sh + ``` -If this project deliberately differs (the usual case — most `verbatim` -files stay that way specifically because they *shouldn't* diverge, but a -change here can still be a considered "no"), reclassify the row `adapted` -in `docs/PROVENANCE.md` and record the reason next to it, same as every -other `adapted` row already does. +4. Resolve every `DRIFTED` row with one of three answers. -## 4. Reclassify as never having been a real comparison + | Answer | When | Do | + | --- | --- | --- | + | Take upstream | Upstream's version is one this project wants | `git -C show : > `; the row stays `verbatim` | + | Accept the divergence | This project differs on purpose | Change the row to `adapted` and write the reason in its Notes | + | Not derived | The resemblance was coincidence, checked with `diff` | Change the row to `original` and remove its upstream path; record the check as the "Out of scope, checked and rejected as rows" section does | -Rare: if the resemblance to upstream was coincidental rather than derived, -reclassify `original` and remove the "Upstream path" claim. Verify with -`diff` first — `docs/PROVENANCE.md`'s own "Out of scope, checked and -rejected as rows" section is the model for how to record that check. +5. Run the check again until it exits 0, then commit `UPSTREAM`, `docs/PROVENANCE.md` and any copied file together. -## 5. Bump `UPSTREAM` +## Verify -Once every drifted row is resolved, update `UPSTREAM` to the commit you -diffed against, so the next monthly run starts from here, not from the -older pin. +```bash +./scripts/check-provenance.sh; echo "exit $?" # "0 drifted, 0 missing, 0 errors", exit 0 +bats tests/provenance.bats +``` -## Done when +## If it fails -`scripts/check-provenance.sh` exits 0, and every row that changed -classification this round says why in `docs/PROVENANCE.md`, not just what. +| Symptom | Fix | +| --- | --- | +| `no local clone at ` | Clone immich there, or set `SCAFFOLD_UPSTREAM_CLONE` | +| ` is not a commit in ` | Step 1: fetch the clone | +| `UPSTREAM (…) is not of the form owner/repo@commit` | Fix the `UPSTREAM` line | +| `MISSING ` | A `verbatim` row names a file that no longer exists here. Remove or correct the row | +| `ERROR : could not read ` | The upstream path moved at the new commit. Correct the row's upstream path | +| `no verbatim rows found` | The table format broke; `check_verbatim_rows` reads rows starting with a backticked path | diff --git a/docs/runbook/update-a-project.md b/docs/runbook/update-a-project.md new file mode 100644 index 0000000..238b44e --- /dev/null +++ b/docs/runbook/update-a-project.md @@ -0,0 +1,69 @@ +# Update a project + +When: a project generated earlier should receive the toolbox changes made since. + +`scaffold update` diffs `common/` and each app's adapter from the commit recorded in `.scaffold.toml` to this checkout's `HEAD`, rewrites the patch onto the project's paths, and applies it. It never commits (ADR-0023). + +![scaffold update](../diagrams/scaffold-update.svg) + +## Steps + +1. Put the toolbox on the version to bring in, with no uncommitted edits. + + ```bash + git -C pull + git -C status --short # empty + ``` + +2. Commit or stash everything in the project. + + ```bash + git -C status --short # empty + ``` + +3. Preview the patch. + + ```bash + scaffold update --dry-run + ``` + +4. Apply it. + + ```bash + scaffold update + ``` + + It exits non-zero when a hunk was rejected, and still records the new version: running it again would offer the applied hunks twice. + +5. Resolve rejected hunks. Each `.rej` beside a file holds what did not apply. Place those hunks by hand, then delete the `.rej`. + + ```bash + git -C ls-files --others --exclude-standard -- '*.rej' + ``` + +6. Review and commit. + + ```bash + git -C diff + (cd && mise run checklist) + git -C add -A + git -C commit -m "chore: update from scaffold" + ``` + +## Verify + +- `scaffold update --dry-run` prints `already up to date with `. +- `version` in `/.scaffold.toml` equals `scaffold --version`. +- No `.rej` file remains. + +## If it fails + +| Symptom | Fix | +| --- | --- | +| `no .scaffold.toml in ` | The project predates the file. Write it with the toolbox commit it came from and each app's adapter, as the message shows, then run again | +| `records -dirty` | The project was generated from uncommitted toolbox edits. There is no commit to diff from, so it cannot be updated (ADR-0023) | +| `this toolbox has no commit ` | A different clone generated the project, or history was rewritten. Run `scaffold` from a clone that has the commit | +| `has uncommitted changes` | Commit or stash, then run again | +| ` was generated by '', which this toolbox no longer has` | That app is skipped; the rest applies | +| Many `.rej` files in compose and workflow files | Normal for a project that predates ADR-0022: an update moves files, not structure | +| The project's `roots:` in `ci.yml` or `images:` in `build.yml` look wrong | They are re-derived after the patch (`resync_derived_files` in `lib/update.sh`). Check `config_roots` in the project's `mise.toml` |