Skip to content

Publish the full DocumentDB package set and prove the repository installs - #149

Merged
guanzhousongmicrosoft merged 2 commits into
documentdb:mainfrom
GuanzhouSong:fix/package-repo-multipackage-and-smoke-test
Aug 24, 2026
Merged

Publish the full DocumentDB package set and prove the repository installs#149
guanzhousongmicrosoft merged 2 commits into
documentdb:mainfrom
GuanzhouSong:fix/package-repo-multipackage-and-smoke-test

Conversation

@GuanzhouSong

Copy link
Copy Markdown
Contributor

Problem

documentdb/documentdb v0.116-0 replaced the single postgresql-N-documentdb extension package with a multi-package layout. This repository's mirror was never taught about it, so publishing v0.116-0 as-is would have shipped a broken package repository.

1. 14 of 22 packages would have been silently dropped. download_packages.sh matched only postgresql-N-documentdb. Running its own regexes against the real v0.116-0 asset list: 8 matched, 14 dropped — including documentdb (the meta package) and everything it depends on. apt install documentdb could never have resolved. The RPM side was worse: the new RPMs carry no rhel9- prefix, so none of them matched.

2. Five distributions would have disappeared. The job rebuilds the repository from the latest release and deploys out/ wholesale. v0.114-0 shipped 7 distributions; v0.116-0 ships Tier-1 (ubuntu24 + rhel9) only. The deb11, deb12, deb13 and ubuntu22 components and the entire rhel8 repository would have vanished, and every host already pointed at one would start failing:

E: Component 'ubuntu22' is not defined for 'stable'

That is a live outage for existing users, not a cosmetic regression.

3. Nothing would have caught either problem. The existing verifier checks that metadata files exist. In both failure modes every file it looks for is present and correct — the repository is simply unsatisfiable.

Changes

download_packages.sh — publish every package, additively

  • Match any DocumentDB package for a distribution, not just the extension, and route the new RPM shapes: distro-prefixed (rhel9-…), dist-tagged (….el9.x86_64.rpm), and the EL-agnostic noarch meta/standalone/common/tools packages, which carry no distro hint and are placed only in pools that same release populated.
  • Fill the pool additively, keyed on (pool, package name, arch): the newest release wins, and older releases contribute only packages no newer release provides.

The additive rule is what stops a narrowed release from deleting things. It also preserves postgresql-16-documentdb on ubuntu24/rhel9, which v0.116-0 dropped when Tier 1 narrowed to PostgreSQL 17/18 — the install docs still tell people to use it. Self-healing: once a release ships every distribution again, older releases stop contributing on their own.

continuous-deployment.yml — prove the repository installs

Resolve documentdb against the repository about to be published, for both APT and DNF. This is dependency resolution only (apt-get -s, dnf --assumeno), so it reads the indexes and solves the graph without downloading ~500 MB of PostgreSQL and PostGIS — a few seconds, while catching the entire "package missing / unsatisfiable dependency" class. It also asserts the transaction really pulls documentdb-common, -gateway, -postgresql-tools and an extension, so a meta package that degenerated into an empty shell cannot pass.

PACKAGE-INSTALL.md — rewrite for the new reality

The page still said the repository "does not currently publish a gateway package, setup helper, or systemd service", which is now false for ubuntu24/rhel9.

  • Document the two-tier reality: full stack on Ubuntu 24.04 / RHEL 9, extension-only elsewhere.
  • Quickstart is now apt install documentdbdocumentdb-setupmongosh, and states that the wizard prompts for the admin password (with the non-interactive flags for servers/CI).
  • Security: the gateway listens on 0.0.0.0:10260 and [::]:10260 by default while the connect example says 127.0.0.1, so the page implied localhost-only. Now states it plainly and shows how to restrict the listener and supply a real certificate.
  • Verify and operate: documentdb-setup --status, how to check the version — with a warning that mongosh buildInfo reports the emulated MongoDB version (7.0.0), not DocumentDB's — plus ports, log paths and config locations.
  • Day 2: corrected to the real per-major unit names (documentdb-local@18.target); no plain documentdb-local.target is shipped by any package.
  • Upgrading: a package upgrade only replaces files; ALTER EXTENSION … UPDATE is required per database. Includes the pre-GA caveat and pinning commands.

Verification

Everything below was run against the real v0.116-0 release assets, using the repository this branch generates.

Check Before After
ubuntu24 pool 6 debs, extension only 13 debs, full stack
rhel9 pool 4 rpms 13 rpms, full stack
PG majors on ubuntu24 / rhel9 17, 18 16, 17, 18
deb11/deb12/deb13/ubuntu22/rhel8 unchanged, nothing disappears
apt install documentdb unsatisfiable exit 0 — 6 packages + documentdb-setup, -tune, -gateway
dnf install documentdb unsatisfiable exit 0 — 6 packages + tools
deb12 regression still serves 0.114-0

The repository was also built signed with a throwaway key and served over HTTP, so the keyring, InRelease and Release.gpg paths were exercised end to end.

The smoke test was verified to fail, not just to pass. Reproducing the original bug — an extension-only ubuntu24 pool with a regenerated, internally consistent Release file, so the only defect was the missing packages — the guard trips with exactly the shipping failure:

E: Unable to locate package documentdb

Notes for reviewers

  • MAX_RELEASES (default 8) bounds how far back the additive fill looks; it stops early once every distribution is covered.
  • release-info.json still describes the primary release only — it is the site's "current version" feed, not an inventory of the pool.
  • The smoke test uses [trusted=yes] / gpgcheck=0 against the local mirror because the deployment may run without a signing key. Signature presence is asserted separately by the existing verifier step.
  • No change to the deploy job, the Pages artifact, or the static site build.

v0.116-0 replaced the single `postgresql-N-documentdb` extension package with a
multi-package layout (`documentdb` meta, `documentdb-N`, `documentdb-common`,
`documentdb-gateway`, `documentdb-postgresql-tools`). The mirror's asset filter
still matched only the extension, so 14 of the release's 22 packages were
dropped: `apt install documentdb` could never have resolved, and none of the new
RPMs matched at all because they carry no `rhel9-` prefix.

Rebuilding from the newest release alone would also have deleted the
deb11/deb12/deb13/ubuntu22 components and the whole rhel8 repository, because
v0.116-0 ships Tier-1 (ubuntu24 + rhel9) only. Every host already pointed at one
of those would have started failing `apt update` with "Component 'ubuntu22' is
not defined".

So the pool is now filled additively, per (pool, package name, arch): the newest
release wins, and older releases contribute only packages no newer release
provides. Nothing disappears when a release narrows its matrix, and
`postgresql-16-documentdb` survives on ubuntu24/rhel9 even though v0.116-0
narrowed Tier 1 to PostgreSQL 17/18. Once a release ships every distribution
again, the older ones stop contributing on their own.

The metadata verifier could not have caught any of this: every file it looks for
was present and correct, the repository was simply unsatisfiable. Add a smoke
test that resolves `documentdb` against the repository about to be published,
for both APT and DNF. It solves the dependency graph without downloading ~500 MB
of PostgreSQL and PostGIS, and asserts the transaction really pulls the stack,
so an empty meta package cannot pass either.

Rewrite PACKAGE-INSTALL.md for the two-tier reality, and document what a
first-time install actually needs: that the gateway listens on all interfaces by
default and how to restrict it, how to verify an install and read its version
(mongosh reports the emulated MongoDB version, not DocumentDB's), where the
ports, logs and config live, the per-major systemd unit names, and how to
upgrade, reset or remove.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 8df9084a-ccaf-432c-b015-2ccd8893a9d8
The site pins the mirrored release with the DOCUMENTDB_VERSION repository
variable (currently v0.113-0). The additive fill walked every other published
release regardless of age, so a pin to a release older than the v0.116-0
multi-package layout would have added a newer `documentdb` meta package on top
of the pinned extension. Its `documentdb-N (>= 0.116.0)` dependency cannot be
satisfied by `postgresql-N-documentdb 0.113-0`, so the published repository
would have been unsatisfiable -- and the pin would have been quietly defeated.

A pin means "serve this version", so only the pinned release and older ones may
contribute.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 8df9084a-ccaf-432c-b015-2ccd8893a9d8
@guanzhousongmicrosoft
guanzhousongmicrosoft merged commit fcbed7b into documentdb:main Aug 24, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants