diff --git a/.claude/agents/code-architect.md b/.claude/agents/code-architect.md index 58c438766..bdbe7a52d 100644 --- a/.claude/agents/code-architect.md +++ b/.claude/agents/code-architect.md @@ -8,12 +8,14 @@ Analyze the codebase architecture and suggest improvements when asked. ## Context -This is a Rust workspace targeting Fastly Compute (`wasm32-wasip1`) with three -crates: `common` (core logic), `fastly` (entry point), and `js` (TS/JS build). +This is a multi-adapter Rust workspace. Shared logic lives in +`trusted-server-core`; Fastly, Cloudflare, Spin, and Axum adapters provide +runtime-specific entry points, and the JS crate builds TSJS assets. Key patterns: -- **RequestWrapper trait** abstracts HTTP handling for different backends +- **RuntimeServices and platform traits** supply stores, outbound HTTP, + backends, geo lookup, and template services to shared request handlers - **Settings-driven config** via `trusted-server.toml` - **Integration system** with Rust registration + per-integration JS bundles - **Runtime JS concatenation** — server assembles core + integration scripts @@ -21,7 +23,8 @@ Key patterns: ## When Analyzing 1. Read relevant source files before making suggestions. -2. Consider WASM constraints (no filesystem, no threads, no Tokio). +2. Apply the constraints of the adapter being reviewed. Core and WASM adapter + code cannot assume Tokio or host filesystem access; Axum is native. 3. Respect existing patterns — suggest improvements that fit the current architecture. 4. Prioritize simplicity and correctness over cleverness. diff --git a/.claude/agents/issue-creator.md b/.claude/agents/issue-creator.md index f3f1d4052..cab2608e5 100644 --- a/.claude/agents/issue-creator.md +++ b/.claude/agents/issue-creator.md @@ -82,7 +82,7 @@ Output the issue URL and type. - Core (Edge Cookies, GDPR) - Integrations (prebid, lockr, permutive, etc.) - HTML processing / JS injection - - Ad serving (Equativ) + - Ad serving / auctions - Fastly runtime - JS build pipeline - Documentation diff --git a/.claude/commands/check-ci.md b/.claude/commands/check-ci.md index ba785ccca..908b4276c 100644 --- a/.claude/commands/check-ci.md +++ b/.claude/commands/check-ci.md @@ -1,11 +1,5 @@ -Run all CI checks locally, in order. Stop and report if any step fails. +Run every check in the [canonical CI gate list](/CLAUDE.md#ci-gates), in its +documented order. Stop and report if any gate fails. -1. `cargo fmt --all -- --check` -2. `cargo clippy-fastly && cargo clippy-axum && cargo clippy-cloudflare` -3. `cargo test-fastly && cargo test-axum && cargo test-cloudflare` -4. `cargo check-cloudflare` (wasm32-unknown-unknown target check, mirrors CI) -5. `cd crates/trusted-server-js/lib && npx vitest run` -6. `cd crates/trusted-server-js/lib && npm run format` -7. `cd docs && npm run format` - -Report a summary of all results when done. +Report the result of each canonical gate. Do not maintain a separate gate list +in this command. diff --git a/.claude/commands/review-changes.md b/.claude/commands/review-changes.md index 8c9546056..cac6a356b 100644 --- a/.claude/commands/review-changes.md +++ b/.claude/commands/review-changes.md @@ -1,5 +1,8 @@ Review all staged and unstaged changes in the working tree. +Use the [canonical CI gate list](/CLAUDE.md#ci-gates) as the sole source for +required verification. Do not copy its commands into this file. + 1. Run `git diff` and `git diff --cached` to see all changes. 2. Review each changed file for: - Correctness and logic errors diff --git a/.claude/commands/test-all.md b/.claude/commands/test-all.md index 3f2cabec3..b71c466b1 100644 --- a/.claude/commands/test-all.md +++ b/.claude/commands/test-all.md @@ -1,13 +1,6 @@ -Run the full test suite for both Rust and JavaScript. +Run the Rust and JavaScript test gates from the +[canonical CI gate list](/CLAUDE.md#ci-gates). Do not maintain a separate test +command list in this file. -```bash -cargo test-fastly && cargo test-axum && cargo test-cloudflare -``` - -Then run JS tests: - -```bash -cd crates/trusted-server-js/lib && npx vitest run -``` - -Report results for both. If any test fails, investigate and suggest a fix. +Report each test-gate result. If a test fails, investigate it before reporting +the command complete. diff --git a/.claude/commands/test-crate.md b/.claude/commands/test-crate.md index f7c37db0c..85626f103 100644 --- a/.claude/commands/test-crate.md +++ b/.claude/commands/test-crate.md @@ -2,16 +2,10 @@ Test a specific crate by name. Usage: /test-crate $ARGUMENTS -Run: +Use the target-aware test rules referenced by the +[canonical CI gate list](/CLAUDE.md#ci-gates). Select the canonical adapter or +JavaScript gate that covers `$ARGUMENTS`; do not substitute a bare workspace +test. For a host-only crate without a covering alias, follow the host-target +procedure in `CLAUDE.md`. -```bash -cargo test -p $ARGUMENTS -``` - -If $ARGUMENTS is "js" or "javascript", run: - -```bash -cd crates/trusted-server-js/lib && npx vitest run -``` - -Report results and investigate any failures. +Report the selected canonical rule and its result. Investigate any failure. diff --git a/.claude/commands/verify.md b/.claude/commands/verify.md index 7a8454891..87275d88a 100644 --- a/.claude/commands/verify.md +++ b/.claude/commands/verify.md @@ -1,12 +1,5 @@ -Full verification: build, test, and lint the entire project. +Run full repository verification from the +[canonical CI gate list](/CLAUDE.md#ci-gates). That region is the sole source +for required gate commands; do not maintain a copy here. -1. `cargo build-fastly && cargo build-axum && cargo build-cloudflare` -2. `cargo build --package trusted-server-adapter-fastly --release --target wasm32-wasip1` -3. `cargo fmt --all -- --check` -4. `cargo clippy-fastly && cargo clippy-axum && cargo clippy-cloudflare` -5. `cargo test-fastly && cargo test-axum && cargo test-cloudflare` -6. `cd crates/trusted-server-js/lib && npx vitest run` -7. `cd crates/trusted-server-js/lib && npm run format` -8. `cd docs && npm run format` - -Report results for each step. Stop and investigate if any step fails. +Report every canonical gate result. Stop and investigate if any gate fails. diff --git a/.env.dev b/.env.dev index fd7aa3ba4..6f926e1ab 100644 --- a/.env.dev +++ b/.env.dev @@ -5,9 +5,11 @@ # [publisher] TRUSTED_SERVER__PUBLISHER__ORIGIN_URL=http://localhost:9090 -# [synthetic] -TRUSTED_SERVER__SYNTHETIC__COUNTER_STORE=counter_store -TRUSTED_SERVER__SYNTHETIC__OPID_STORE=opid_store +# [ec] +# The passphrase is secret-store material, not a development overlay. This +# non-secret threshold is applied only when the file is exported before a +# `ts config` command. +TRUSTED_SERVER__EC__CLUSTER_TRUST_THRESHOLD=10 # [proxy] # Disable TLS certificate verification for local dev with self-signed certs diff --git a/.env.example b/.env.example index a7f5973cd..4120e3b60 100644 --- a/.env.example +++ b/.env.example @@ -18,27 +18,26 @@ TRUSTED_SERVER__PUBLISHER__COOKIE_DOMAIN=.publisher.com TRUSTED_SERVER__PUBLISHER__ORIGIN_URL=https://origin.publisher.com # ============================================================================= -# Synthetic ID Settings +# Edge Cookie Settings # ============================================================================= -TRUSTED_SERVER__SYNTHETIC__COUNTER_STORE=counter_store -TRUSTED_SERVER__SYNTHETIC__OPID_STORE=opid_store -# Template variables: client_ip, user_agent, first_party_id, auth_user_id, publisher_domain, accept_language -TRUSTED_SERVER__SYNTHETIC__TEMPLATE={{ client_ip }}:{{ user_agent }}:{{ first_party_id }} +# Non-secret CLI overlay. Keep the EC passphrase in the app-config secret +# store; do not place it in this file. +TRUSTED_SERVER__EC__CLUSTER_TRUST_THRESHOLD=10 # ============================================================================= -# Request Signing (optional) +# Request Signing (optional CLI overlay) # ============================================================================= TRUSTED_SERVER__REQUEST_SIGNING__ENABLED=false # TRUSTED_SERVER__REQUEST_SIGNING__CONFIG_STORE_ID= # TRUSTED_SERVER__REQUEST_SIGNING__SECRET_STORE_ID= # ============================================================================= -# Response Headers (optional) +# Response Headers (optional CLI overlay) # ============================================================================= # TRUSTED_SERVER__RESPONSE_HEADERS__X_CUSTOM_HEADER=custom-value # ============================================================================= -# Integrations +# Integrations (optional CLI overlays) # ============================================================================= # Prebid diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 000000000..1e362ac10 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,2 @@ +# Authenticated CLI-help goldens preserve the runners' exact output bytes. +tools/docs-parity/goldens/*.txt -whitespace diff --git a/.github/actions/setup-integration-test-env/action.yml b/.github/actions/setup-integration-test-env/action.yml index 841d8d5bd..3ff8d4971 100644 --- a/.github/actions/setup-integration-test-env/action.yml +++ b/.github/actions/setup-integration-test-env/action.yml @@ -44,6 +44,11 @@ runs: shell: bash run: echo "node-version=$(grep '^nodejs ' .tool-versions | awk '{print $2}')" >> "$GITHUB_OUTPUT" + - name: Set up Node.js + uses: actions/setup-node@v7.0.0 + with: + node-version: ${{ steps.node-version.outputs.node-version }} + - name: Retrieve Viceroy version id: viceroy-version if: ${{ inputs.install-viceroy == 'true' }} @@ -52,7 +57,7 @@ runs: run: echo "viceroy-version=$(grep '^viceroy ' .tool-versions | awk '{print $2}')" >> "$GITHUB_OUTPUT" - name: Set up Rust toolchain - uses: actions-rust-lang/setup-rust-toolchain@v1 + uses: actions-rust-lang/setup-rust-toolchain@v1.17.0 with: toolchain: ${{ steps.rust-version.outputs.rust-version }} target: wasm32-wasip1 @@ -61,7 +66,7 @@ runs: - name: Cache Viceroy binary if: ${{ inputs.install-viceroy == 'true' }} id: cache-viceroy - uses: actions/cache@v4 + uses: actions/cache@v6.1.0 with: path: ~/.cargo/bin/viceroy key: viceroy-${{ runner.os }}-v${{ steps.viceroy-version.outputs.viceroy-version }} diff --git a/.github/dependabot.yml b/.github/dependabot.yml index 9c5c2c93b..d8999602b 100644 --- a/.github/dependabot.yml +++ b/.github/dependabot.yml @@ -1,21 +1,43 @@ -# To get started with Dependabot version updates, you'll need to specify which -# package ecosystems to update and where the package manifests are located. -# Please see the documentation for all configuration options: -# https://docs.github.com/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file - version: 2 updates: - - package-ecosystem: "cargo" # See documentation for possible values - directory: "/" # Location of package manifests + - package-ecosystem: "github-actions" + directory: "/" + schedule: + interval: "weekly" + target-branch: "main" + + - package-ecosystem: "cargo" + directory: "/" + schedule: + interval: "weekly" + target-branch: "main" + + - package-ecosystem: "cargo" + directory: "/tools/docs-parity" + schedule: + interval: "weekly" + target-branch: "main" + + - package-ecosystem: "npm" + directory: "/crates/trusted-server-js/lib" + schedule: + interval: "weekly" + target-branch: "main" + + - package-ecosystem: "npm" + directory: "/crates/trusted-server-integration-tests/browser" schedule: interval: "weekly" + target-branch: "main" - - package-ecosystem: "npm" # See documentation for possible values - directory: "crates/trusted-server-js/lib/" # Location of package manifests + - package-ecosystem: "npm" + directory: "/crates/trusted-server-integration-tests/fixtures/frameworks/nextjs" schedule: interval: "weekly" + target-branch: "main" - - package-ecosystem: "npm" # See documentation for possible values - directory: "docs/" # Location of package manifests + - package-ecosystem: "npm" + directory: "/docs" schedule: interval: "weekly" + target-branch: "main" diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md index 5f4cc6ca9..34db797ac 100644 --- a/.github/pull_request_template.md +++ b/.github/pull_request_template.md @@ -21,22 +21,15 @@ Closes # ## Test plan - - -- [ ] `cargo test-fastly && cargo test-axum` -- [ ] `cargo clippy-fastly && cargo clippy-axum` -- [ ] `cargo fmt --all -- --check` -- [ ] JS tests: `cd crates/trusted-server-js/lib && npx vitest run` -- [ ] JS format: `cd crates/trusted-server-js/lib && npm run format` -- [ ] Docs format: `cd docs && npm run format` -- [ ] WASM build: `cargo build --package trusted-server-adapter-fastly --release --target wasm32-wasip1` -- [ ] Manual testing via `fastly compute serve` -- [ ] Other: + + +- [ ] Completed the [canonical CI gate list](/CLAUDE.md#ci-gates) +- Evidence: ## Checklist - [ ] Changes follow [CLAUDE.md](/CLAUDE.md) conventions - [ ] No `unwrap()` in production code — use `expect("should ...")` -- [ ] Uses `tracing` macros (not `println!`) +- [ ] Uses `log` macros (not `println!`) - [ ] New code has tests - [ ] No secrets or credentials committed diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml index 4b963e015..bc87c3b24 100644 --- a/.github/workflows/codeql.yml +++ b/.github/workflows/codeql.yml @@ -15,7 +15,7 @@ on: push: branches: ["main"] pull_request: - branches: ["main"] + branches: ["main", "rc/*"] schedule: - cron: "42 6 * * 0" @@ -59,7 +59,7 @@ jobs: # your codebase is analyzed, see https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages steps: - name: Checkout repository - uses: actions/checkout@v4 + uses: actions/checkout@v7.0.1 # Add any setup steps before running the `github/codeql-action/init` action. # This includes steps like installing compilers or runtimes (`actions/setup-node` @@ -69,7 +69,7 @@ jobs: # Initializes the CodeQL tools for scanning. - name: Initialize CodeQL - uses: github/codeql-action/init@v4 + uses: github/codeql-action/init@v4.37.9 with: languages: ${{ matrix.language }} build-mode: ${{ matrix.build-mode }} @@ -98,6 +98,6 @@ jobs: exit 1 - name: Perform CodeQL Analysis - uses: github/codeql-action/analyze@v4 + uses: github/codeql-action/analyze@v4.37.9 with: category: "/language:${{matrix.language}}" diff --git a/.github/workflows/deploy-docs.yml b/.github/workflows/deploy-docs.yml index 3fa0da9b6..52da1d1e7 100644 --- a/.github/workflows/deploy-docs.yml +++ b/.github/workflows/deploy-docs.yml @@ -5,6 +5,7 @@ on: branches: [main] paths: - "docs/**" + - ".tool-versions" - ".github/workflows/deploy-docs.yml" workflow_dispatch: # Allow manual triggers @@ -24,23 +25,23 @@ jobs: runs-on: ubuntu-latest steps: - name: Checkout - uses: actions/checkout@v4 + uses: actions/checkout@v7.0.1 with: fetch-depth: 0 # For lastUpdated feature - name: Retrieve Node.js version id: node-version - run: echo "node-version=$(grep '^nodejs ' .tool-versions | awk '{print $2}')" >> $GITHUB_OUTPUT + run: echo "node-version=$(grep '^nodejs ' .tool-versions | awk '{print $2}')" >> "$GITHUB_OUTPUT" - name: Setup Node.js - uses: actions/setup-node@v4 + uses: actions/setup-node@v7.0.0 with: node-version: ${{ steps.node-version.outputs.node-version }} cache: "npm" cache-dependency-path: docs/package-lock.json - name: Setup Pages - uses: actions/configure-pages@v4 + uses: actions/configure-pages@v6.0.0 - name: Install dependencies working-directory: docs @@ -48,10 +49,18 @@ jobs: - name: Build with VitePress working-directory: docs + env: + GITHUB_SHA: ${{ github.sha }} run: npm run build + - name: Assert exact source provenance + working-directory: docs + env: + GITHUB_SHA: ${{ github.sha }} + run: grep -R --fixed-strings --quiet "$GITHUB_SHA" .vitepress/dist + - name: Upload artifact - uses: actions/upload-pages-artifact@v3 + uses: actions/upload-pages-artifact@v5.0.0 with: path: docs/.vitepress/dist @@ -64,4 +73,4 @@ jobs: steps: - name: Deploy to GitHub Pages id: deployment - uses: actions/deploy-pages@v4 + uses: actions/deploy-pages@v5.0.1 diff --git a/.github/workflows/docs-links.yml b/.github/workflows/docs-links.yml new file mode 100644 index 000000000..3d31550f8 --- /dev/null +++ b/.github/workflows/docs-links.yml @@ -0,0 +1,167 @@ +name: Documentation automation +permissions: {} +on: + pull_request: + schedule: + - cron: "17 9 * * 1" + workflow_dispatch: +concurrency: + group: "documentation-automation-${{ github.event_name == 'pull_request' && format('pr-{0}', github.event.pull_request.number) || 'default-branch-writers' }}" + cancel-in-progress: "${{ github.event_name == 'pull_request' }}" +jobs: + pull-request: + name: Documentation parity + if: github.event_name == 'pull_request' + runs-on: ubuntu-latest + timeout-minutes: 30 + permissions: + contents: read + steps: + - uses: actions/checkout@v7.0.1 + with: + ref: "${{ github.event.pull_request.head.sha }}" + persist-credentials: false + fetch-depth: 0 + - name: Read pinned Node version + id: node-version + run: echo "node=$(awk '$1 == \"nodejs\" { print $2 }' .tool-versions)" >> "$GITHUB_OUTPUT" + - uses: actions-rust-lang/setup-rust-toolchain@v1.17.0 + with: + toolchain: "1.95.0" + cache: false + - uses: actions/setup-node@v7.0.0 + with: + node-version: "${{ steps.node-version.outputs.node }}" + - name: Install JSDoc lint dependencies + working-directory: crates/trusted-server-js/lib + run: npm ci + - run: cargo run --manifest-path tools/docs-parity/Cargo.toml -- check --all + link-reader: + name: Check external documentation links + if: github.repository == 'IABTechLab/trusted-server' && github.ref == 'refs/heads/main' && (github.event_name == 'schedule' || github.event_name == 'workflow_dispatch') + runs-on: ubuntu-latest + timeout-minutes: 30 + permissions: + contents: read + outputs: + artifact-sha256: "${{ steps.digest.outputs.sha256 }}" + checked-at: "${{ steps.link-result.outputs.checked-at }}" + steps: + - uses: actions/checkout@v7.0.1 + with: + ref: "${{ github.sha }}" + persist-credentials: false + fetch-depth: 0 + - uses: actions-rust-lang/setup-rust-toolchain@v1.17.0 + with: + toolchain: "1.95.0" + cache: false + - name: Generate closed link result + id: link-result + run: ./scripts/generate-docs-link-results.sh + - name: Record link artifact digest + id: digest + run: echo "sha256=$(sha256sum "$RUNNER_TEMP/link-results.zip" | cut -d ' ' -f 1)" >> "$GITHUB_OUTPUT" + - uses: actions/upload-artifact@v7.0.1 + with: + name: link-results + path: "${{ runner.temp }}/link-results.zip" + if-no-files-found: error + retention-days: 7 + compression-level: 0 + issue-writer: + name: Reconcile external-link issue + if: github.repository == 'IABTechLab/trusted-server' && github.ref == 'refs/heads/main' && (github.event_name == 'schedule' || github.event_name == 'workflow_dispatch') + needs: link-reader + runs-on: ubuntu-latest + timeout-minutes: 5 + permissions: + contents: read + issues: write + steps: + - uses: actions/checkout@v7.0.1 + with: + ref: "${{ github.sha }}" + persist-credentials: false + fetch-depth: 0 + - uses: actions-rust-lang/setup-rust-toolchain@v1.17.0 + with: + toolchain: "1.95.0" + cache: false + - uses: actions/download-artifact@v8.0.1 + with: + name: link-results + path: "${{ runner.temp }}/link-results" + - name: Validate LinkResultsV1 and reconcile one owned issue + env: + DOCS_PARITY_CHECKED_AT: "${{ needs.link-reader.outputs.checked-at }}" + EXPECTED_SHA256: "${{ needs.link-reader.outputs.artifact-sha256 }}" + EXPECTED_REPOSITORY: "${{ github.repository }}" + EXPECTED_REF: "${{ github.ref }}" + EXPECTED_SOURCE_SHA: "${{ github.sha }}" + EXPECTED_RUN_ID: "${{ github.run_id }}" + EXPECTED_RUN_ATTEMPT: "${{ github.run_attempt }}" + GH_TOKEN: "${{ github.token }}" + run: ./scripts/docs-links-reconcile.sh + dependency-reader: + name: Generate dependency snapshot + if: github.repository == 'IABTechLab/trusted-server' && github.ref == 'refs/heads/main' && (github.event_name == 'schedule' || github.event_name == 'workflow_dispatch') + runs-on: ubuntu-latest + timeout-minutes: 20 + permissions: + contents: read + outputs: + artifact-sha256: "${{ steps.digest.outputs.sha256 }}" + steps: + - uses: actions/checkout@v7.0.1 + with: + ref: "${{ github.sha }}" + persist-credentials: false + fetch-depth: 0 + - uses: actions-rust-lang/setup-rust-toolchain@v1.17.0 + with: + toolchain: "1.95.0" + cache: false + - name: Generate closed dependency snapshot + run: cargo run --manifest-path tools/docs-parity/Cargo.toml -- dependency-snapshot generate --output "$RUNNER_TEMP/dependency-snapshot.zip" + - name: Record dependency artifact digest + id: digest + run: echo "sha256=$(sha256sum "$RUNNER_TEMP/dependency-snapshot.zip" | cut -d ' ' -f 1)" >> "$GITHUB_OUTPUT" + - uses: actions/upload-artifact@v7.0.1 + with: + name: dependency-snapshot + path: "${{ runner.temp }}/dependency-snapshot.zip" + if-no-files-found: error + retention-days: 7 + compression-level: 0 + dependency-writer: + name: Submit dependency snapshot + if: github.repository == 'IABTechLab/trusted-server' && github.ref == 'refs/heads/main' && (github.event_name == 'schedule' || github.event_name == 'workflow_dispatch') + needs: dependency-reader + runs-on: ubuntu-latest + timeout-minutes: 5 + permissions: + contents: write + steps: + - uses: actions/checkout@v7.0.1 + with: + ref: "${{ github.sha }}" + persist-credentials: false + fetch-depth: 0 + - uses: actions-rust-lang/setup-rust-toolchain@v1.17.0 + with: + toolchain: "1.95.0" + cache: false + - uses: actions/download-artifact@v8.0.1 + with: + name: dependency-snapshot + path: "${{ runner.temp }}/dependency-snapshot" + - name: Validate and submit unchanged version-0 snapshot + env: + EXPECTED_SHA256: "${{ needs.dependency-reader.outputs.artifact-sha256 }}" + EXPECTED_REF: "${{ github.ref }}" + EXPECTED_SOURCE_SHA: "${{ github.sha }}" + EXPECTED_RUN_ID: "${{ github.run_id }}" + EXPECTED_RUN_ATTEMPT: "${{ github.run_attempt }}" + GH_TOKEN: "${{ github.token }}" + run: ./scripts/dependency-snapshot-submit.sh diff --git a/.github/workflows/format.yml b/.github/workflows/format.yml index ecfc3cd39..f5b789f6e 100644 --- a/.github/workflows/format.yml +++ b/.github/workflows/format.yml @@ -13,23 +13,33 @@ jobs: name: cargo fmt runs-on: ubuntu-latest steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v7.0.1 - name: Retrieve Rust version id: rust-version - run: echo "rust-version=$(grep '^rust ' .tool-versions | awk '{print $2}')" >> $GITHUB_OUTPUT + run: echo "rust-version=$(grep '^rust ' .tool-versions | awk '{print $2}')" >> "$GITHUB_OUTPUT" + shell: bash + + - name: Retrieve Node.js version + id: node-version + run: echo "node-version=$(grep '^nodejs ' .tool-versions | awk '{print $2}')" >> "$GITHUB_OUTPUT" shell: bash - name: Set up rust toolchain - uses: actions-rust-lang/setup-rust-toolchain@v1 + uses: actions-rust-lang/setup-rust-toolchain@v1.17.0 with: toolchain: ${{ steps.rust-version.outputs.rust-version }} target: wasm32-wasip1,wasm32-unknown-unknown components: "clippy, rustfmt" cache-shared-key: cargo-${{ runner.os }} + - name: Use Node.js for Rust build scripts + uses: actions/setup-node@v7.0.0 + with: + node-version: ${{ steps.node-version.outputs.node-version }} + - name: Run cargo fmt - uses: actions-rust-lang/rustfmt@v1 + uses: actions-rust-lang/rustfmt@v1.1.2 - name: Run cargo clippy (Fastly — wasm32-wasip1) run: cargo clippy-fastly @@ -64,20 +74,20 @@ jobs: working-directory: crates/trusted-server-js/lib steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v7.0.1 - name: Retrieve Node.js version id: node-version working-directory: . - run: echo "node-version=$(grep '^nodejs ' .tool-versions | awk '{print $2}')" >> $GITHUB_OUTPUT + run: echo "node-version=$(grep '^nodejs ' .tool-versions | awk '{print $2}')" >> "$GITHUB_OUTPUT" shell: bash - name: Use Node.js - uses: actions/setup-node@v4 + uses: actions/setup-node@v7.0.0 with: node-version: ${{ steps.node-version.outputs.node-version }} cache: "npm" - cache-dependency-path: crates/trusted-server-js/lib/package.json + cache-dependency-path: crates/trusted-server-js/lib/package-lock.json - name: Install dependencies run: npm ci @@ -95,20 +105,20 @@ jobs: working-directory: docs steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v7.0.1 - name: Retrieve Node.js version id: node-version working-directory: . - run: echo "node-version=$(grep '^nodejs ' .tool-versions | awk '{print $2}')" >> $GITHUB_OUTPUT + run: echo "node-version=$(grep '^nodejs ' .tool-versions | awk '{print $2}')" >> "$GITHUB_OUTPUT" shell: bash - name: Use Node.js - uses: actions/setup-node@v4 + uses: actions/setup-node@v7.0.0 with: node-version: ${{ steps.node-version.outputs.node-version }} cache: "npm" - cache-dependency-path: docs/package.json + cache-dependency-path: docs/package-lock.json - name: Install dependencies run: npm ci @@ -121,3 +131,47 @@ jobs: - name: Build with VitePress (fails on dead links) run: npm run build + + documentation-parity: + name: Documentation parity + runs-on: ubuntu-latest + timeout-minutes: 30 + steps: + - uses: actions/checkout@v7.0.1 + with: + fetch-depth: 0 + + - name: Read tool versions + id: tool-versions + run: ./scripts/read-tool-versions.sh + + - uses: actions-rust-lang/setup-rust-toolchain@v1.17.0 + with: + toolchain: ${{ steps.tool-versions.outputs.rust }} + components: "clippy, rustfmt" + cache-workspaces: "tools/docs-parity -> tools/docs-parity/target" + + - uses: actions/setup-node@v7.0.0 + with: + node-version: ${{ steps.tool-versions.outputs.node }} + cache: "npm" + cache-dependency-path: crates/trusted-server-js/lib/package-lock.json + + - name: Install JSDoc lint dependencies + working-directory: crates/trusted-server-js/lib + run: npm ci + + - name: Check documentation parity tool formatting + run: cargo fmt --manifest-path tools/docs-parity/Cargo.toml -- --check + + - name: Lint documentation parity tool + run: cargo clippy --manifest-path tools/docs-parity/Cargo.toml --all-targets --all-features -- -D warnings + + - name: Test documentation parity tool + run: cargo test --manifest-path tools/docs-parity/Cargo.toml + + - name: Run every deterministic documentation check + run: cargo run --manifest-path tools/docs-parity/Cargo.toml -- check --all + + - name: Require generated documentation to be current + run: cargo run --manifest-path tools/docs-parity/Cargo.toml -- generate --check diff --git a/.github/workflows/integration-tests.yml b/.github/workflows/integration-tests.yml index 4973afe44..ece0272c1 100644 --- a/.github/workflows/integration-tests.yml +++ b/.github/workflows/integration-tests.yml @@ -24,7 +24,7 @@ jobs: runs-on: ubuntu-latest timeout-minutes: 30 steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v7.0.1 - name: Set up shared integration build environment uses: ./.github/actions/setup-integration-test-env @@ -49,7 +49,7 @@ jobs: test-wordpress:latest test-nextjs:latest - name: Upload integration test artifacts - uses: actions/upload-artifact@v4 + uses: actions/upload-artifact@v7.0.1 with: name: integration-test-artifacts path: ${{ env.ARTIFACTS_DIR }} @@ -61,7 +61,7 @@ jobs: runs-on: ubuntu-latest timeout-minutes: 20 steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v7.0.1 - name: Set up integration test runtime id: shared-setup @@ -74,7 +74,7 @@ jobs: build-test-images: "false" - name: Download integration test artifacts - uses: actions/download-artifact@v4 + uses: actions/download-artifact@v8.0.1 with: name: integration-test-artifacts path: ${{ env.ARTIFACTS_DIR }} @@ -83,20 +83,18 @@ jobs: run: chmod +x "$AXUM_ARTIFACT_PATH" - name: Restore Cloudflare Workers bundle - run: | - mkdir -p crates/trusted-server-adapter-cloudflare/build - cp -r "$CF_BUILD_ARTIFACT_PATH/." crates/trusted-server-adapter-cloudflare/build/ + run: ./scripts/restore-cloudflare-build.sh - name: Load integration test Docker images run: docker load --input "$DOCKER_ARTIFACT_PATH" - name: Set up Node.js - uses: actions/setup-node@v4 + uses: actions/setup-node@v7.0.0 with: node-version: ${{ steps.shared-setup.outputs.node-version }} - - name: Install wrangler - run: npm install -g wrangler + - name: Install pinned Wrangler + run: ./scripts/install-wrangler.sh - name: Run integration tests run: >- @@ -114,13 +112,78 @@ jobs: VICEROY_CONFIG_PATH: ${{ env.ARTIFACTS_DIR }}/configs/viceroy.toml RUST_LOG: info + adapter-first-success-smokes: + name: adapter first success (${{ matrix.adapter }}) + needs: prepare-artifacts + runs-on: ubuntu-latest + timeout-minutes: 20 + strategy: + fail-fast: false + matrix: + adapter: [axum, fastly, cloudflare] + steps: + - uses: actions/checkout@v7.0.1 + + - name: Set up smoke runtime + id: shared-setup + uses: ./.github/actions/setup-integration-test-env + with: + origin-port: ${{ env.ORIGIN_PORT }} + install-viceroy: "false" + build-wasm: "false" + build-axum: "false" + build-test-images: "false" + + - name: Download integration test artifacts + uses: actions/download-artifact@v8.0.1 + with: + name: integration-test-artifacts + path: ${{ env.ARTIFACTS_DIR }} + + - name: Prepare Axum artifact + if: matrix.adapter == 'axum' + run: chmod +x "$AXUM_ARTIFACT_PATH" + + - name: Restore Cloudflare Workers bundle + if: matrix.adapter == 'cloudflare' + run: ./scripts/restore-cloudflare-build.sh + + - name: Set up Node.js + if: matrix.adapter == 'cloudflare' + uses: actions/setup-node@v7.0.0 + with: + node-version: ${{ steps.shared-setup.outputs.node-version }} + + - name: Install pinned Wrangler + if: matrix.adapter == 'cloudflare' + run: ./scripts/install-wrangler.sh + + - name: Read pinned Fastly tool versions + if: matrix.adapter == 'fastly' + id: fastly-versions + run: ./scripts/read-tool-versions.sh + + - name: Install pinned Fastly CLI + if: matrix.adapter == 'fastly' + uses: fastly/compute-actions/setup@v14 + with: + cli_version: ${{ steps.fastly-versions.outputs.fastly }} + viceroy_version: ${{ steps.fastly-versions.outputs.viceroy }} + token: ${{ secrets.GITHUB_TOKEN }} + + - name: Run adapter first-success smoke + env: + WASM_BINARY_PATH: ${{ env.WASM_ARTIFACT_PATH }} + AXUM_BINARY_PATH: ${{ env.AXUM_ARTIFACT_PATH }} + run: ./scripts/smoke-${{ matrix.adapter }}.sh + integration-tests-fastly-ec: name: integration tests (Fastly EC lifecycle) needs: prepare-artifacts runs-on: ubuntu-latest timeout-minutes: 15 steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v7.0.1 - name: Set up integration test runtime id: shared-setup @@ -132,7 +195,7 @@ jobs: build-test-images: "false" - name: Download integration test artifacts - uses: actions/download-artifact@v4 + uses: actions/download-artifact@v8.0.1 with: name: integration-test-artifacts path: ${{ env.ARTIFACTS_DIR }} @@ -158,7 +221,7 @@ jobs: runs-on: ubuntu-latest timeout-minutes: 15 steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v7.0.1 - name: Set up browser test runtime id: shared-setup @@ -170,7 +233,7 @@ jobs: build-test-images: "false" - name: Download integration test artifacts - uses: actions/download-artifact@v4 + uses: actions/download-artifact@v8.0.1 with: name: integration-test-artifacts path: ${{ env.ARTIFACTS_DIR }} @@ -179,7 +242,7 @@ jobs: run: docker load --input "$DOCKER_ARTIFACT_PATH" - name: Set up Node.js - uses: actions/setup-node@v4 + uses: actions/setup-node@v7.0.0 with: node-version: ${{ steps.shared-setup.outputs.node-version }} cache: npm @@ -211,7 +274,7 @@ jobs: run: npx playwright test - name: Upload Playwright report (Next.js) - uses: actions/upload-artifact@v4 + uses: actions/upload-artifact@v7.0.1 if: always() with: name: playwright-report-nextjs @@ -230,7 +293,7 @@ jobs: run: npx playwright test - name: Upload Playwright report (WordPress) - uses: actions/upload-artifact@v4 + uses: actions/upload-artifact@v7.0.1 if: always() with: name: playwright-report-wordpress @@ -238,7 +301,7 @@ jobs: retention-days: 7 - name: Upload Playwright traces and screenshots - uses: actions/upload-artifact@v4 + uses: actions/upload-artifact@v7.0.1 if: failure() with: name: playwright-traces diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 1f1bbe27a..b4fbed0a0 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -9,32 +9,90 @@ on: pull_request: jobs: + cli-help-capture: + name: Capture CLI help (${{ matrix.platform }}) + if: github.event_name == 'pull_request' + runs-on: ${{ matrix.os }} + timeout-minutes: 30 + permissions: + contents: read + strategy: + fail-fast: false + matrix: + include: + - os: ubuntu-latest + platform: linux + - os: macos-latest + platform: macos + steps: + # actions/checkout v7.0.1 + - uses: actions/checkout@v7.0.1 + with: + ref: ${{ github.event.pull_request.head.sha }} + persist-credentials: false + fetch-depth: 0 + + - name: Assert exact pull-request head + run: test "$(git rev-parse HEAD)" = "${{ github.event.pull_request.head.sha }}" + + - name: Read tool versions + id: tool-versions + run: ./scripts/read-tool-versions.sh + + # actions-rust-lang/setup-rust-toolchain v1.17.0 + - uses: actions-rust-lang/setup-rust-toolchain@v1.17.0 + with: + toolchain: ${{ steps.tool-versions.outputs.rust }} + cache: false + + # actions/setup-node v7.0.0 + - uses: actions/setup-node@v7.0.0 + with: + node-version: ${{ steps.tool-versions.outputs.node }} + + - name: Build Trusted Server JavaScript + run: ./scripts/build-trusted-server-js.sh + + - name: Capture recursive native CLI help + env: + TSJS_SKIP_BUILD: "1" + run: cargo run --manifest-path tools/docs-parity/Cargo.toml -- cli-help capture --output "${{ runner.temp }}/cli-help-${{ matrix.platform }}.zip" + + # actions/upload-artifact v7.0.1 + - uses: actions/upload-artifact@v7.0.1 + with: + name: cli-help-${{ matrix.platform }} + path: ${{ runner.temp }}/cli-help-${{ matrix.platform }}.zip + if-no-files-found: error + retention-days: 7 + compression-level: 0 + test-rust: name: cargo test runs-on: ubuntu-latest timeout-minutes: 45 steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v7.0.1 - name: Retrieve Rust version id: rust-version - run: echo "rust-version=$(grep '^rust ' .tool-versions | awk '{print $2}')" >> $GITHUB_OUTPUT + run: echo "rust-version=$(grep '^rust ' .tool-versions | awk '{print $2}')" >> "$GITHUB_OUTPUT" shell: bash - name: Retrieve Viceroy version id: viceroy-version # `.tool-versions` is the single source of truth so this workflow and # `.github/actions/setup-integration-test-env/action.yml` can't drift. - run: echo "viceroy-version=$(grep '^viceroy ' .tool-versions | awk '{print $2}')" >> $GITHUB_OUTPUT + run: echo "viceroy-version=$(grep '^viceroy ' .tool-versions | awk '{print $2}')" >> "$GITHUB_OUTPUT" shell: bash - name: Retrieve Node.js version id: node-version - run: echo "node-version=$(grep '^nodejs ' .tool-versions | awk '{print $2}')" >> $GITHUB_OUTPUT + run: echo "node-version=$(grep '^nodejs ' .tool-versions | awk '{print $2}')" >> "$GITHUB_OUTPUT" shell: bash - name: Set up Rust toolchain - uses: actions-rust-lang/setup-rust-toolchain@v1 + uses: actions-rust-lang/setup-rust-toolchain@v1.17.0 with: toolchain: ${{ steps.rust-version.outputs.rust-version }} target: wasm32-wasip1 @@ -42,7 +100,7 @@ jobs: - name: Cache Viceroy binary id: cache-viceroy - uses: actions/cache@v4 + uses: actions/cache@v6.1.0 with: path: ~/.cargo/bin/viceroy key: viceroy-${{ runner.os }}-v${{ steps.viceroy-version.outputs.viceroy-version }} @@ -52,7 +110,7 @@ jobs: run: cargo install viceroy --version "${{ steps.viceroy-version.outputs.viceroy-version }}" --locked --force - name: Use Node.js for the served-seam contract - uses: actions/setup-node@v4 + uses: actions/setup-node@v7.0.0 with: node-version: ${{ steps.node-version.outputs.node-version }} @@ -69,15 +127,20 @@ jobs: name: cargo test (axum native) runs-on: ubuntu-latest steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v7.0.1 - name: Retrieve Rust version id: rust-version - run: echo "rust-version=$(grep '^rust ' .tool-versions | awk '{print $2}')" >> $GITHUB_OUTPUT + run: echo "rust-version=$(grep '^rust ' .tool-versions | awk '{print $2}')" >> "$GITHUB_OUTPUT" + shell: bash + + - name: Retrieve Node.js version + id: node-version + run: echo "node-version=$(grep '^nodejs ' .tool-versions | awk '{print $2}')" >> "$GITHUB_OUTPUT" shell: bash - name: Set up Rust toolchain - uses: actions-rust-lang/setup-rust-toolchain@v1 + uses: actions-rust-lang/setup-rust-toolchain@v1.17.0 with: toolchain: ${{ steps.rust-version.outputs.rust-version }} # wasm32-wasip1 is required by the "Verify Fastly WASM release @@ -85,6 +148,11 @@ jobs: target: wasm32-wasip1 cache-shared-key: cargo-${{ runner.os }} + - name: Use Node.js for Rust build scripts + uses: actions/setup-node@v7.0.0 + with: + node-version: ${{ steps.node-version.outputs.node-version }} + - name: Build Axum adapter run: cargo build -p trusted-server-adapter-axum @@ -95,14 +163,24 @@ jobs: # -- --test runs each benchmark as a regular test (no timing harness) so CI stays fast run: cargo bench -p trusted-server-core --bench html_processor_bench -- --test + # The separate macOS job covers macOS-only CLI code without duplicating + # these cross-platform browser fixtures. + - name: Set up Chrome for browser fixture tests + id: setup-chrome + uses: browser-actions/setup-chrome@v2.2.0 + - name: Run host-target CLI tests run: ./scripts/test-cli.sh + env: + CHROME: ${{ steps.setup-chrome.outputs.chrome-path }} + # Hosted Ubuntu disables the user namespace required by the downloaded + # browser's sandbox. Only the ignored fixture test build reads this. + TS_AUDIT_BROWSER_FIXTURE_NO_SANDBOX: "1" # The codegen crate is not a default member and no adapter alias builds it, # so its tests only run if invoked explicitly on the host target. - name: Run host-target OpenRTB codegen tests - run: | - cargo test --package trusted-server-openrtb-codegen --target "$(rustc -vV | sed -n 's/host: //p')" + run: "cargo test --package trusted-server-openrtb-codegen --target \"$(rustc -vV | sed -n 's/host: //p')\"" - name: Verify Fastly WASM release build run: cargo build --package trusted-server-adapter-fastly --release --target wasm32-wasip1 @@ -111,20 +189,30 @@ jobs: name: cargo check (cloudflare native + wasm32-unknown-unknown) runs-on: ubuntu-latest steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v7.0.1 - name: Retrieve Rust version id: rust-version - run: echo "rust-version=$(grep '^rust ' .tool-versions | awk '{print $2}')" >> $GITHUB_OUTPUT + run: echo "rust-version=$(grep '^rust ' .tool-versions | awk '{print $2}')" >> "$GITHUB_OUTPUT" + shell: bash + + - name: Retrieve Node.js version + id: node-version + run: echo "node-version=$(grep '^nodejs ' .tool-versions | awk '{print $2}')" >> "$GITHUB_OUTPUT" shell: bash - name: Set up Rust toolchain (native + wasm32-unknown-unknown) - uses: actions-rust-lang/setup-rust-toolchain@v1 + uses: actions-rust-lang/setup-rust-toolchain@v1.17.0 with: toolchain: ${{ steps.rust-version.outputs.rust-version }} target: wasm32-unknown-unknown cache-shared-key: cargo-${{ runner.os }} + - name: Use Node.js for Rust build scripts + uses: actions/setup-node@v7.0.0 + with: + node-version: ${{ steps.node-version.outputs.node-version }} + - name: Check Cloudflare adapter (native host) run: cargo check -p trusted-server-adapter-cloudflare @@ -138,20 +226,30 @@ jobs: name: cargo check/build/test (spin native + wasm32-wasip1) runs-on: ubuntu-latest steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v7.0.1 - name: Retrieve Rust version id: rust-version - run: echo "rust-version=$(grep '^rust ' .tool-versions | awk '{print $2}')" >> $GITHUB_OUTPUT + run: echo "rust-version=$(grep '^rust ' .tool-versions | awk '{print $2}')" >> "$GITHUB_OUTPUT" + shell: bash + + - name: Retrieve Node.js version + id: node-version + run: echo "node-version=$(grep '^nodejs ' .tool-versions | awk '{print $2}')" >> "$GITHUB_OUTPUT" shell: bash - name: Set up Rust toolchain (native + wasm32-wasip1) - uses: actions-rust-lang/setup-rust-toolchain@v1 + uses: actions-rust-lang/setup-rust-toolchain@v1.17.0 with: toolchain: ${{ steps.rust-version.outputs.rust-version }} target: wasm32-wasip1 cache-shared-key: cargo-${{ runner.os }} + - name: Use Node.js for Rust build scripts + uses: actions/setup-node@v7.0.0 + with: + node-version: ${{ steps.node-version.outputs.node-version }} + - name: Check Spin adapter (native host) run: cargo check -p trusted-server-adapter-spin @@ -159,15 +257,9 @@ jobs: run: cargo check-spin - name: Build Spin adapter release WASM - # Mirror the Fastly release-build overrides so the artifact embeds usable - # settings instead of the trusted-server.toml placeholders, which startup - # rejects (falling back to the generic 503 router). This proves the - # documented Spin artifact can boot, not just compile. - env: - TRUSTED_SERVER__PUBLISHER__ORIGIN_URL: http://127.0.0.1:8080 - TRUSTED_SERVER__PUBLISHER__PROXY_SECRET: integration-test-proxy-secret - TRUSTED_SERVER__EC__PASSPHRASE: integration-test-ec-secret-padded-32 - TRUSTED_SERVER__PROXY__CERTIFICATE_CHECK: "false" + # This verifies the release artifact compiles. Spin loads runtime + # settings from its configured KV store; boot behavior is covered by a + # separately provisioned smoke test. run: cargo build --package trusted-server-adapter-spin --target wasm32-wasip1 --features spin --release - name: Run Spin adapter tests (native host) @@ -177,20 +269,30 @@ jobs: name: cargo test (cross-adapter parity) runs-on: ubuntu-latest steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v7.0.1 - name: Retrieve Rust version id: rust-version - run: echo "rust-version=$(grep '^rust ' .tool-versions | awk '{print $2}')" >> $GITHUB_OUTPUT + run: echo "rust-version=$(grep '^rust ' .tool-versions | awk '{print $2}')" >> "$GITHUB_OUTPUT" + shell: bash + + - name: Retrieve Node.js version + id: node-version + run: echo "node-version=$(grep '^nodejs ' .tool-versions | awk '{print $2}')" >> "$GITHUB_OUTPUT" shell: bash - name: Set up Rust toolchain - uses: actions-rust-lang/setup-rust-toolchain@v1 + uses: actions-rust-lang/setup-rust-toolchain@v1.17.0 with: toolchain: ${{ steps.rust-version.outputs.rust-version }} components: clippy, rustfmt cache-shared-key: cargo-${{ runner.os }} + - name: Use Node.js for Rust build scripts + uses: actions/setup-node@v7.0.0 + with: + node-version: ${{ steps.node-version.outputs.node-version }} + - name: Format check (parity test crate) run: cargo fmt --manifest-path crates/trusted-server-integration-tests/Cargo.toml -- --check @@ -201,44 +303,45 @@ jobs: run: cargo clippy --manifest-path crates/trusted-server-integration-tests/Cargo.toml --all-targets -- -D warnings test-cli: - # `trusted-server-cli` is a workspace member, but the workspace default target - # is wasm32-wasip1 (see .cargo/config.toml), where the crate is an empty shell - # — so the wasm workspace jobs do not exercise its real code. `ts dev proxy` is - # a macOS tool (Safari/keychain/networksetup), so validate the crate on macOS - # with an explicit native --target that overrides the wasm default. + # Adapter jobs set their WASM targets explicitly and do not exercise the + # CLI's native code. `ts dev proxy` is macOS-only, so this job validates the + # CLI with the runner's explicit native target. name: cargo test (ts CLI, native) runs-on: macos-latest steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v7.0.1 - name: Retrieve Rust version id: rust-version - run: echo "rust-version=$(grep '^rust ' .tool-versions | awk '{print $2}')" >> $GITHUB_OUTPUT + run: echo "rust-version=$(grep '^rust ' .tool-versions | awk '{print $2}')" >> "$GITHUB_OUTPUT" + shell: bash + + - name: Retrieve Node.js version + id: node-version + run: echo "node-version=$(grep '^nodejs ' .tool-versions | awk '{print $2}')" >> "$GITHUB_OUTPUT" shell: bash - name: Set up Rust toolchain - uses: actions-rust-lang/setup-rust-toolchain@v1 + uses: actions-rust-lang/setup-rust-toolchain@v1.17.0 with: toolchain: ${{ steps.rust-version.outputs.rust-version }} components: "clippy, rustfmt" cache-shared-key: cargo-cli-${{ runner.os }} + - name: Use Node.js for Rust build scripts + uses: actions/setup-node@v7.0.0 + with: + node-version: ${{ steps.node-version.outputs.node-version }} + # No separate `cargo fmt` here: `trusted-server-cli` is a workspace member, # so the main `cargo fmt --all` job already formats it. Clippy still needs a # native run — the workspace clippy jobs are wasm/adapter-scoped and never # lint the CLI's host-only code (macOS dev proxy vs Linux audit differ by cfg). - name: cargo clippy - run: | - cargo clippy --manifest-path crates/trusted-server-cli/Cargo.toml --target "$(rustc -vV | sed -n 's/host: //p')" --all-targets -- -D warnings + run: "cargo clippy --manifest-path crates/trusted-server-cli/Cargo.toml --target \"$(rustc -vV | sed -n 's/host: //p')\" --all-targets -- -D warnings" - - name: Set up Chrome for browser fixture tests - id: setup-chrome - uses: browser-actions/setup-chrome@v1 - - - name: cargo test - run: ./scripts/test-cli.sh - env: - CHROME: ${{ steps.setup-chrome.outputs.chrome-path }} + - name: Run native CLI tests + run: "cargo test --package trusted-server-cli --target \"$(rustc -vV | sed -n 's/host: //p')\"" test-typescript: name: vitest @@ -247,20 +350,20 @@ jobs: run: working-directory: crates/trusted-server-js/lib steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@v7.0.1 - name: Retrieve Node.js version id: node-version working-directory: . - run: echo "node-version=$(grep '^nodejs ' .tool-versions | awk '{print $2}')" >> $GITHUB_OUTPUT + run: echo "node-version=$(grep '^nodejs ' .tool-versions | awk '{print $2}')" >> "$GITHUB_OUTPUT" shell: bash - name: Use Node.js - uses: actions/setup-node@v4 + uses: actions/setup-node@v7.0.0 with: node-version: ${{ steps.node-version.outputs.node-version }} cache: "npm" - cache-dependency-path: crates/trusted-server-js/lib/package.json + cache-dependency-path: crates/trusted-server-js/lib/package-lock.json - name: Install dependencies run: npm ci @@ -270,3 +373,50 @@ jobs: - name: Run unit tests run: npm test -- --run + + documentation-rustdoc: + name: Rust API documentation + runs-on: ubuntu-latest + timeout-minutes: 30 + steps: + - uses: actions/checkout@v7.0.1 + + - name: Read tool versions + id: tool-versions + run: ./scripts/read-tool-versions.sh + + - uses: actions-rust-lang/setup-rust-toolchain@v1.17.0 + with: + toolchain: ${{ steps.tool-versions.outputs.rust }} + target: wasm32-wasip1,wasm32-unknown-unknown + + - uses: actions/setup-node@v7.0.0 + with: + node-version: ${{ steps.tool-versions.outputs.node }} + cache: "npm" + cache-dependency-path: crates/trusted-server-js/lib/package-lock.json + + - name: Install JavaScript build dependencies + working-directory: crates/trusted-server-js/lib + run: npm ci + + - name: Document portable core crates + run: RUSTDOCFLAGS='-D warnings' cargo doc --no-deps --all-features -p trusted-server-core -p trusted-server-js -p trusted-server-openrtb --target wasm32-wasip1 + + - name: Document Fastly adapter + run: RUSTDOCFLAGS='-D warnings' cargo doc --no-deps -p trusted-server-adapter-fastly --target wasm32-wasip1 + + - name: Document Cloudflare adapter + run: RUSTDOCFLAGS='-D warnings' cargo doc --no-deps -p trusted-server-adapter-cloudflare --target wasm32-unknown-unknown --features cloudflare + + - name: Document Spin adapter + run: RUSTDOCFLAGS='-D warnings' cargo doc --no-deps -p trusted-server-adapter-spin --target wasm32-wasip1 --features spin + + - name: Document Axum adapter + run: RUSTDOCFLAGS='-D warnings' cargo doc --no-deps --all-features -p trusted-server-adapter-axum + + - name: Document host crates + run: RUSTDOCFLAGS='-D warnings' cargo doc --no-deps --all-features -p trusted-server-cli -p trusted-server-openrtb-codegen --target x86_64-unknown-linux-gnu + + - name: Run native core doctests + run: cargo test --doc -p trusted-server-core diff --git a/.tool-versions b/.tool-versions index 758146800..d34ca63d5 100644 --- a/.tool-versions +++ b/.tool-versions @@ -1,5 +1,6 @@ fastly 15.1.0 rust 1.95.0 nodejs 24.12.0 +wrangler 4.129.0 viceroy 0.17.0 wasmtime 44.0.1 diff --git a/AGENTS.md b/AGENTS.md index 836df88d5..50834d19c 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -11,19 +11,76 @@ fallback. --- +## CI Gates + + + +### Documentation parity + + + +- `cargo fmt --manifest-path tools/docs-parity/Cargo.toml -- --check` +- `cargo clippy --manifest-path tools/docs-parity/Cargo.toml --all-targets --all-features -- -D warnings` +- `cargo test --manifest-path tools/docs-parity/Cargo.toml` +- `cargo run --manifest-path tools/docs-parity/Cargo.toml -- check --all` +- `cargo run --manifest-path tools/docs-parity/Cargo.toml -- generate --check` + +### JavaScript and documentation site + + + +- `cd crates/trusted-server-js/lib && npm ci && npm run lint && npx vitest run && npm run format && npm run build` +- `cd docs && npm ci && npm run lint && npm run format && npm run build` + +### Rust formatting and linting + + + +- `cargo fmt --all -- --check` +- `cargo clippy-fastly` +- `cargo clippy-axum` +- `cargo clippy-cloudflare` +- `cargo clippy-cloudflare-wasm` +- `cargo clippy-spin-native` +- `cargo clippy-spin-wasm` +- `cargo clippy --package trusted-server-cli --target $(rustc -vV | sed -n 's/host: //p') --all-targets --all-features -- -D warnings` +- `cargo clippy --package trusted-server-openrtb-codegen --target $(rustc -vV | sed -n 's/host: //p') --all-targets -- -D warnings` + +### Rust tests and release builds + + + +- `cargo test-fastly` +- `cargo test-axum` +- `cargo test-cloudflare` +- `cargo test-spin` +- `cargo test --manifest-path crates/trusted-server-integration-tests/Cargo.toml --test parity` +- `cargo test --manifest-path crates/trusted-server-integration-tests/Cargo.toml --test documentation_snippets` +- `./scripts/test-cli.sh` +- `cargo test --package trusted-server-openrtb-codegen --target $(rustc -vV | sed -n 's/host: //p')` +- `cargo build --package trusted-server-adapter-fastly --release --target wasm32-wasip1` +- `cargo build --package trusted-server-adapter-spin --target wasm32-wasip1 --features spin --release` + +### Rust API documentation + + + +- `RUSTDOCFLAGS='-D warnings' cargo doc --no-deps --all-features -p trusted-server-core -p trusted-server-js -p trusted-server-openrtb --target wasm32-wasip1` +- `RUSTDOCFLAGS='-D warnings' cargo doc --no-deps -p trusted-server-adapter-fastly --target wasm32-wasip1` +- `RUSTDOCFLAGS='-D warnings' cargo doc --no-deps -p trusted-server-adapter-cloudflare --target wasm32-unknown-unknown --features cloudflare` +- `RUSTDOCFLAGS='-D warnings' cargo doc --no-deps -p trusted-server-adapter-spin --target wasm32-wasip1 --features spin` +- `RUSTDOCFLAGS='-D warnings' cargo doc --no-deps --all-features -p trusted-server-adapter-axum` +- `RUSTDOCFLAGS='-D warnings' cargo doc --no-deps --all-features -p trusted-server-cli -p trusted-server-openrtb-codegen --target $(rustc -vV | sed -n 's/host: //p')` +- `cargo test --doc -p trusted-server-core` + + + ## Fallback Summary If you cannot read `CLAUDE.md`, follow these rules: 1. Present a plan and get approval before coding. 2. Keep changes minimal — do not refactor unrelated code. -3. Run tests after every code change — use the workspace aliases defined in `.cargo/config.toml`: - - `cargo test-fastly` — Fastly adapter + core (wasm32-wasip1 via Viceroy) - - `cargo test-axum` — Axum dev server adapter (native) - - `cargo test-cloudflare` — Cloudflare Workers adapter (native host) - Do NOT use bare `cargo test --workspace` — it will attempt to compile the Fastly adapter for the host target. -4. Run `cargo fmt --all -- --check` and `cargo clippy-fastly && cargo clippy-axum && cargo clippy-cloudflare`. -5. Run JS tests with `cd crates/trusted-server-js/lib && npx vitest run` when touching JS/TS code. -6. Use `error-stack` (`Report`) for error handling — not anyhow, eyre, or thiserror. -7. Use `log` macros (not `println!`) and `expect("should ...")` (not `unwrap()`). -8. Target is `wasm32-wasip1` — no Tokio or OS-specific dependencies in core crates. +3. Use `error-stack` (`Report`) for error handling — not anyhow, eyre, or thiserror. +4. Use `log` macros (not `println!`) and `expect("should ...")` (not `unwrap()`). +5. Target is `wasm32-wasip1` — no Tokio or OS-specific dependencies in core crates. diff --git a/CHANGELOG.md b/CHANGELOG.md index a4cc36f68..cf4df2e96 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,16 +10,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Changed - **Breaking:** Auction providers and bidder routes now use the configuration-first `[auction.providers.]` and `[auction.bidders.]` maps. The removed `[auction].providers = [...]` list and removed server fields under `[integrations.prebid]` and `[integrations.aps]` are rejected even when those integrations are disabled, and `ts config push` rejects the old shape before publication. Move PBS `server_url` to provider `endpoint`, server timeout to provider `timeout_ms`, request controls and bidder-parameter overrides to the `prebid-server` `profile_config`, notification suppression to `notifications`, and each former server bidder to an `[auction.bidders.]` route. Move APS endpoint, timeout, account, inventory, debug, and creative controls to an `aps` provider and its `profile_config`. Browser Prebid settings remain under `[integrations.prebid]`; values such as timeout and debug that previously affected both browser and server behavior must now be configured for each owner. Provider endpoints must be absolute HTTPS URLs. Only bidder codes present in `[auction.bidders]` are folded into Trusted Server requests; unlisted publisher bids remain native browser demand. This schema has no mixed-version-safe deployment order: old binaries reject the maps and new binaries reject the retired fields, so activate the new binary and config blob together. Rollbacks must restore an old-schema blob together with the old binary. -- **Breaking** — Admin Basic-auth coverage now includes `GET /_ts/admin/ec`, `GET /_ts/admin/ec/{id}`, and `GET /_ts/admin/eids`. Existing configurations whose `[[handlers]]` patterns protect only the key-management endpoints now fail startup; broaden coverage before deploying, preferably with a namespace-boundary pattern such as `^/_ts/admin(?:/|$)`. Coverage of the dynamic `/_ts/admin/ec/{id}` route is no longer inferred from ID-shaped samples: the router accepts any segment after `/_ts/admin/ec/` and Basic Auth runs on the raw path before routing, so patterns anchored to the EC ID grammar (for example `^/_ts/admin/ec/[a-f0-9]{64}[.][A-Za-z0-9]{6}$`) are rejected in favor of a prefix-level matcher. Placeholder and well-known weak handler passwords (`changeme`, `password`, `admin`, `replace-with-…`) now fail startup on every handler rather than only on handlers inferred to cover an admin endpoint, because first-match-wins handler selection lets a narrow handler shadow the admin namespace. +- **Breaking:** Admin Basic-auth coverage now includes `GET /_ts/admin/ec`, `GET /_ts/admin/ec/{id}`, and `GET /_ts/admin/eids`. Existing configurations whose `[[handlers]]` patterns protect only the key-management endpoints now fail startup; broaden coverage before deploying, preferably with a namespace-boundary pattern such as `^/_ts/admin(?:/|$)`. Coverage of the dynamic `/_ts/admin/ec/{id}` route is no longer inferred from ID-shaped samples: the router accepts any segment after `/_ts/admin/ec/` and Basic Auth runs on the raw path before routing, so patterns anchored to the EC ID grammar (for example `^/_ts/admin/ec/[a-f0-9]{64}[.][A-Za-z0-9]{6}$`) are rejected in favor of a prefix-level matcher. Placeholder and well-known weak handler passwords (`changeme`, `password`, `admin`, `replace-with-…`) now fail startup on every handler rather than only on handlers inferred to cover an admin endpoint, because first-match-wins handler selection lets a narrow handler shadow the admin namespace. - Prebid Server provider endpoints now normalize origin-only legacy `server_url` values to `/openrtb2/auction`. Query parameters are preserved, the canonical path loses a trailing slash, and configured non-root custom paths remain exact. - Publisher HTML uses the browser-only `Cache-Control: private, max-age=60` policy for successful GET document responses and their `304 Not Modified` revalidations when server-side ad templates are structurally inactive, while preserving origin `private`/`no-store` policies and request-scoped bot, prefetch, or consent-denied responses. The `private` directive prevents shared caches that use `Cache-Control` from storing the document. Cookie-bearing responses using the generated inactive policy are finalized as `private, max-age=0`; CDN-specific cache headers remain unchanged and continue to control supporting CDNs independently. Set `[creative_opportunities].enabled = false` to disable publisher HTML and SPA template delivery without disabling direct `POST /auction` callers; an absent configuration, an unmatched slot, or a disabled auction also make the stack structurally inactive. An explicit `enabled = false` is not compatible with older binaries: restore the default, re-push and finalize the config before rolling back. -- **Breaking** — Replaced the legacy APS contextual integration with APS OpenRTB at `/e/pb/bid`. APS configuration now uses canonical `account_id` (`pub_id` remains a compatibility alias), no longer requires APS-specific slot IDs, and defaults script creative eligibility off. Operators must update the endpoint, disable native APS demand for Trusted Server cohorts, and prepare GAM/Universal Creative targeting for `hb_bidder=aps` before rollout. `aps` entries in Prebid bidder lists are logged and stripped. APS renderer winners now preserve the upstream bid `id`, omit `crid` when APS omits it, and carry `ext.trusted_server.renderer` instead of `adm`; external `/auction` consumers must support this response shape. -- **Breaking** — All auction paths now forward only a validated publisher-owned page URL as `site.page`, removing query and fragment data. APS OpenRTB omits `site.ref`; the existing Prebid Server path continues to forward the browser `Referer` as `site.ref`. Query-driven sites may lose contextual targeting and per-page reporting signals that previously came from query parameters. +- **Breaking:** Replaced the legacy APS contextual integration with APS OpenRTB at `/e/pb/bid`. APS configuration now uses canonical `account_id` (`pub_id` remains a compatibility alias), no longer requires APS-specific slot IDs, and defaults script creative eligibility off. Operators must update the endpoint, disable native APS demand for Trusted Server cohorts, and prepare GAM/Universal Creative targeting for `hb_bidder=aps` before rollout. `aps` entries in Prebid bidder lists are logged and stripped. APS renderer winners now preserve the upstream bid `id`, omit `crid` when APS omits it, and carry `ext.trusted_server.renderer` instead of `adm`; external `/auction` consumers must support this response shape. +- **Breaking:** All auction paths now forward only a validated publisher-owned page URL as `site.page`, removing query and fragment data. APS OpenRTB omits `site.ref`; the existing Prebid Server path continues to forward the browser `Referer` as `site.ref`. Query-driven sites may lose contextual targeting and per-page reporting signals that previously came from query parameters. - Publisher HTML now uses `Cache-Control: max-age=60` when server-side ad templates are inactive, while preserving origin `private`/`no-store` policies and CDN-specific cache headers. Set `[creative_opportunities].enabled = false` to disable publisher HTML and SPA template delivery without disabling direct `POST /auction` callers. -- **Breaking** — `bid_param_zone_overrides` inner values must now be JSON objects; previously non-object or empty values (`"header" = "x"`, `"header" = {}`) were accepted and silently produced a dead rule at runtime. They now fail at startup with a configuration error. Operators upgrading should audit their `bid_param_zone_overrides` config for non-object zone entries. -- **Breaking** — Integration configuration strings are no longer globally reinterpreted as JSON scalars. Operators upgrading should audit `[integrations.*]` settings and use native TOML/typed-config booleans and numbers (for example, `enabled = true`, not `enabled = "true"`); quoted numeric and boolean scalars now fail validation instead of silently converting. -- **Breaking** — Sourcepoint browser module inclusion now requires explicit `[integrations.sourcepoint].enabled = true`; operators relying on the previous unconditional Sourcepoint module should enable the integration before upgrading. -- **Breaking** — Auction creative sanitization is now opt-in: the new `[auction].sanitize_creatives` defaults to `false` because unconditional sanitization blanked script-based creatives (the majority of programmatic display) while recording normal impressions. `[auction].rewrite_creatives` keeps its `true` default. The per-creative cap is now enforced on rewritten output as well as raw input and in every processing mode (1 MiB for auction `adm`; proxied HTML documents keep the proxy's own 10 MiB bound), rewriting fails closed on parser errors instead of emitting partial output and never turns a rejected creative into a runtime-only `adm`, and `hb_cache_host`/`hb_cache_path` are emitted only for bids that supplied no creative — any bid carrying its own `adm` ships without them, so a processed or rejected creative can never be re-fetched raw from PBS Cache. Creative markup with no `` token now receives the click-guard runtime, and bidder `` elements are stripped whenever rewriting is enabled. The creative iframe sandbox no longer grants `allow-same-origin`, restoring origin isolation; rewritten-click recovery from the resulting opaque-origin iframe uses the GET `/first-party/proxy-rebuild` navigation fallback, now registered in every adapter and documented alongside the POST JSON form. Inside those iframes, dynamic resource signing and CORS-mode subresources (ES modules, `crossorigin` fonts) are unavailable pending the constrained asset capability in [#982](https://github.com/IABTechLab/trusted-server/issues/982); ordinary image, script, and stylesheet loads are unaffected. Upgrading: binaries that predate `sanitize_creatives` reject a blob carrying it, so upgrade the binary first, then push the config. Rollback: non-default values (`sanitize_creatives = true`, `rewrite_creatives = false`) are serialized into the config blob and older binaries reject unknown fields — before rolling back to a binary that predates a field, restore its default, push the default-compatible blob, then roll back. +- **Breaking:** `bid_param_zone_overrides` inner values must now be JSON objects; previously non-object or empty values (`"header" = "x"`, `"header" = {}`) were accepted and silently produced a dead rule at runtime. They now fail at startup with a configuration error. Operators upgrading should audit their `bid_param_zone_overrides` config for non-object zone entries. +- **Breaking:** Integration configuration strings are no longer globally reinterpreted as JSON scalars. Operators upgrading should audit `[integrations.*]` settings and use native TOML/typed-config booleans and numbers (for example, `enabled = true`, not `enabled = "true"`); quoted numeric and boolean scalars now fail validation instead of silently converting. +- **Breaking:** Sourcepoint browser module inclusion now requires explicit `[integrations.sourcepoint].enabled = true`; operators relying on the previous unconditional Sourcepoint module should enable the integration before upgrading. +- **Breaking:** Auction creative sanitization is now opt-in: the new `[auction].sanitize_creatives` defaults to `false` because unconditional sanitization blanked script-based creatives (the majority of programmatic display) while recording normal impressions. `[auction].rewrite_creatives` keeps its `true` default. The per-creative cap is now enforced on rewritten output as well as raw input and in every processing mode (1 MiB for auction `adm`; proxied HTML documents keep the proxy's own 10 MiB bound), rewriting fails closed on parser errors instead of emitting partial output and never turns a rejected creative into a runtime-only `adm`, and `hb_cache_host`/`hb_cache_path` are emitted only for bids that supplied no creative — any bid carrying its own `adm` ships without them, so a processed or rejected creative can never be re-fetched raw from PBS Cache. Creative markup with no `` token now receives the click-guard runtime, and bidder `` elements are stripped whenever rewriting is enabled. The creative iframe sandbox no longer grants `allow-same-origin`, restoring origin isolation; rewritten-click recovery from the resulting opaque-origin iframe uses the GET `/first-party/proxy-rebuild` navigation fallback, now registered in every adapter and documented alongside the POST JSON form. Inside those iframes, dynamic resource signing and CORS-mode subresources (ES modules, `crossorigin` fonts) are unavailable pending the constrained asset capability in [#982](https://github.com/IABTechLab/trusted-server/issues/982); ordinary image, script, and stylesheet loads are unaffected. Upgrading: binaries that predate `sanitize_creatives` reject a blob carrying it, so upgrade the binary first, then push the config. Rollback: non-default values (`sanitize_creatives = true`, `rewrite_creatives = false`) are serialized into the config blob and older binaries reject unknown fields — before rolling back to a binary that predates a field, restore its default, push the default-compatible blob, then roll back. - The SPA re-auction endpoint moved from `/__ts/page-bids` to `/_ts/page-bids`, joining every other internal route in the `/_ts/` namespace. The old path stays registered as a deprecated alias so already-loaded bundles keep serving ads, and responses on it carry a `Link: …; rel="deprecation"` header so remaining traffic is measurable from edge logs; removal is tracked in [#970](https://github.com/IABTechLab/trusted-server/issues/970). Two deployment notes: audit `[[handlers]]` for patterns broad enough to cover `/_ts` (for example `^/_ts`), which would put this browser-facing endpoint behind Basic Auth and return `401` to every visitor — scope them to `^/_ts/admin`; and prefer rolling forward over rolling back, since a server reverted past this release does not register the canonical path. In both cases the shipped client falls back to the deprecated alias, so the exposure is bounded until that alias is removed. - Added optional APS `inventory_domain` and `inventory_page_origin` overrides for deployments whose edge hostname differs from the APS-authorized inventory identity. - Preserved APS renderer capabilities through the client-side `trustedServer` Prebid adapter, allowing its generated `hb_adid` to render through GAM and Prebid Universal Creative instead of producing an empty creative. @@ -45,7 +45,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Implemented basic authentication for configurable endpoint paths (#73) - Added integrations guide with example `testlight` integration -## [1.2.0] - 2025-10-14 +## 1.2.0 - 2025-10-14 (tag v1.2.0 was never published) ### Changed @@ -138,8 +138,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Initial implementation of Trusted Server -[Unreleased]: https://github.com/IABTechLab/trusted-server/compare/v1.2.0...HEAD -[1.2.0]: https://github.com/IABTechLab/trusted-server/compare/v1.1.0...v1.2.0 +[Unreleased]: https://github.com/IABTechLab/trusted-server/compare/v1.1.0...HEAD [1.1.0]: https://github.com/IABTechLab/trusted-server/compare/v1.0.6...v1.1.0 [1.0.6]: https://github.com/IABTechLab/trusted-server/compare/v1.0.5...v1.0.6 [1.0.5]: https://github.com/IABTechLab/trusted-server/compare/v1.0.4...v1.0.5 diff --git a/CLAUDE.md b/CLAUDE.md index 546a3bf52..d9c4192aa 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -5,10 +5,10 @@ ## Project Overview -Rust-based edge computing application targeting **Fastly Compute**. Handles -Edge Cookie (EC) ID generation, ad serving with consent signal extraction -and enforcement, real-time bidding integration, and publisher-side -JavaScript injection. +Rust-based publisher edge application with a portable core and adapters for +Fastly Compute, Cloudflare Workers, Fermyon Spin, and native Axum development. +It handles Edge Cookie (EC) ID generation, consent-aware ad serving, auction +orchestration, first-party proxying, and publisher-side JavaScript injection. ## Workspace Layout @@ -22,6 +22,9 @@ crates/ trusted-server-cli/ # Host-target `ts` operator CLI trusted-server-js/ # TypeScript/JS build — per-integration IIFE bundles lib/ # TS source, Vitest tests, esbuild pipeline + trusted-server-integration-tests/ # Native parity, end-to-end, and documentation tests + trusted-server-openrtb/ # Checked-in OpenRTB JSON model + trusted-server-openrtb-codegen/ # Host-only manual OpenRTB generator ``` Supporting files: `edgezero.toml`, `fastly.toml`, @@ -99,7 +102,7 @@ cargo test-axum # Axum dev server adapter (native) cargo test-cloudflare # Cloudflare Workers adapter (native host) cargo test-spin # Spin adapter route tests (native host) -# Run host-target CLI tests (workspace default target is wasm32-wasip1) +# Run host-target CLI tests (the workspace has no global target) # Use your host triple, for example x86_64-unknown-linux-gnu on CI/Linux # or aarch64-apple-darwin on Apple Silicon macOS. # Use the local helper (recommended): @@ -253,7 +256,8 @@ impl core::error::Error for MyError {} - Always use intra-doc links (`[`Item`]`) for referenced types. - Document errors with `# Errors` section for all fallible functions. - Document panics with `# Panics` section. -- Add `# Examples` sections for public API functions. +- Add `# Examples` sections when an example proves behavior or prevents a + likely misuse; do not add ceremonial examples that restate the signature. - Add `# Performance` sections for performance-critical functions. - Skip documentation for standard trait implementations unless behavior is unique. - Use `cargo doc --no-deps --all-features` to verify. @@ -269,11 +273,18 @@ impl core::error::Error for MyError {} ## Other guidelines -- Use only example or fictional information in comments, tests, docs, examples, - and similar non-runtime materials. (eg. for urls use: example.com domains only) -- Do not write or commit real domains, customer names, credentials, - configuration values, or other potentially sensitive real-world information in - comments, tests, docs, or examples. +- Use example or fictional information by default in comments, tests, docs, + examples, and similar non-runtime materials. Use `example.com` domains for + invented URLs. +- Do not write or commit customer names, credentials, private configuration, or + other sensitive real-world information. A necessary public vendor endpoint is + permitted only through the exact, typed, owned, expiring exception below. +- Sensitive-data exceptions are limited to these types: vendor URL, + hash-pinned fake-credential fixture, historical example, service ID, and + project-owned public domain. +- Record every exception before use with its exact type and scope, owner, + rationale, and expiry timestamp. Ownerless, expired, broad, or untyped + exceptions are prohibited. --- @@ -303,9 +314,16 @@ IntegrationRegistration::builder(ID) .build() ``` -- Integration IDs match JS directory names: `prebid` (deferred), `lockr`, `permutive`, `datadome`, `didomi`, `testlight`. -- `creative` is JS-only (no Rust registration); `nextjs`, `aps`, `adserver_mock` are Rust-only. -- Integrations opt into deferred loading via `.with_deferred_js()` on the registration builder. Deferred modules are served as separate `` - - The bundle guards anchor clicks by restoring the originally rewritten first‑party link at click time. - - Served through the unified endpoint described below. - -Helpers: - -- `rewrite_creative_html(settings, markup) -> String` — rewrite an HTML fragment -- `rewrite_css_body(settings, css) -> String` — rewrite a CSS body (`url(...)` entries) -- `rewrite_srcset(settings, srcset) -> String` — proxy absolute candidates; preserve descriptors (`1x`, `1.5x`, `100w`) -- `split_srcset_candidates(srcset) -> Vec<&str>` — robust splitting for commas with/without spaces; avoids splitting the first `data:` mediatype comma - -JS bundles (served by publisher module): - -- Dynamic endpoint: `/static/tsjs=tsjs-unified.min.js?v=` - - At build time, embedded integrations are compiled as separate IIFEs (`tsjs-core.js`, `tsjs-creative.js`, etc.); Prebid is generated externally and served through `/integrations/prebid/bundle.js`. - - At runtime, the server concatenates `tsjs-core.js` + enabled integration modules based on `IntegrationRegistry` config - - The URL filename is fixed for backward compatibility; the `?v=` hash changes when modules change - -Behavior is covered by an extensive test suite in `crates/trusted-server-core/src/creative.rs`. - -## Edge Cookie (EC) Identifier Propagation - -- The `ec/` module owns the EC identity subsystem: - - `ec/generation.rs` — creates HMAC-based IDs using the client IP and publisher passphrase (format: `64hex.6alnum`). - - `ec/mod.rs` — `EcContext` struct with two-phase lifecycle (`read_from_request` + `generate_if_needed`), `get_ec_id` helper. - - `ec/consent.rs` — EC-specific consent gating wrapper. - - `ec/cookies.rs` — `Set-Cookie` header creation and expiration helpers. -- `publisher.rs::handle_publisher_request` issues the `ts-ec` cookie when absent so the browser keeps the identifier on subsequent requests. -- `proxy.rs::handle_first_party_proxy` replays the identifier to third-party creative origins by appending `ts-ec=` to the reconstructed target URL, follows redirects (301/302/303/307/308) up to four hops, and keeps downstream fetches linked to the same user scope. -- `proxy.rs::handle_first_party_click` adds `ts-ec=` to outbound click redirect URLs so analytics endpoints can associate clicks with impressions without third-party cookies. +Portable application core shared by every Trusted Server adapter. It targets +both supported WASM environments and native adapter tests, so it must not depend +on an edge SDK, Tokio runtime, filesystem, socket, or host-only process API. + +## Responsibilities + +- `settings` and `settings_data` define typed application configuration, + validation, secret references, and runtime normalization. +- `platform` defines the HTTP, backend, store, geo, client-info, and telemetry + service boundary adapters implement. +- `publisher`, `router`, `handlers`, and `response` dispatch publisher and + administrative requests through platform-neutral request/response types. +- `auction` builds plans, invokes providers and mediators, selects bids, and + emits bounded telemetry events. +- `integrations` registers explicit proxy, rewrite, injection, filter, + post-processing, provider, and browser-module capabilities. +- `ec` owns edge-cookie generation, consent decisions, identity graph access, + and partner synchronization. +- `html_processor`, `host_rewrite`, `streaming_processor`, and `rsc_flight` + transform eligible publisher responses without corrupting non-HTML or RSC + payloads. +- `proxy`, `creative`, `image_optimizer`, and `asset_routes` implement bounded + first-party asset and creative handling. +- `auth`, `request_signing`, `key_manager`, and `jwk` implement authentication + and signing contracts. +- `cache_policy`, `template_cache`, and `template_assembly` define portable + cache and assembly decisions; adapters supply storage and streaming I/O. +- `tsjs` selects embedded browser modules supplied by `trusted-server-js`. +- `openrtb` connects auction logic to the checked OpenRTB data model. + +Adapter crates own runtime startup, SDK conversion, concrete storage, outbound +transport, and target-specific limitations. Adding an integration should use +the narrowest registration hook and the core-neutral `RuntimeServices` +boundary; see the [Integration Guide](../../docs/guide/integration-guide.md). + +## Build and test + +From the repository root: + +```bash +cargo build-fastly +cargo test-fastly +``` + +The Fastly aliases compile and test the core for `wasm32-wasip1` through +Viceroy. Cloudflare and native adapter suites exercise the same core under +their target configurations. Use [TESTING.md](../../TESTING.md) for the complete +target matrix and [Architecture](../../docs/guide/architecture.md) for the +request-level system view. diff --git a/crates/trusted-server-core/src/auction/README.md b/crates/trusted-server-core/src/auction/README.md index 3599624f6..a3fb28cd3 100644 --- a/crates/trusted-server-core/src/auction/README.md +++ b/crates/trusted-server-core/src/auction/README.md @@ -1,528 +1,63 @@ -# Auction Orchestration System - -A flexible, extensible framework for managing multi-provider header bidding auctions with support for parallel execution and mediation. - -## Overview - -The auction orchestration system allows you to: -- Run multiple auction providers (Prebid, Amazon APS, etc.) in parallel or sequentially -- Implement mediation strategies where a primary ad server makes the final decision -- Configure different auction flows for different scenarios -- Easily add new auction providers - -## Architecture - -``` -┌─────────────────────────────────────────────────────────┐ -│ Auction Orchestrator │ -│ - Manages auction workflow & sequencing │ -│ - Combines bids from multiple sources │ -│ - Applies business logic │ -└─────────────────────────────────────────────────────────┘ - │ - │ uses - ▼ -┌─────────────────────────────────────────────────────────┐ -│ AuctionProvider Trait │ -│ - request_bids() async │ -│ - parse_response() │ -│ - provider_name() │ -│ - timeout_ms() │ -│ - is_enabled() │ -└─────────────────────────────────────────────────────────┘ - │ - ┌─────────────────┼─────────────────┐ - │ │ │ - ▼ ▼ ▼ - ┌──────────┐ ┌──────────┐ ┌──────────┐ - │ Prebid │ │ Amazon │ │ AdServer │ - │ Provider │ │ APS │ │ Mock │ - └──────────┘ └──────────┘ └──────────┘ -``` - -## Request Flow - -When a request arrives at the `/auction` endpoint, it goes through the following steps: - -``` -┌──────────────────────────────────────────────────────────────────────┐ -│ 1. HTTP POST /auction │ -│ - Body: AdRequest (Prebid.js/tsjs format) │ -│ - Headers: User-Agent, cookies, etc. │ -└──────────────────────────────────────────────────────────────────────┘ - │ - ▼ -┌──────────────────────────────────────────────────────────────────────┐ -│ 2. Route Matching (crates/trusted-server-adapter-fastly/src/main.rs)│ -│ - Pattern: (Method::POST, "/auction") │ -│ - Handler: handle_auction(settings, &orchestrator, │ -│ &runtime_services, req) │ -└──────────────────────────────────────────────────────────────────────┘ - │ - ▼ -┌──────────────────────────────────────────────────────────────────────┐ -│ 3. Parse Request Body (mod.rs:149) │ -│ - Deserialize JSON → AdRequest struct │ -│ - Extract ad units with media types │ -└──────────────────────────────────────────────────────────────────────┘ - │ - ▼ -┌──────────────────────────────────────────────────────────────────────┐ -│ 4. Generate User IDs (mod.rs:206-214) │ -│ - Create/retrieve EC ID (persistent) │ -│ - Generate fresh ID (per-request) │ -└──────────────────────────────────────────────────────────────────────┘ - │ - ▼ -┌──────────────────────────────────────────────────────────────────────┐ -│ 5. Transform Request Format (mod.rs:216-240) │ -│ - AdRequest → AuctionRequest │ -│ - AdUnit.code → AdSlot.id │ -│ - mediaTypes.banner.sizes → AdFormat[] │ -│ - Build PublisherInfo, UserInfo, DeviceInfo │ -└──────────────────────────────────────────────────────────────────────┘ - │ - ▼ -┌──────────────────────────────────────────────────────────────────────┐ -│ 6. Use Provided Orchestrator (mod.rs:150) │ -│ - Reused across requests from startup construction │ -│ - Contains all registered providers (APS, Prebid, etc.) │ -└──────────────────────────────────────────────────────────────────────┘ - │ - ▼ -┌──────────────────────────────────────────────────────────────────────┐ -│ 7. Create Auction Context (mod.rs:172-176) │ -│ - Attach settings │ -│ - Attach original request │ -│ - Set timeout from config │ -└──────────────────────────────────────────────────────────────────────┘ - │ - ▼ -┌──────────────────────────────────────────────────────────────────────┐ -│ 8. Run Auction Strategy (orchestrator.rs:42) │ -│ ┌────────────────────────────────────────────────────────────┐ │ -│ │ Strategy: parallel_only │ │ -│ │ 1. Launch all bidders concurrently │ │ -│ │ 2. Wait for all responses │ │ -│ │ 3. Select highest bid per slot │ │ -│ └────────────────────────────────────────────────────────────┘ │ -│ ┌────────────────────────────────────────────────────────────┐ │ -│ │ Strategy: parallel_mediation │ │ -│ │ 1. Launch all bidders concurrently │ │ -│ │ 2. Collect all bids │ │ -│ │ 3. Send to mediator for final decision │ │ -│ └────────────────────────────────────────────────────────────┘ │ -└──────────────────────────────────────────────────────────────────────┘ - │ - ▼ -┌──────────────────────────────────────────────────────────────────────┐ -│ 9. Each Provider Processes Request │ -│ - Transform AuctionRequest → Provider OpenRTB request │ -│ - Send HTTP request to provider endpoint │ -│ - Parse provider response │ -│ - Transform → AuctionResponse with Bid[] │ -│ - Return to orchestrator │ -└──────────────────────────────────────────────────────────────────────┘ - │ - ▼ -┌──────────────────────────────────────────────────────────────────────┐ -│ 10. Select Winning Bids (orchestrator.rs:363-385) │ -│ - For each slot, find highest CPM bid │ -│ - Create HashMap │ -│ - Log winning selections │ -└──────────────────────────────────────────────────────────────────────┘ - │ - ▼ -┌──────────────────────────────────────────────────────────────────────┐ -│ 11. Transform to OpenRTB Response (mod.rs:274-322) │ -│ - Build seatbid array (one per winning bid) │ -│ - Sanitize creative HTML when enabled (opt-in) │ -│ - Rewrite creative HTML when enabled (default) │ -│ - Add orchestrator metadata (timing, strategy, bid count) │ -└──────────────────────────────────────────────────────────────────────┘ - │ - ▼ -┌──────────────────────────────────────────────────────────────────────┐ -│ 12. Return HTTP Response │ -│ - Status: 200 OK │ -│ - Content-Type: application/json │ -│ - Body: OpenRTB BidResponse │ -└──────────────────────────────────────────────────────────────────────┘ -``` - -### Step-by-Step Breakdown - -#### 1. Request Arrival -Client (browser, Prebid.js, tsjs) sends a POST request to `/auction` with ad unit definitions: - -```json -{ - "adUnits": [ - { - "code": "header-banner", - "mediaTypes": { - "banner": { - "sizes": [[728, 90], [970, 250]] - } - } - } - ] -} -``` - -#### 2. Format Transformation -The system transforms the Prebid.js format into an internal `AuctionRequest`: - -```rust -// From: AdUnit with sizes [[728, 90], [970, 250]] -// To: AdSlot with formats -AdSlot { - id: "header-banner", - formats: vec![ - AdFormat { width: 728, height: 90, media_type: Banner }, - AdFormat { width: 970, height: 250, media_type: Banner }, - ], - floor_price: None, - targeting: HashMap::new(), -} -``` - -#### 3. Provider Execution -Each registered provider (APS, Prebid, etc.) receives the `AuctionRequest` and: -- Transforms it to the provider's OpenRTB request format -- Makes HTTP request to their endpoint -- Parses the response -- Returns `AuctionResponse` with `Bid[]` - -For example, APS provider: -```rust -// Transform AuctionRequest → APS OpenRTB request -// - ext.account = configured account_id -// - ext.sdk = { source: "prebid", version: "2.2.0" } -// - banner slots become secure impressions with matching formats/floors -// - existing consent, identity, device, and geo privacy gates apply - -// HTTP POST to https://aps.example.com/e/pb/bid -// Parse decoded-price response → AuctionResponse with a typed renderer -``` - -#### 4. Response Assembly -The orchestrator collects all bids and creates an OpenRTB response: - -```json -{ - "id": "auction-response", - "seatbid": [ - { - "seat": "aps", - "bid": [ - { - "id": "fictional-selected-bid-id", - "impid": "header-banner", - "price": 2.5, - "w": 728, - "h": 90, - "ext": { - "trusted_server": { - "renderer": { - "type": "aps", - "version": 1, - "accountId": "example-account", - "bidId": "fictional-selected-bid-id", - "tagType": "iframe", - "creativeUrl": "https://creative.example/render", - "aaxResponse": "", - "width": 728, - "height": 90 - } - } - } - } - ] - } - ], - "ext": { - "orchestrator": { - "strategy": "parallel_only", - "bidders": 1, - "total_bids": 1, - "time_ms": 5 - } - } -} -``` - -With `[auction].sanitize_creatives = true` (opt-in, default `false`), -executable markup is stripped with its inner content before delivery. With -`[auction].rewrite_creatives = true` (the default), each auction delivery path -rewrites eligible URLs through the first-party proxy (`/first-party/proxy`) and -removes bidder `` elements. The `POST /auction` response also injects the -creative runtime; the publisher SSAT inline path uses absolute first-party URLs -without injecting that bundle. With both disabled, the creative ships exactly -as the bidder returned it. In every mode, creatives over the 1 MiB cap are -rejected. - -## Route Registration & Endpoints - -### Auction-Related Routes - -The trusted-server handles several types of routes defined in `crates/trusted-server-adapter-fastly/src/main.rs`: - -| Route | Method | Handler | Purpose | Line | -|---------------------------|--------|--------------------------------|--------------------------------------------------|------| -| `/auction` | POST | `handle_auction()` | Main auction endpoint (Prebid.js/tsjs format) | 84 | -| `/first-party/proxy` | GET | `handle_first_party_proxy()` | Proxy creatives through first-party domain | 84 | -| `/first-party/click` | GET | `handle_first_party_click()` | Track clicks on ads | 85 | -| `/first-party/sign` | GET/POST | `handle_first_party_proxy_sign()` | Generate signed URLs for creatives | 86 | -| `/first-party/proxy-rebuild` | GET/POST | `handle_first_party_proxy_rebuild()` | Re-sign mutated click URLs (GET 302s for the opaque-origin click guard) | 89 | -| `/static/tsjs=*` | GET | `handle_tsjs_dynamic()` | Serve tsjs library (Prebid.js alternative) | 66 | -| `/.well-known/ts.jwks.json` | GET | `handle_jwks_endpoint()` | Public key distribution for request signing | 71 | -| `/verify-signature` | POST | `handle_verify_signature()` | Verify signed requests | 74 | -| `/_ts/admin/keys/rotate` | POST | `handle_rotate_key()` | Rotate signing keys (admin only) | 77 | -| `/_ts/admin/keys/deactivate` | POST | `handle_deactivate_key()` | Deactivate signing keys (admin only) | 78 | -| `/integrations/*` | * | Integration Registry | Provider-specific endpoints (Prebid, etc.) | 92 | -| `*` (fallback) | * | `handle_publisher_request()` | Proxy to publisher origin | 108 | - -### How Routing Works - -#### 1. Main Router (main.rs) -The Fastly Compute entrypoint uses pattern matching on `(Method, path)` tuples: - -```rust -let result = match (method, path.as_str()) { - // Auction endpoint - (Method::POST, "/auction") => { - handle_auction(&settings, &orchestrator, &runtime_services, req).await - }, - - // First-party endpoints - (Method::GET, "/first-party/proxy") => handle_first_party_proxy(&settings, req).await, - - // Integration registry (dynamic routes) - (m, path) if integration_registry.has_route(&m, path) => { - integration_registry.handle_proxy(&m, path, &settings, req).await - }, - - // Fallback to publisher origin - _ => handle_publisher_request(&settings, &integration_registry, &runtime_services, req), -} -``` - -#### 2. Integration Registry (Dynamic Routes) -Some integrations register their own routes dynamically. For example, Prebid registers `/integrations/prebid/auction`: - -```rust -// In integrations/prebid.rs -impl Integration for PrebidIntegration { - fn routes(&self) -> Vec { - vec![ - IntegrationRoute { - path: "/integrations/prebid/auction", - method: Method::POST, - handler: handle_prebid_auction, - } - ] - } -} -``` - -The integration registry checks if a route matches any registered integration routes before falling back to the publisher origin. - -#### 3. Route Priority -Routes are matched in this order: -1. **Exact top-level routes** (`/auction`, `/first-party/proxy`, etc.) -2. **Admin routes** (`/_ts/admin/*`) -3. **Integration routes** (`/integrations/*`) -4. **Fallback to publisher origin** (all other paths) - -This ensures auction and first-party endpoints take precedence over publisher content. - -### Auction Endpoint Deep Dive - -The `/auction` endpoint is the primary entry point for auctions: - -**Input Format (Prebid.js compatible):** -```json -{ - "adUnits": [ - { - "code": "div-id", - "mediaTypes": { - "banner": { - "sizes": [[300, 250], [728, 90]] - } - } - } - ], - "config": { /* optional Prebid.js config */ } -} -``` - -**Output Format (OpenRTB 2.x):** -```json -{ - "id": "auction-response", - "seatbid": [ - { - "seat": "bidder-name", - "bid": [ - { - "id": "bid-id", - "impid": "div-id", - "price": 2.5, - "adm": "", - "w": 300, - "h": 250 - } - ] - } - ], - "ext": { - "orchestrator": { - "strategy": "parallel_only", - "bidders": 2, - "total_bids": 3, - "time_ms": 150 - } - } -} -``` - -**Key Transformations:** -- `adUnits[].code` → `seatbid[].bid[].impid` (slot identifier) -- `mediaTypes.banner.sizes` → evaluated by providers, winning size in `bid.w` and `bid.h` -- Creative HTML: `[auction].sanitize_creatives = true` (opt-in) strips executable markup; `[auction].rewrite_creatives = true` (default) rewrites eligible URLs to `/first-party/proxy` in both delivery paths (with creative runtime injection on `POST /auction` only); with both disabled the creative ships as the bidder returned it -- Multiple bids per slot become separate `seatbid` entries -- Orchestrator metadata added in `ext.orchestrator` - -## Key Concepts - -### Auction Provider -Implements the `AuctionProvider` trait to integrate with a specific SSP/ad exchange. - -### Auction Flow -A named configuration that defines: -- Which providers participate -- Execution strategy (parallel mediation or parallel only) -- Timeout settings -- Optional mediator - -### Orchestrator -Manages the execution of an auction flow, coordinates providers, and collects results. - -## Auction Strategies - -### 1. Parallel + Mediation - -```toml -[auction] -enabled = true -timeout_ms = 2000 -mediator = "adserver_mock" - -[auction.providers.pbs-main] -protocol = "openrtb-2.6" -profile = "prebid-server" -endpoint = "https://prebid.example.com/openrtb2/auction" -routing = "explicit" - -[auction.providers.aps-main] -protocol = "openrtb-2.6" -profile = "aps" -endpoint = "https://aps.example.com/e/pb/bid" -routing = "all_eligible" -profile_config = { account_id = "example-aps-account" } -``` - -Providers run in parallel, then the separately registered mediator chooses from -decoded-price bids. - -### 2. Parallel Only - -Omit `mediator` from the same map-shaped configuration. The orchestrator selects -the highest decoded CPM per slot and applies floors locally. - -## Configuration - -`[auction.providers.]` is the only bidder-provider inventory. -`[auction.bidders.]` maps a client-visible bidder to exactly one -provider. The mediator is selected separately by `[auction].mediator`. - -```toml -[auction] -enabled = true -timeout_ms = 2000 - -[auction.providers.pbs-main] -protocol = "openrtb-2.6" -profile = "prebid-server" -endpoint = "https://prebid.example.com/openrtb2/auction" -timeout_ms = 900 -routing = "explicit" - -[auction.providers.pbs-main.profile_config] -debug = false -test_mode = false -consent_forwarding = "both" - -[auction.providers.pbs-main.notifications] -suppress_all = false -suppress_seats = ["example-seat"] - -[auction.bidders.example-server] -provider = "pbs-main" -``` - -Provider IDs own backend correlation and response identity. The configured -profile supplies typed OpenRTB behavior. Common endpoint, timeout, routing, and -notification policy do not belong to browser integration configuration. - -## Adding a Provider - -A standards-compatible OpenRTB 2.6 endpoint does not require a Rust provider -implementation. Add an `[auction.providers.]` table, select the `standard` -profile, and route bidder codes through `[auction.bidders.]`. Endpoint, -timeout, routing, and notification behavior are compiled into the shared -`AuctionPlan` at startup. - -Add Rust code only when an endpoint needs behavior that the existing -`standard`, `prebid-server`, or `aps` profiles cannot express. New profile work -belongs in `profile.rs` and `openrtb.rs`: define and validate typed profile -configuration, register the profile with the central profile registry, and add -request/response golden tests. Production provider registration is plan-backed; -`AuctionOrchestrator::register_provider` exists only in the legacy test parity -harness and is not an application extension API. - -See the maintained [auction orchestration guide](../../../../docs/guide/auction-orchestration.md) -and [integration guide](../../../../docs/guide/integration-guide.md) for complete -configuration and validation examples. - -## Testing - -Compile test settings with `compile_auction_plan`, construct the orchestrator and -integration registry from the same `Arc`, and exercise requests -through the normal adapter or auction endpoint. Profile tests should cover typed -configuration validation, exact OpenRTB request output, response admission, -provider-local failures, routing, and target capability validation. Legacy -provider constructors and manual registration are retained only for parity tests. - -## Performance Considerations - -- **Parallel Execution**: Providers are launched concurrently via `select()` over `PendingRequest`s; responses are processed as they become ready within the auction deadline -- **Timeouts**: Each provider has independent timeout; global timeout enforced at flow level -- **Error Handling**: Provider failures don't fail the entire auction; partial results are returned - -## Related Files - -- `src/auction/mod.rs` - Plan compilation and module exports -- `src/auction/plan.rs` - Typed provider plan and target validation -- `src/auction/profile.rs` - Typed OpenRTB profile registry -- `src/auction/routing.rs` - Central bidder-to-provider routing -- `src/auction/openrtb.rs` - Shared request construction and response parsing -- `src/auction/provider.rs` - Plan-backed provider execution -- `src/auction/orchestrator.rs` - Fan-out, deadline, and mediation flow -- `src/auction/types.rs` - Core auction types - -## Questions? - -See the main project [README](../../../../README.md) or [integration guide](../../../../docs/guide/integration-guide.md). +# Auction subsystem + +The auction subsystem compiles configuration into an immutable `AuctionPlan` +and uses that plan for provider execution, bidder routing, mediation, and +response assembly. Production adapters do not build an independent provider +registry. + +## Source map + +| File | Responsibility | +| --- | --- | +| `plan.rs` | Compile `[auction]`, provider instances, bidder routes, notifications, and mediator selection into `AuctionPlan` | +| `profile.rs` | Own `PROFILE_REGISTRATIONS` and compile the `standard`, `prebid-server`, and `aps` profile schemas | +| `provider.rs` | Execute plan-backed OpenRTB providers through `RuntimeServices` | +| `routing.rs` | Select impressions and bidder parameters for each provider | +| `orchestrator.rs` | Launch supported providers and select or mediate valid bids | +| `formats.rs` | Convert the browser request and internal bids to response shapes | +| `openrtb.rs` | Build and validate provider OpenRTB payloads | +| `endpoints.rs` | Implement `handle_auction` for `POST /auction` | +| `telemetry.rs` | Emit bounded auction telemetry where the adapter supports it | +| `config.rs`, `context.rs`, `types.rs` | Shared configuration aliases, request context, and domain types | + +Adapters register the public route with their own route tables. The canonical +route inventory is checked in `tools/docs-parity/manifests/routes.toml`; this +README does not duplicate unrelated adapter routes or line numbers. + +## Configuration ownership + +- `[auction.providers.]` is the complete provider-instance map. +- `[auction.bidders.]` maps a browser-visible bidder code to exactly + one provider instance. +- `PROFILE_REGISTRATIONS` is the complete provider-profile registry. +- `[auction].mediator` selects a separately registered mediator. The current + mediator inventory contains `adserver_mock`. + +The `standard` profile handles generic OpenRTB 2.6 endpoints. The +`prebid-server` and `aps` profiles add typed, provider-specific behavior. A +new standards-compatible endpoint normally needs configuration and tests, not +a new Rust provider type. + +## Validation and runtime + +Deploy validation compiles target-independent invariants. Adapter startup uses +the same plan and adds target capability checks. Fastly and Axum allow multiple +providers; Cloudflare and Spin currently require at most one enabled provider. + +Provider timeouts are logical budgets used for launch decisions and OpenRTB +`tmax`. No adapter currently guarantees an abortable provider-wide wall-clock +deadline. Provider errors and malformed responses fail locally and cannot be +converted into fabricated bids. + +`handle_auction` returns a no-bid response when auctions are disabled. With no +mediator, the orchestrator selects the highest valid bid per impression. When a +mediator is configured, that mediator owns final selection. APS renderable bids +use the typed renderer contract; ordinary OpenRTB bids carry validated creative +markup. + +## Tests + +Run the public [auction testing guide](../../../../docs/guide/auction-testing.md) +and the target-specific repository aliases in the root +[`TESTING.md`](../../../../TESTING.md). Unit tests live beside the modules; +cross-adapter behavior is checked by the integration-test parity suite. diff --git a/crates/trusted-server-core/src/auction/endpoints.rs b/crates/trusted-server-core/src/auction/endpoints.rs index de05dfdd8..0bfdf7221 100644 --- a/crates/trusted-server-core/src/auction/endpoints.rs +++ b/crates/trusted-server-core/src/auction/endpoints.rs @@ -76,7 +76,7 @@ const MAX_AUCTION_BODY_SIZE: usize = 256 * 1024; /// ## Context passthrough (`config`) /// /// The optional `config` object is filtered through -/// [`auction.allowed_context_keys`][`crate::settings::AuctionConfig::allowed_context_keys`]. +/// [`auction.allowed_context_keys`][`crate::auction_config_types::AuctionConfig::allowed_context_keys`]. /// Only keys listed there reach the auction providers (e.g. `"permutive_segments"`). /// All other keys are silently dropped. Values must be either strings or arrays of /// strings. diff --git a/crates/trusted-server-core/src/auction/formats.rs b/crates/trusted-server-core/src/auction/formats.rs index 571d9d484..1cd886f6e 100644 --- a/crates/trusted-server-core/src/auction/formats.rs +++ b/crates/trusted-server-core/src/auction/formats.rs @@ -37,7 +37,7 @@ use super::types::{ /// /// `adUnits` lists the placements to bid on. `config` carries optional /// context values (e.g. audience segments) filtered through -/// [`auction.allowed_context_keys`][`crate::settings::AuctionConfig::allowed_context_keys`]. +/// [`auction.allowed_context_keys`][`crate::auction_config_types::AuctionConfig::allowed_context_keys`]. #[derive(Debug, Deserialize)] #[serde(rename_all = "camelCase")] pub struct AdRequest { diff --git a/crates/trusted-server-core/src/auction/profile.rs b/crates/trusted-server-core/src/auction/profile.rs index 8f5b25806..9021d0bd9 100644 --- a/crates/trusted-server-core/src/auction/profile.rs +++ b/crates/trusted-server-core/src/auction/profile.rs @@ -323,3 +323,84 @@ fn reject_reserved_fields( } Ok(()) } + +#[cfg(test)] +mod tests { + use std::collections::BTreeSet; + + use serde::Deserialize; + use serde_json::json; + + use super::*; + + #[derive(Deserialize)] + struct Task7CheckedManifest { + template: Task7CheckedTemplate, + } + + #[derive(Deserialize)] + struct Task7CheckedTemplate { + profile_ids: BTreeSet, + } + + fn task7_checked_profile_ids() -> BTreeSet { + let checked: Task7CheckedManifest = toml::from_str(include_str!(concat!( + env!("CARGO_MANIFEST_DIR"), + "/../../tools/docs-parity/manifests/settings-companions.toml" + ))) + .expect("checked Task 7 settings manifest should parse"); + checked.template.profile_ids + } + + #[test] + fn task7_profile_registry_and_compiler_probes_are_exact() { + let registrations = profile_registrations() + .iter() + .map(|registration| registration.id.to_owned()) + .collect::>(); + assert_eq!( + registrations, + BTreeSet::from([ + APS_PROFILE_ID.to_owned(), + PREBID_PROFILE_ID.to_owned(), + STANDARD_PROFILE_ID.to_owned(), + ]) + ); + assert_eq!( + registrations, + task7_checked_profile_ids(), + "profile registry and checked documentation record should be equal" + ); + + for (id, positive, negative) in [ + ( + STANDARD_PROFILE_ID, + json!({"request_ext": {"fictional": true}}), + json!({"unknown": true}), + ), + ( + PREBID_PROFILE_ID, + json!({"debug": true}), + json!({"unknown": true}), + ), + ( + APS_PROFILE_ID, + json!({"account_id": "fictional-account"}), + json!({}), + ), + ] { + let registration = find_profile(id).expect("registered profile should resolve"); + let compiled = registration + .compile(&positive) + .unwrap_or_else(|error| panic!("{id} positive probe should compile: {error:?}")); + assert_eq!(compiled.id(), id); + let error = registration + .compile(&negative) + .expect_err("negative profile probe should fail"); + assert!( + error.to_string().contains("profile_config"), + "{id} negative probe should identify profile_config: {error:?}" + ); + } + } +} diff --git a/crates/trusted-server-core/src/auction/types.rs b/crates/trusted-server-core/src/auction/types.rs index 406915706..293923bad 100644 --- a/crates/trusted-server-core/src/auction/types.rs +++ b/crates/trusted-server-core/src/auction/types.rs @@ -136,7 +136,8 @@ pub struct SiteInfo { /// client state from `context.request`** — the placeholder has none of the /// real headers. If a future mediator needs that data, snapshot it into a new /// field on this struct at dispatch time and stash it on the -/// [`DispatchedAuction`] token so collect can attach it to the mediator's +/// [`DispatchedAuction`](crate::auction::orchestrator::DispatchedAuction) token +/// so collect can attach it to the mediator's /// context. See /// (P2-1) for the open follow-up. /// diff --git a/crates/trusted-server-core/src/auth.rs b/crates/trusted-server-core/src/auth.rs index f5e45bbd3..6eec97878 100644 --- a/crates/trusted-server-core/src/auth.rs +++ b/crates/trusted-server-core/src/auth.rs @@ -1,3 +1,8 @@ +//! Edge-terminated HTTP Basic authentication for protected routes. +//! +//! Successful checks attach a digest-bound marker used by cache policy. Failed +//! checks return a challenge without removing the original authorization value. + use base64::{Engine as _, engine::general_purpose::STANDARD}; use edgezero_core::body::Body as EdgeBody; use error_stack::Report; @@ -58,7 +63,7 @@ impl EdgeTerminatedAuthorization { /// /// # Request mutation /// -/// Takes `req` mutably because it owns [`EdgeTerminatedAuthorization`]. Any +/// Takes `req` mutably because it owns the `EdgeTerminatedAuthorization` marker. Any /// inherited marker is cleared on entry, and a fresh one is inserted only on the /// success path, so the marker present after this call always describes this /// call's own decision. Nothing else about the request is touched — in diff --git a/crates/trusted-server-core/src/config.rs b/crates/trusted-server-core/src/config.rs index fb88448a0..5e7f8f738 100644 --- a/crates/trusted-server-core/src/config.rs +++ b/crates/trusted-server-core/src/config.rs @@ -432,13 +432,15 @@ fn report_to_validation_error( #[cfg(test)] mod tests { - use std::collections::HashSet; + use std::collections::{BTreeMap, BTreeSet}; use super::*; + use crate::platform::{PlatformError, PlatformSecretStore, StoreId, StoreName}; use crate::redacted::Redacted; use crate::settings::{ProxyAssetRoute, S3SigV4AuthConfig}; use crate::test_support::tests::crate_test_settings_str; use edgezero_core::app_config::AppConfigMeta; + use edgezero_core::blob_envelope::BlobEnvelope; #[derive(Debug, Deserialize)] #[serde(deny_unknown_fields)] @@ -453,6 +455,33 @@ mod tests { slot: Vec, } + #[derive(Deserialize)] + struct Task7CheckedManifest { + template: Task7CheckedTemplate, + } + + #[derive(Deserialize)] + struct Task7CheckedTemplate { + placeholder_paths: BTreeSet, + integration_ids: BTreeSet, + profile_ids: BTreeSet, + consumer_literals: BTreeSet, + expected_failure_diagnostic: String, + } + + fn task7_checked_template() -> Task7CheckedTemplate { + let checked: Task7CheckedManifest = toml::from_str(include_str!(concat!( + env!("CARGO_MANIFEST_DIR"), + "/../../tools/docs-parity/manifests/settings-companions.toml" + ))) + .expect("checked Task 7 settings manifest should parse"); + checked.template + } + + fn task7_checked_integration_ids() -> BTreeSet { + task7_checked_template().integration_ids + } + fn app_config_with_creative_opportunities( gam_unit_path: Option<&str>, ) -> TrustedServerAppConfig { @@ -495,6 +524,40 @@ formats = [{ width = 300, height = 250 }] settings } + struct Task7SecretStore; + + impl PlatformSecretStore for Task7SecretStore { + fn get_bytes( + &self, + _store_name: &StoreName, + key: &str, + ) -> Result, Report> { + let resolved = match key { + "publisher_proxy_secret" => "fictional-proxy-secret-32-bytes-ok", + "ec_passphrase" => "fictional-ec-passphrase-32-bytes-ok", + "handler_password" => "fictional-admin-password-32-bytes-ok", + _ => { + return Err(Report::new(PlatformError::SecretStore) + .attach(format!("unexpected Task 7 secret key: {key}"))); + } + }; + Ok(resolved.as_bytes().to_vec()) + } + + fn create( + &self, + _store_id: &StoreId, + _name: &str, + _value: &str, + ) -> Result<(), Report> { + Ok(()) + } + + fn delete(&self, _store_id: &StoreId, _name: &str) -> Result<(), Report> { + Ok(()) + } + } + /// Source-controlled operator-facing config template. const EXAMPLE_TEMPLATE: &str = include_str!(concat!( env!("CARGO_MANIFEST_DIR"), @@ -537,7 +600,12 @@ formats = [{ width = 300, height = 250 }] .strip_prefix("# ") .or_else(|| line.strip_prefix('#')) .unwrap_or(line); - out.push(bare.to_owned()); + let syntax = bare.trim_start(); + if syntax.starts_with('[') || syntax.contains('=') { + out.push(bare.to_owned()); + } else { + out.push(line.to_owned()); + } } else { out.push(line.to_owned()); } @@ -1140,17 +1208,121 @@ password = "production-admin-password-32-bytes" } #[test] - fn deploy_validation_covers_registered_integration_builders() { - let validated_ids: HashSet<&'static str> = - DEPLOY_VALIDATED_INTEGRATION_IDS.iter().copied().collect(); - let missing_ids = crate::integrations::registered_builder_ids() - .filter(|id| !validated_ids.contains(id)) - .collect::>(); + fn deploy_validation_ids_match_checked_settings_record() { + let validated_ids: BTreeSet = DEPLOY_VALIDATED_INTEGRATION_IDS + .iter() + .map(|id| (*id).to_owned()) + .collect(); + let registered_ids: BTreeSet = crate::integrations::registered_builder_ids() + .map(str::to_owned) + .collect(); assert!( - missing_ids.is_empty(), - "deploy validation should cover all registered integration builders: {missing_ids:?}" + registered_ids.is_subset(&validated_ids), + "deploy validation IDs should cover every registered integration builder" + ); + + assert_eq!( + validated_ids, + task7_checked_integration_ids(), + "deploy validation IDs and the checked documentation record should be equal" + ); + } + + #[test] + fn task7_enables_every_deploy_validated_integration_in_isolation() { + let blocks = task7_source_integration_blocks(); + assert_eq!( + blocks + .keys() + .map(|id| (*id).to_owned()) + .collect::>(), + task7_checked_integration_ids(), + "source-template integration blocks and checked IDs must be equal" ); + for (id, header) in blocks { + let mut settings = task7_settings_with_source_integration_enabled(id, header); + validate_settings_for_deploy(&settings) + .unwrap_or_else(|error| panic!("{id} should validate while enabled: {error:?}")); + + settings + .integrations + .get_mut(id) + .and_then(serde_json::Value::as_object_mut) + .expect("source integration block should be an object") + .insert( + "enabled".to_owned(), + serde_json::Value::String("not-a-bool".to_owned()), + ); + assert!( + validate_settings_for_deploy(&settings).is_err(), + "{id} should reject its source-derived negative probe" + ); + } + } + + fn task7_source_integration_blocks() -> BTreeMap<&'static str, &'static str> { + BTreeMap::from([ + ("adserver_mock", "[integrations.adserver_mock]"), + ("aps", "[integrations.aps]"), + ("datadome", "[integrations.datadome]"), + ("didomi", "[integrations.didomi]"), + ("google_tag_manager", "[integrations.google_tag_manager]"), + ("gpt", "[integrations.gpt]"), + ("gpt_diagnostics", "[integrations.gpt_diagnostics]"), + ("lockr", "[integrations.lockr]"), + ("nextjs", "[integrations.nextjs]"), + ("osano", "[integrations.osano]"), + ("permutive", "[integrations.permutive]"), + ("prebid", "[integrations.prebid]"), + ("sourcepoint", "[integrations.sourcepoint]"), + ("testlight", "[integrations.testlight]"), + ]) + } + + fn task7_settings_with_source_integration_enabled(id: &str, header: &str) -> Settings { + let source = if EXAMPLE_TEMPLATE.lines().any(|line| line == header) { + EXAMPLE_TEMPLATE.to_owned() + } else { + let commented = format!("# {header}"); + assert!( + EXAMPLE_TEMPLATE.lines().any(|line| line == commented), + "{id} must be represented by an exact source-template block" + ); + uncomment_block(EXAMPLE_TEMPLATE, header) + }; + let mut settings = toml::from_str::(&source) + .unwrap_or_else(|error| panic!("source-derived {id} block should parse: {error}")) + .into_settings(); + task7_customize_nonsecret_values(&mut settings); + let raw = settings + .integrations + .get_mut(id) + .unwrap_or_else(|| panic!("source-derived {id} block should exist")); + let object = raw + .as_object_mut() + .unwrap_or_else(|| panic!("source-derived {id} block should be an object")); + object.insert("enabled".to_owned(), serde_json::Value::Bool(true)); + match id { + "prebid" => { + object.insert( + "external_bundle_url".to_owned(), + serde_json::json!("https://assets.example.com/prebid.js"), + ); + settings.proxy.allowed_domains = vec!["assets.example.com".to_owned()]; + } + "google_tag_manager" => { + object.insert("container_id".to_owned(), serde_json::json!("GTM-ABC123")); + } + _ => {} + } + settings + } + + fn task7_customize_nonsecret_values(settings: &mut Settings) { + settings.publisher.domain = "publisher.example".to_owned(); + settings.publisher.cookie_domain = ".publisher.example".to_owned(); + settings.publisher.origin_url = "https://origin.publisher.example".to_owned(); } #[test] @@ -1230,4 +1402,401 @@ password = "production-admin-password-32-bytes" "validation error should mention invalid provider" ); } + + #[test] + fn task7_source_template_round_trips_through_production_apis() { + let observation = run_task7_template_harness() + .expect("source template should pass every production harness phase"); + let checked = task7_checked_template(); + assert_eq!(observation.placeholder_paths, checked.placeholder_paths); + assert_eq!( + observation.failure_diagnostic, + checked.expected_failure_diagnostic + ); + assert_eq!( + observation.integration_ids, checked.integration_ids, + "source-derived integration probe set must be exact" + ); + assert_eq!( + observation.profile_ids, checked.profile_ids, + "source-derived profile compile set must be exact" + ); + assert_eq!( + observation.consumer_literals, checked.consumer_literals, + "serialized secret-key consumer literals must be exact" + ); + } + + #[test] + fn task7_standard_profile_probe_is_derived_from_the_source_template() { + let cases = task7_source_profile_cases().expect("source profile cases should parse"); + let (positive, negative) = cases + .get("standard") + .expect("source profile cases should contain standard"); + let registration = crate::auction::profile::find_profile("standard") + .expect("standard profile should be registered"); + let compiled = registration + .compile(positive) + .expect("source standard profile config should compile"); + assert_eq!(compiled.id(), "standard"); + let error = registration + .compile(negative) + .expect_err("malformed source-derived request_ext should fail"); + assert!(error.to_string().contains("request_ext")); + } + + #[test] + fn task7_production_harness_rejects_all_eight_contract_mutations() { + let typo_source = EXAMPLE_TEMPLATE.replacen( + "origin_url = \"https://origin.example.com\"", + "orgin_url = \"https://origin.example.com\"", + 1, + ); + toml::from_str::(&typo_source) + .expect_err("a source-template key typo must fail typed parsing"); + + let mut disabled_unknown = toml::from_str::(EXAMPLE_TEMPLATE) + .expect("source template should parse") + .into_settings(); + task7_customize_nonsecret_values(&mut disabled_unknown); + disabled_unknown + .integrations + .get_mut("prebid") + .and_then(serde_json::Value::as_object_mut) + .expect("source Prebid block should be an object") + .insert("task7_unknown".to_owned(), serde_json::Value::Bool(true)); + let _error = validate_settings_for_deploy(&disabled_unknown) + .expect_err("an unknown key must fail even in a disabled source block"); + + let mut bad_profile = toml::from_str::(EXAMPLE_TEMPLATE) + .expect("source template should parse") + .into_settings(); + task7_customize_nonsecret_values(&mut bad_profile); + bad_profile + .auction + .providers + .values_mut() + .find(|provider| provider.profile == "prebid-server") + .expect("source Prebid provider should exist") + .profile_config + .as_object_mut() + .expect("source profile config should be an object") + .insert("task7_unknown".to_owned(), serde_json::Value::Bool(true)); + let _error = validate_settings_for_deploy(&bad_profile) + .expect_err("a bad source-derived provider profile must fail compilation"); + + let mut unresolved = task7_serialized_source_config(EXAMPLE_TEMPLATE) + .expect("customized source template should serialize"); + *unresolved + .pointer_mut("/publisher/proxy_secret") + .expect("serialized publisher secret path should exist") = + serde_json::json!("task7_missing_secret"); + task7_resolve_envelope(unresolved) + .expect_err("an unresolved secret key must fail fake-store resolution"); + + let stranded_source = EXAMPLE_TEMPLATE.replacen( + "password = \"handler_password\"", + "password = \"handler_password_stranded\"", + 1, + ); + let stranded = task7_serialized_source_config(&stranded_source) + .expect("stranded-literal source mutation should serialize"); + assert_eq!( + task7_serialized_secret_literals(&stranded) + .expect("stranded source should retain all secret paths"), + BTreeSet::from([ + "ec_passphrase".to_owned(), + "handler_password_stranded".to_owned(), + "publisher_proxy_secret".to_owned(), + ]), + "stranded source literal must produce the exact wrong consumer set" + ); + task7_resolve_envelope(stranded) + .expect_err("a stranded literal must fail the production fake-store path"); + + let mut inactive = + task7_settings_with_source_integration_enabled("permutive", "[integrations.permutive]"); + let inactive_raw = inactive + .integrations + .get_mut("permutive") + .and_then(serde_json::Value::as_object_mut) + .expect("source Permutive block should be an object"); + inactive_raw.insert("enabled".to_owned(), serde_json::Value::Bool(false)); + inactive_raw.insert( + "organization_id".to_owned(), + serde_json::Value::String(String::new()), + ); + validate_settings_for_deploy(&inactive) + .expect("inactive invalid value demonstrates why forced-enabled probing is required"); + inactive + .integrations + .get_mut("permutive") + .and_then(serde_json::Value::as_object_mut) + .expect("source Permutive block should be an object") + .insert("enabled".to_owned(), serde_json::Value::Bool(true)); + let _error = validate_settings_for_deploy(&inactive) + .expect_err("the same source-derived block must fail when forced enabled"); + + let missing_profile = task7_probe_source_profiles(Some("aps")) + .expect("remaining source-derived profiles should compile"); + let checked_profiles = task7_checked_template().profile_ids; + assert_eq!( + missing_profile, + BTreeSet::from(["prebid-server".to_owned(), "standard".to_owned()]) + ); + assert_ne!( + missing_profile, checked_profiles, + "omitting a compiled provider profile must fail exact set equality" + ); + + let mut wrong_diagnostic = toml::from_str::(EXAMPLE_TEMPLATE) + .expect("source template should parse") + .into_settings(); + wrong_diagnostic.publisher.origin_url = "https://origin.publisher.example".to_owned(); + let (wrong_paths, wrong_text) = task7_placeholder_observation(&wrong_diagnostic) + .expect("remaining placeholders should produce a typed diagnostic"); + assert_eq!( + wrong_paths, + BTreeSet::from([ + "publisher.cookie_domain".to_owned(), + "publisher.domain".to_owned(), + ]) + ); + assert_eq!( + wrong_text, + "unmodified template rejects publisher.cookie_domain, publisher.domain" + ); + assert_ne!( + wrong_text, + task7_checked_template().expected_failure_diagnostic, + "a partial placeholder diagnostic must fail exact equality" + ); + } + + #[derive(Debug, Eq, PartialEq)] + struct Task7HarnessObservation { + placeholder_paths: BTreeSet, + integration_ids: BTreeSet, + profile_ids: BTreeSet, + consumer_literals: BTreeSet, + failure_diagnostic: String, + } + + fn run_task7_template_harness() -> Result { + let source: TrustedServerAppConfig = toml::from_str(EXAMPLE_TEMPLATE) + .map_err(|error| format!("source template parse failed: {error}"))?; + let (placeholder_paths, failure_diagnostic) = + task7_placeholder_observation(source.settings())?; + + let mut settings = source.into_settings(); + task7_customize_nonsecret_values(&mut settings); + validate_settings_for_deploy(&settings) + .map_err(|error| format!("deploy validation failed: {error:?}"))?; + + let app_config = TrustedServerAppConfig::new(settings) + .map_err(|error| format!("typed app config failed: {error:?}"))?; + let data = serde_json::to_value(&app_config) + .map_err(|error| format!("app config serialization failed: {error}"))?; + let consumer_literals = task7_serialized_secret_literals(&data)?; + + let envelope = BlobEnvelope::new(data, "2026-01-01T00:00:00Z".to_owned()); + let envelope_json = serde_json::to_string(&envelope) + .map_err(|error| format!("blob envelope serialization failed: {error}"))?; + let runtime = crate::config_payload::settings_from_config_blob( + &envelope_json, + &Task7SecretStore, + &StoreName::from("trusted_server_secrets"), + ) + .map_err(|error| format!("runtime settings resolution failed: {error:?}"))?; + + assert_eq!( + runtime.publisher.proxy_secret.expose(), + "fictional-proxy-secret-32-bytes-ok" + ); + assert_eq!( + runtime.ec.passphrase.expose(), + "fictional-ec-passphrase-32-bytes-ok" + ); + assert_eq!( + runtime.handlers[0].password.expose(), + "fictional-admin-password-32-bytes-ok" + ); + let integration_ids = task7_probe_source_integrations()?; + let profile_ids = task7_probe_source_profiles(None)?; + Ok(Task7HarnessObservation { + placeholder_paths, + integration_ids, + profile_ids, + consumer_literals, + failure_diagnostic, + }) + } + + fn task7_placeholder_observation( + settings: &Settings, + ) -> Result<(BTreeSet, String), String> { + let error = validate_settings_for_deploy(settings) + .expect_err("unmodified source template must reject its placeholders"); + let TrustedServerError::InsecureDefault { field } = error.current_context() else { + return Err(format!( + "unexpected source-template failure context: {:?}", + error.current_context() + )); + }; + let paths = field + .split(", ") + .map(str::to_owned) + .collect::>(); + let diagnostic = format!( + "unmodified template rejects {}", + paths.iter().cloned().collect::>().join(", ") + ); + Ok((paths, diagnostic)) + } + + fn task7_serialized_secret_literals( + data: &serde_json::Value, + ) -> Result, String> { + [ + "/publisher/proxy_secret", + "/ec/passphrase", + "/handlers/0/password", + ] + .into_iter() + .map(|pointer| { + data.pointer(pointer) + .and_then(serde_json::Value::as_str) + .map(str::to_owned) + .ok_or_else(|| format!("missing serialized secret-key literal at {pointer}")) + }) + .collect() + } + + fn task7_serialized_source_config(source: &str) -> Result { + let mut settings = toml::from_str::(source) + .map_err(|error| format!("source template parse failed: {error}"))? + .into_settings(); + task7_customize_nonsecret_values(&mut settings); + validate_settings_for_deploy(&settings) + .map_err(|error| format!("source deploy validation failed: {error:?}"))?; + let app_config = TrustedServerAppConfig::new(settings) + .map_err(|error| format!("source typed app config failed: {error:?}"))?; + serde_json::to_value(app_config) + .map_err(|error| format!("source app config serialization failed: {error}")) + } + + fn task7_resolve_envelope(data: serde_json::Value) -> Result { + let envelope = BlobEnvelope::new(data, "2026-01-01T00:00:00Z".to_owned()); + let envelope_json = serde_json::to_string(&envelope) + .map_err(|error| format!("blob envelope serialization failed: {error}"))?; + crate::config_payload::settings_from_config_blob( + &envelope_json, + &Task7SecretStore, + &StoreName::from("trusted_server_secrets"), + ) + .map_err(|error| format!("runtime settings resolution failed: {error:?}")) + } + + fn task7_probe_source_integrations() -> Result, String> { + let mut observed = BTreeSet::new(); + for (id, header) in task7_source_integration_blocks() { + let settings = task7_settings_with_source_integration_enabled(id, header); + validate_settings_for_deploy(&settings) + .map_err(|error| format!("source-derived {id} probe failed: {error:?}"))?; + observed.insert(id.to_owned()); + } + Ok(observed) + } + + fn task7_probe_source_profiles(omitted: Option<&str>) -> Result, String> { + let cases = task7_source_profile_cases()?; + let mut observed = BTreeSet::new(); + for (id, (positive, negative)) in cases { + if omitted == Some(id) { + continue; + } + let registration = crate::auction::profile::find_profile(id) + .ok_or_else(|| format!("source-derived profile {id} is not registered"))?; + let compiled = registration + .compile(&positive) + .map_err(|error| format!("source-derived profile {id} failed: {error:?}"))?; + if compiled.id() != id { + return Err(format!( + "source-derived profile {id} compiled as {}", + compiled.id() + )); + } + if registration.compile(&negative).is_ok() { + return Err(format!("source-derived profile {id} negative probe passed")); + } + observed.insert(id.to_owned()); + } + Ok(observed) + } + + fn task7_source_profile_cases() + -> Result, String> { + let source = toml::from_str::(EXAMPLE_TEMPLATE) + .map_err(|error| format!("source template profile parse failed: {error}"))? + .into_settings(); + let pbs = source + .auction + .providers + .values() + .find(|provider| provider.profile == "prebid-server") + .ok_or_else(|| "source template has no prebid-server provider".to_owned())?; + let aps_source = uncomment_block( + &uncomment_block(EXAMPLE_TEMPLATE, "[auction.providers.aps-main]"), + "[auction.providers.aps-main.profile_config]", + ); + let aps_settings = toml::from_str::(&aps_source) + .map_err(|error| format!("source APS provider parse failed: {error}"))? + .into_settings(); + let aps = aps_settings + .auction + .providers + .values() + .find(|provider| provider.profile == "aps") + .ok_or_else(|| "source template has no APS provider".to_owned())?; + let standard_source = uncomment_block( + &uncomment_block(EXAMPLE_TEMPLATE, "[auction.providers.standard-main]"), + "[auction.providers.standard-main.profile_config]", + ); + let standard_settings = toml::from_str::(&standard_source) + .map_err(|error| format!("source standard provider parse failed: {error}"))? + .into_settings(); + let standard = standard_settings + .auction + .providers + .values() + .find(|provider| provider.profile == "standard") + .ok_or_else(|| "source template has no standard provider".to_owned())?; + let mut malformed_standard = standard.profile_config.clone(); + let request_ext = malformed_standard + .as_object_mut() + .and_then(|object| object.get_mut("request_ext")) + .ok_or_else(|| "source standard profile does not exercise request_ext".to_owned())?; + *request_ext = serde_json::Value::String("not-an-object".to_owned()); + + Ok(BTreeMap::from([ + ( + "aps", + ( + aps.profile_config.clone(), + serde_json::json!({"debug": false}), + ), + ), + ( + "prebid-server", + ( + pbs.profile_config.clone(), + serde_json::json!({"unknown": true}), + ), + ), + ( + "standard", + (standard.profile_config.clone(), malformed_standard), + ), + ])) + } } diff --git a/crates/trusted-server-core/src/constants.rs b/crates/trusted-server-core/src/constants.rs index e1152b1e7..81bb828b6 100644 --- a/crates/trusted-server-core/src/constants.rs +++ b/crates/trusted-server-core/src/constants.rs @@ -1,45 +1,77 @@ +//! Canonical cookie, header, and environment-variable names. + use http::header::HeaderName; +/// Edge Cookie identifier cookie. pub const COOKIE_TS_EC: &str = "ts-ec"; /// Cookie written by the Trusted Server JS SDK containing a standard-base64-encoded /// JSON array of Extended User IDs (`[{ source, uids }]`) from identity providers. pub const COOKIE_TS_EIDS: &str = "ts-eids"; +/// Opt-in cookie for tester-only behavior. pub const COOKIE_TS_TESTER: &str = "ts-tester"; +/// Prebid shared-ID cookie. pub const COOKIE_SHAREDID: &str = "sharedId"; +/// Publisher-provided user identifier header. pub const HEADER_X_PUB_USER_ID: HeaderName = HeaderName::from_static("x-pub-user-id"); +/// Edge Cookie identifier header. pub const HEADER_X_TS_EC: HeaderName = HeaderName::from_static("x-ts-ec"); +/// Extended User IDs header. pub const HEADER_X_TS_EIDS: HeaderName = HeaderName::from_static("x-ts-eids"); +/// Edge Cookie consent-decision header. pub const HEADER_X_TS_EC_CONSENT: HeaderName = HeaderName::from_static("x-ts-ec-consent"); +/// Marker indicating the Extended User IDs value was truncated. pub const HEADER_X_TS_EIDS_TRUNCATED: HeaderName = HeaderName::from_static("x-ts-eids-truncated"); +/// Normalized advertising-consent header. pub const HEADER_X_CONSENT_ADVERTISING: HeaderName = HeaderName::from_static("x-consent-advertising"); +/// Standard client forwarding-chain header. pub const HEADER_X_FORWARDED_FOR: HeaderName = HeaderName::from_static("x-forwarded-for"); +/// Edge-derived client city header. pub const HEADER_X_GEO_CITY: HeaderName = HeaderName::from_static("x-geo-city"); +/// Edge-derived client continent header. pub const HEADER_X_GEO_CONTINENT: HeaderName = HeaderName::from_static("x-geo-continent"); +/// Edge-derived client latitude/longitude header. pub const HEADER_X_GEO_COORDINATES: HeaderName = HeaderName::from_static("x-geo-coordinates"); +/// Edge-derived client country header. pub const HEADER_X_GEO_COUNTRY: HeaderName = HeaderName::from_static("x-geo-country"); +/// Marker indicating whether edge geo data is available. pub const HEADER_X_GEO_INFO_AVAILABLE: HeaderName = HeaderName::from_static("x-geo-info-available"); +/// Edge-derived DMA or metro-code header. pub const HEADER_X_GEO_METRO_CODE: HeaderName = HeaderName::from_static("x-geo-metro-code"); +/// Edge-derived client region header. pub const HEADER_X_GEO_REGION: HeaderName = HeaderName::from_static("x-geo-region"); +/// Consent subject identifier header. pub const HEADER_X_SUBJECT_ID: HeaderName = HeaderName::from_static("x-subject-id"); +/// Request correlation identifier header. pub const HEADER_X_REQUEST_ID: HeaderName = HeaderName::from_static("x-request-id"); +/// Origin compression-preference hint. pub const HEADER_X_COMPRESS_HINT: HeaderName = HeaderName::from_static("x-compress-hint"); +/// Fastly point-of-presence diagnostic header. pub const HEADER_X_DEBUG_FASTLY_POP: HeaderName = HeaderName::from_static("x-debug-fastly-pop"); // Staging / version identification headers +/// Trusted Server deployment-version response header. pub const HEADER_X_TS_VERSION: HeaderName = HeaderName::from_static("x-ts-version"); +/// Trusted Server environment response header. pub const HEADER_X_TS_ENV: HeaderName = HeaderName::from_static("x-ts-env"); // Fastly environment variables +/// Fastly service-version environment variable. pub const ENV_FASTLY_SERVICE_VERSION: &str = "FASTLY_SERVICE_VERSION"; +/// Fastly staging-mode environment variable. pub const ENV_FASTLY_IS_STAGING: &str = "FASTLY_IS_STAGING"; // Common standard header names used across modules +/// Canonical `User-Agent` header name. pub const HEADER_USER_AGENT: HeaderName = HeaderName::from_static("user-agent"); +/// Canonical `Accept` header name. pub const HEADER_ACCEPT: HeaderName = HeaderName::from_static("accept"); +/// Canonical `Accept-Language` header name. pub const HEADER_ACCEPT_LANGUAGE: HeaderName = HeaderName::from_static("accept-language"); +/// Canonical `Accept-Encoding` header name. pub const HEADER_ACCEPT_ENCODING: HeaderName = HeaderName::from_static("accept-encoding"); +/// Canonical `Referer` header name. pub const HEADER_REFERER: HeaderName = HeaderName::from_static("referer"); /// TS-internal header names that must NOT be forwarded to downstream third-party services. @@ -77,10 +109,15 @@ pub const INTERNAL_HEADERS: &[&str] = &[ ]; // Consent-related cookie names +/// IAB TCF v2 consent-string cookie. pub const COOKIE_EUCONSENT_V2: &str = "euconsent-v2"; +/// IAB GPP consent-string cookie. pub const COOKIE_GPP: &str = "__gpp"; +/// IAB GPP section-identifier cookie. pub const COOKIE_GPP_SID: &str = "__gpp_sid"; +/// Legacy IAB US Privacy consent cookie. pub const COOKIE_US_PRIVACY: &str = "us_privacy"; // Consent-related header names +/// Global Privacy Control request header. pub const HEADER_SEC_GPC: HeaderName = HeaderName::from_static("sec-gpc"); diff --git a/crates/trusted-server-core/src/creative.rs b/crates/trusted-server-core/src/creative.rs index a4d641bdf..f1b10a3ce 100644 --- a/crates/trusted-server-core/src/creative.rs +++ b/crates/trusted-server-core/src/creative.rs @@ -589,7 +589,7 @@ pub fn rewrite_creative_html(settings: &Settings, markup: &str) -> String { /// Rewrite an HTML document proxied through `/first-party/proxy`. /// /// Same rewrite pass as [`rewrite_creative_html`], but bounded by the proxy's -/// own [`MAX_REWRITABLE_BODY_SIZE`] rather than the per-creative auction cap: +/// own `MAX_REWRITABLE_BODY_SIZE` rather than the per-creative auction cap: /// a proxied document is a whole page, not an `adm`, and legitimately exceeds /// 1 MiB. The creative runtime is still injected so click mediation survives. #[must_use] diff --git a/crates/trusted-server-core/src/creative_opportunities.rs b/crates/trusted-server-core/src/creative_opportunities.rs index 044d49ed3..7fe9929b3 100644 --- a/crates/trusted-server-core/src/creative_opportunities.rs +++ b/crates/trusted-server-core/src/creative_opportunities.rs @@ -709,7 +709,7 @@ impl CreativeOpportunitySlot { } /// Compile [`page_patterns`](Self::page_patterns) into the - /// [`compiled_patterns`](Self::compiled_patterns) cache. + /// private `compiled_patterns` cache. /// /// Patterns that fail to compile (either directly or after the `**`→`*` /// normalisation that [`matches_path`](Self::matches_path) does) are @@ -784,7 +784,7 @@ impl CreativeOpportunitySlot { /// /// # Performance /// - /// The hot path reads the [`compiled_unit`](Self::compiled_unit) cache. A + /// The hot path reads the private `compiled_unit` cache. A /// slot with an explicit `gam_unit_path` but no cache (built by hand, or /// deserialized without [`CreativeOpportunitiesConfig::compile_unit_templates`]) /// re-parses its template on every call — same fallback shape as diff --git a/crates/trusted-server-core/src/ec/admin.rs b/crates/trusted-server-core/src/ec/admin.rs index e44708742..32cfe538a 100644 --- a/crates/trusted-server-core/src/ec/admin.rs +++ b/crates/trusted-server-core/src/ec/admin.rs @@ -261,7 +261,7 @@ struct SkippedPartnerId { /// Resolves the EC ID from the path when present, falling back to the /// request's `ts-ec` cookie for the bare route. Responds: /// -/// - `200 OK` with an [`AdminEcLookupResponse`] JSON body when the key +/// - `200 OK` with an `AdminEcLookupResponse` JSON body when the key /// exists (including corrupt entries, which are reported with /// `entry_error` and `raw_body` instead of failing closed); /// - `400 Bad Request` when the resolved ID is not a valid EC ID; diff --git a/crates/trusted-server-core/src/ec/generation.rs b/crates/trusted-server-core/src/ec/generation.rs index 2924b7692..f94c16275 100644 --- a/crates/trusted-server-core/src/ec/generation.rs +++ b/crates/trusted-server-core/src/ec/generation.rs @@ -73,7 +73,8 @@ fn generate_random_suffix(length: usize) -> String { /// the client IP address, then appends a random suffix for additional /// uniqueness. The resulting format is `{64hex}.{6alnum}`. /// -/// **Important:** `client_ip` must be pre-normalized via [`extract_client_ip`]. +/// **Important:** `client_ip` must be pre-normalized via the internal +/// `normalize_ip` helper. /// Raw IPv6 addresses produce different hashes than their normalized /64 /// form, which would create duplicate identity graph entries. /// diff --git a/crates/trusted-server-core/src/ec/kv.rs b/crates/trusted-server-core/src/ec/kv.rs index 3572581ce..9c1112967 100644 --- a/crates/trusted-server-core/src/ec/kv.rs +++ b/crates/trusted-server-core/src/ec/kv.rs @@ -299,7 +299,7 @@ impl KvIdentityGraph { /// - **No existing key** — creates the entry (same as [`create`](Self::create)). /// - **Existing live entry** (`consent.ok = true`) — no-op, returns `Ok(())`. /// - **Existing tombstone** (`consent.ok = false`) — CAS overwrite with - /// the new entry. Retries up to [`MAX_CAS_RETRIES`] on conflict. + /// the new entry. Retries up to `MAX_CAS_RETRIES` on conflict. /// /// Called by `generate_if_needed()` instead of `create()` so that a /// user who re-consents within the 24-hour tombstone window recovers @@ -467,7 +467,7 @@ impl KvIdentityGraph { /// Atomically merges a partner ID into the existing entry. /// /// Uses CAS (generation markers) to avoid clobbering concurrent writes - /// from other partners. Retries up to [`MAX_CAS_RETRIES`] on conflict. + /// from other partners. Retries up to `MAX_CAS_RETRIES` on conflict. /// /// If the root entry does not exist, returns an error. This method /// intentionally fails closed to prevent phantom identity entries. @@ -475,7 +475,7 @@ impl KvIdentityGraph { /// # Errors /// /// Returns [`TrustedServerError::KvStore`] on store error or CAS - /// exhaustion after [`MAX_CAS_RETRIES`] attempts. + /// exhaustion after `MAX_CAS_RETRIES` attempts. pub fn upsert_partner_id( &self, ec_id: &str, @@ -660,7 +660,7 @@ impl KvIdentityGraph { /// Counts the number of keys sharing the same EC hash prefix. /// /// Uses the platform KV list API with a prefix filter, limited to - /// [`CLUSTER_LIST_LIMIT`] keys. If the limit is reached, the count + /// `CLUSTER_LIST_LIMIT` keys. If the limit is reached, the count /// is capped — the exact number beyond the limit is not meaningful /// for disambiguation. /// diff --git a/crates/trusted-server-core/src/ec/mod.rs b/crates/trusted-server-core/src/ec/mod.rs index 840ce90d3..b75e15c4f 100644 --- a/crates/trusted-server-core/src/ec/mod.rs +++ b/crates/trusted-server-core/src/ec/mod.rs @@ -359,7 +359,7 @@ impl EcContext { /// include the [`KvDevice`] record. The adapter derives these from /// `req.get_tls_ja4()`, `req.get_client_h2_fingerprint()`, and UA. /// - /// [`KvDevice`]: super::kv_types::KvDevice + /// [`KvDevice`]: crate::ec::kv_types::KvDevice /// [`generate_if_needed`]: Self::generate_if_needed pub fn set_device_signals(&mut self, signals: DeviceSignals) { self.device_signals = Some(signals); diff --git a/crates/trusted-server-core/src/host_rewrite.rs b/crates/trusted-server-core/src/host_rewrite.rs index 816dc52b8..c91705cc5 100644 --- a/crates/trusted-server-core/src/host_rewrite.rs +++ b/crates/trusted-server-core/src/host_rewrite.rs @@ -1,3 +1,8 @@ +//! Boundary-aware host replacement shared by HTML and RSC rewriting. +//! +//! The matcher refuses suffix and subdomain collisions and preserves numeric +//! ports, preventing broad string replacement from corrupting unrelated hosts. + /// Rewrite bare host occurrences (e.g. `origin.example.com/news`) only when the match is a full /// hostname token, not part of a larger hostname like `cdn.origin.example.com`. /// diff --git a/crates/trusted-server-core/src/html_processor.rs b/crates/trusted-server-core/src/html_processor.rs index 62ecab229..6d028f6ed 100644 --- a/crates/trusted-server-core/src/html_processor.rs +++ b/crates/trusted-server-core/src/html_processor.rs @@ -1,6 +1,8 @@ -//! Simplified HTML processor that combines URL replacement and integration injection +//! Streaming HTML transformation and integration injection. //! -//! This module provides a `StreamProcessor` implementation for HTML content. +//! The processor combines core URL rewriting with the enabled integration +//! registry. It streams when no final post-processor is registered and buffers +//! only when an integration explicitly requires whole-document context. use std::cell::Cell; use std::io; use std::rc::Rc; diff --git a/crates/trusted-server-core/src/http_util.rs b/crates/trusted-server-core/src/http_util.rs index 34656f63d..174dd9bf5 100644 --- a/crates/trusted-server-core/src/http_util.rs +++ b/crates/trusted-server-core/src/http_util.rs @@ -1,3 +1,8 @@ +//! Shared HTTP policy and cryptographic request helpers. +//! +//! This module centralizes internal-header filtering, request classification, +//! bounded body handling, client-IP trust, and encrypted token operations. + use base64::{Engine as _, engine::general_purpose::URL_SAFE_NO_PAD}; use chacha20poly1305::{XChaCha20Poly1305, XNonce, aead::Aead as _, aead::KeyInit as _}; use edgezero_core::body::Body as EdgeBody; diff --git a/crates/trusted-server-core/src/integrations/aps.rs b/crates/trusted-server-core/src/integrations/aps.rs index 6bffddf16..64ca0edc7 100644 --- a/crates/trusted-server-core/src/integrations/aps.rs +++ b/crates/trusted-server-core/src/integrations/aps.rs @@ -1963,6 +1963,8 @@ pub fn register_providers( #[cfg(test)] mod tests { + use std::collections::BTreeSet; + use super::*; use crate::auction::test_support::canonical_parity_auction_request; use crate::auction::types::{ @@ -1975,6 +1977,53 @@ mod tests { use crate::test_support::tests::create_test_settings; use serde_json::json; + #[derive(Clone, Debug, Deserialize, Eq, Ord, PartialEq, PartialOrd)] + struct Task7CompanionReceipt { + source: String, + symbol: String, + kind: String, + value: Option, + positive_probe: String, + negative_probe: String, + } + + #[derive(Deserialize)] + struct Task7CompanionManifest { + companions: Vec, + } + + macro_rules! task7_companion_receipt { + ($positive:ident, $negative:ident, $symbol:literal, $kind:literal, $value:expr, $positive_body:block, $negative_body:block) => {{ + fn $positive() $positive_body + fn $negative() $negative_body + $positive(); + $negative(); + let value: Option<&str> = $value; + Task7CompanionReceipt { + source: "crates/trusted-server-core/src/integrations/aps.rs".to_owned(), + symbol: $symbol.to_owned(), + kind: $kind.to_owned(), + value: value.map(str::to_owned), + positive_probe: stringify!($positive).to_owned(), + negative_probe: stringify!($negative).to_owned(), + } + }}; + ($positive:ident, $negative:ident, $symbol:literal, $kind:literal, dynamic $value:expr, $positive_body:block, $negative_body:block) => {{ + fn $positive() $positive_body + fn $negative() $negative_body + $positive(); + $negative(); + Task7CompanionReceipt { + source: "crates/trusted-server-core/src/integrations/aps.rs".to_owned(), + symbol: $symbol.to_owned(), + kind: $kind.to_owned(), + value: Some($value), + positive_probe: stringify!($positive).to_owned(), + negative_probe: stringify!($negative).to_owned(), + } + }}; + } + fn config() -> LegacyApsProviderConfig { LegacyApsProviderConfig { enabled: true, @@ -3248,4 +3297,29 @@ mod tests { assert!(APS_RENDERER_CSP.contains("sandbox allow-forms")); assert!(!APS_RENDERER_CSP.contains("allow-same-origin")); } + + #[test] + fn task7_aps_companions_are_exact_and_compiled_per_record() { + let actual = BTreeSet::from([task7_companion_receipt!( + task7_crates_trusted_server_core_src_integrations_aps_rs_deserialize_account_id_deserializer_positive, + task7_crates_trusted_server_core_src_integrations_aps_rs_deserialize_account_id_deserializer_negative, + "deserialize_account_id", "deserializer", None, + { + let numeric: ApsProfileConfig = serde_json::from_value(json!({"account_id":12345})).expect("numeric account ID should deserialize"); + assert_eq!(numeric.account_id, "12345"); + }, + { serde_json::from_value::(json!({"account_id":[]})).expect_err("array account ID should fail"); } + )]); + let checked: Task7CompanionManifest = toml::from_str(include_str!(concat!( + env!("CARGO_MANIFEST_DIR"), + "/../../tools/docs-parity/manifests/settings-companions.toml" + ))) + .expect("checked Task 7 companion manifest should parse"); + let expected = checked + .companions + .into_iter() + .filter(|record| record.source == "crates/trusted-server-core/src/integrations/aps.rs") + .collect::>(); + assert_eq!(actual, expected, "compiled APS receipts must be exact"); + } } diff --git a/crates/trusted-server-core/src/integrations/datadome.rs b/crates/trusted-server-core/src/integrations/datadome.rs index ba2a217c9..b0f2fdefa 100644 --- a/crates/trusted-server-core/src/integrations/datadome.rs +++ b/crates/trusted-server-core/src/integrations/datadome.rs @@ -3,6 +3,8 @@ //! This module provides transparent proxying for `DataDome`'s JavaScript tag and signal //! collection API, enabling first-party bot protection while maintaining the permissionless //! Trusted Server approach (no DNS/CNAME changes required). +//! Runtime registration owns the proxy routes, script rewriting, request filter, +//! and client-tag suppression decision as one integration contract. //! //! # Overview //! diff --git a/crates/trusted-server-core/src/integrations/nextjs/html_post_process.rs b/crates/trusted-server-core/src/integrations/nextjs/html_post_process.rs index 53e573db8..14010cb51 100644 --- a/crates/trusted-server-core/src/integrations/nextjs/html_post_process.rs +++ b/crates/trusted-server-core/src/integrations/nextjs/html_post_process.rs @@ -1,3 +1,8 @@ +//! Buffered final pass for cross-script Next.js RSC payload rewriting. +//! +//! Streaming hooks leave placeholders for complete payloads; this processor +//! combines them under a configured size limit and restores rewritten scripts. + use std::cell::{Cell, RefCell}; use std::rc::Rc; use std::sync::Arc; diff --git a/crates/trusted-server-core/src/integrations/nextjs/mod.rs b/crates/trusted-server-core/src/integrations/nextjs/mod.rs index 015ba2475..7b32dfe9a 100644 --- a/crates/trusted-server-core/src/integrations/nextjs/mod.rs +++ b/crates/trusted-server-core/src/integrations/nextjs/mod.rs @@ -1,3 +1,8 @@ +//! Next.js origin rewriting for `__NEXT_DATA__` and RSC Flight payloads. +//! +//! Streaming script hooks handle complete payloads, while a bounded buffered +//! post-process joins fragmented RSC scripts that span HTML chunks. + use std::sync::Arc; use error_stack::Report; diff --git a/crates/trusted-server-core/src/integrations/nextjs/rsc.rs b/crates/trusted-server-core/src/integrations/nextjs/rsc.rs index fbd2c693c..e8968190d 100644 --- a/crates/trusted-server-core/src/integrations/nextjs/rsc.rs +++ b/crates/trusted-server-core/src/integrations/nextjs/rsc.rs @@ -1,3 +1,8 @@ +//! Escape-aware rewriting of combined Next.js RSC script payloads. +//! +//! The rewriter preserves JavaScript escapes and recalculates Flight `T`-chunk +//! byte lengths after origin URLs change. + use std::sync::LazyLock; use regex::Regex; diff --git a/crates/trusted-server-core/src/integrations/nextjs/rsc_placeholders.rs b/crates/trusted-server-core/src/integrations/nextjs/rsc_placeholders.rs index 117b8f982..5a7aefb5b 100644 --- a/crates/trusted-server-core/src/integrations/nextjs/rsc_placeholders.rs +++ b/crates/trusted-server-core/src/integrations/nextjs/rsc_placeholders.rs @@ -1,3 +1,8 @@ +//! Streaming placeholders for complete Next.js RSC push payloads. +//! +//! Payloads are removed during element rewriting and restored by the final HTML +//! post-processor so cross-script rewriting can operate on a coherent stream. + use std::sync::{Arc, Mutex}; use crate::integrations::{ diff --git a/crates/trusted-server-core/src/integrations/nextjs/script_rewriter.rs b/crates/trusted-server-core/src/integrations/nextjs/script_rewriter.rs index fb54ce1b3..3bff4b5a5 100644 --- a/crates/trusted-server-core/src/integrations/nextjs/script_rewriter.rs +++ b/crates/trusted-server-core/src/integrations/nextjs/script_rewriter.rs @@ -1,3 +1,8 @@ +//! Next.js script-element rewriting. +//! +//! This module rewrites origin URLs in `__NEXT_DATA__` and buffers text-node +//! fragments only until `lol_html` signals the element is complete. + use std::sync::{Arc, Mutex}; use error_stack::Report; diff --git a/crates/trusted-server-core/src/integrations/nextjs/shared.rs b/crates/trusted-server-core/src/integrations/nextjs/shared.rs index 7b88aa0ae..83ab5204a 100644 --- a/crates/trusted-server-core/src/integrations/nextjs/shared.rs +++ b/crates/trusted-server-core/src/integrations/nextjs/shared.rs @@ -1,4 +1,7 @@ -//! Shared utilities for Next.js integration modules. +//! Shared parsing and boundary-safe URL rewriting for Next.js integration modules. +//! +//! These helpers identify RSC push payloads, decode JavaScript string escapes, +//! and delegate bare-host replacement to the core boundary matcher. use std::borrow::Cow; use std::cell::RefCell; diff --git a/crates/trusted-server-core/src/integrations/prebid.rs b/crates/trusted-server-core/src/integrations/prebid.rs index 31b265cae..e800ff82e 100644 --- a/crates/trusted-server-core/src/integrations/prebid.rs +++ b/crates/trusted-server-core/src/integrations/prebid.rs @@ -1,3 +1,9 @@ +//! Prebid.js and Prebid Server integration. +//! +//! This module registers browser assets and proxy routes, builds provider +//! requests from auction plans, forwards bounded consent and identity context, +//! and normalizes Prebid Server responses for orchestration. + use std::collections::HashMap; #[cfg(test)] use std::collections::HashSet; @@ -3425,10 +3431,43 @@ mod tests { use bytes::Bytes; use http::Method; use serde_json::json; - use std::collections::{BTreeMap, HashMap}; + use std::collections::{BTreeMap, BTreeSet, HashMap}; use std::io::Cursor; use std::str::FromStr as _; + #[derive(Clone, Debug, Deserialize, Eq, Ord, PartialEq, PartialOrd)] + struct Task7CompanionReceipt { + source: String, + symbol: String, + kind: String, + value: Option, + positive_probe: String, + negative_probe: String, + } + + #[derive(Deserialize)] + struct Task7CompanionManifest { + companions: Vec, + } + + macro_rules! task7_companion_receipt { + ($positive:ident, $negative:ident, $symbol:literal, $kind:literal, $value:expr, $positive_body:block, $negative_body:block) => {{ + fn $positive() $positive_body + fn $negative() $negative_body + $positive(); + $negative(); + let value: Option<&str> = $value; + Task7CompanionReceipt { + source: "crates/trusted-server-core/src/integrations/prebid.rs".to_owned(), + symbol: $symbol.to_owned(), + kind: $kind.to_owned(), + value: value.map(str::to_owned), + positive_probe: stringify!($positive).to_owned(), + negative_probe: stringify!($negative).to_owned(), + } + }}; + } + #[test] fn external_bundle_sha256_validation_matches_hex_pattern() { use validator::Validate as _; @@ -9221,4 +9260,61 @@ set = { networkId = 42 } "should preserve the complete enabled PBS wire shape" ); } + + #[test] + fn task7_prebid_companions_are_exact_and_compiled_per_record() { + let actual = BTreeSet::from([ + task7_companion_receipt!( + task7_crates_trusted_server_core_src_integrations_prebid_rs_default_script_patterns_default_positive, + task7_crates_trusted_server_core_src_integrations_prebid_rs_default_script_patterns_default_negative, + "default_script_patterns", "default", Some("[/prebid.js,/prebid.min.js,/prebidjs.js,/prebidjs.min.js]"), + { assert_eq!(default_script_patterns(), ["/prebid.js", "/prebid.min.js", "/prebidjs.js", "/prebidjs.min.js"]); }, + { assert_ne!(default_script_patterns(), ["/prebid.js"]); } + ), + task7_companion_receipt!( + task7_crates_trusted_server_core_src_integrations_prebid_rs_crate_settings_vec_from_seq_or_map_deserializer_positive, + task7_crates_trusted_server_core_src_integrations_prebid_rs_crate_settings_vec_from_seq_or_map_deserializer_negative, + "crate::settings::vec_from_seq_or_map", "deserializer", None, + { + let parsed: PrebidIntegrationConfig = toml::from_str("script_patterns = { 0 = \"first\", 1 = \"second\" }").expect("map script patterns should deserialize"); + assert_eq!(parsed.script_patterns, ["first", "second"]); + }, + { toml::from_str::("script_patterns = 42").expect_err("scalar script patterns should fail"); } + ), + task7_companion_receipt!( + task7_crates_trusted_server_core_src_integrations_prebid_rs_validate_external_bundle_url_validator_positive, + task7_crates_trusted_server_core_src_integrations_prebid_rs_validate_external_bundle_url_validator_negative, + "validate_external_bundle_url", "validator", None, + { validate_external_bundle_url("https://assets.example/prebid.js").expect("HTTPS URL should validate"); }, + { validate_external_bundle_url("http://assets.example/prebid.js").expect_err("insecure URL should fail"); } + ), + task7_companion_receipt!( + task7_crates_trusted_server_core_src_integrations_prebid_rs_validate_external_bundle_sri_validator_positive, + task7_crates_trusted_server_core_src_integrations_prebid_rs_validate_external_bundle_sri_validator_negative, + "validate_external_bundle_sri", "validator", None, + { validate_external_bundle_sri(&test_sri("sha384", &[0; 48])).expect("algorithm-sized SRI should validate"); }, + { validate_external_bundle_sri("sha384-AAAA").expect_err("wrong-size SRI should fail"); } + ), + task7_companion_receipt!( + task7_crates_trusted_server_core_src_integrations_prebid_rs_validate_excluded_gam_ad_unit_path_suffixes_validator_positive, + task7_crates_trusted_server_core_src_integrations_prebid_rs_validate_excluded_gam_ad_unit_path_suffixes_validator_negative, + "validate_excluded_gam_ad_unit_path_suffixes", "validator", None, + { validate_excluded_gam_ad_unit_path_suffixes(&["/excluded".to_owned()]).expect("non-root suffix should validate"); }, + { validate_excluded_gam_ad_unit_path_suffixes(&["/".to_owned()]).expect_err("root suffix should fail"); } + ), + ]); + let checked: Task7CompanionManifest = toml::from_str(include_str!(concat!( + env!("CARGO_MANIFEST_DIR"), + "/../../tools/docs-parity/manifests/settings-companions.toml" + ))) + .expect("checked Task 7 companion manifest should parse"); + let expected = checked + .companions + .into_iter() + .filter(|record| { + record.source == "crates/trusted-server-core/src/integrations/prebid.rs" + }) + .collect::>(); + assert_eq!(actual, expected, "compiled Prebid receipts must be exact"); + } } diff --git a/crates/trusted-server-core/src/integrations/registry.rs b/crates/trusted-server-core/src/integrations/registry.rs index 5b11b105e..cda13c90b 100644 --- a/crates/trusted-server-core/src/integrations/registry.rs +++ b/crates/trusted-server-core/src/integrations/registry.rs @@ -1,3 +1,8 @@ +//! Runtime registry for enabled integrations and their rewrite hooks. +//! +//! Registration collects routes, request filters, auction hooks, script and +//! attribute rewriters, and HTML post-processors in deterministic order. + use std::any::{Any, TypeId}; use std::collections::BTreeMap; use std::sync::{Arc, Mutex}; @@ -1383,10 +1388,16 @@ impl IntegrationRegistry { #[cfg(test)] mod tests { + use std::collections::BTreeSet; + use super::*; + use crate::auction::plan::AuctionPlanConfig; + use crate::auction::{AuctionPlan, NotificationConfig, ProviderConfig, RoutingMode}; use crate::constants::COOKIE_TS_EC; + use crate::integrations::aps::{ApsConfig, ApsRenderingMode}; use crate::platform::test_support::noop_services; use http::{HeaderValue, StatusCode, header}; + use serde::Deserialize; struct DefaultMetadataHeadInjector; @@ -2364,4 +2375,675 @@ mod tests { "should reconstruct full module list from immediate + deferred" ); } + + #[test] + fn task8_behavioral_capability_matrix_matches_checked_records_exactly() { + let manifest = task8_checked_manifest(); + task8_validate_bundle_receipt(&manifest.js_bundle_ids) + .expect("compiled JS bundles must equal the reviewed bundle inventory"); + let observed = task8_observed_capabilities(); + assert_eq!( + observed, + manifest.capabilities.into_iter().collect(), + "real integration registrations across every checked predicate must equal the reviewed capability records" + ); + let mut observed_loading = BTreeMap::new(); + for capability in &observed { + if capability.js_mode != "none" { + let previous = + observed_loading.insert(capability.id.clone(), capability.js_mode.clone()); + assert!( + previous + .as_ref() + .is_none_or(|mode| mode == &capability.js_mode), + "one integration must not report conflicting JS loading modes" + ); + } + } + assert_eq!( + observed_loading, + manifest + .loading_modes + .into_iter() + .map(|row| (row.id, row.mode)) + .collect(), + "real registration loading modes must equal the reviewed loading records" + ); + } + + #[test] + fn task8_predicate_negative_matrix_matches_registration_contracts() { + for (id, mut config) in task8_builder_fixtures() { + config["enabled"] = serde_json::Value::Bool(false); + let mut settings = crate::test_support::tests::create_test_settings(); + settings + .integrations + .insert_config(id, &config) + .expect("should insert disabled Task 8 integration config"); + let registration = crate::integrations::builders() + .iter() + .find(|builder| builder.id == id) + .and_then(|builder| { + (builder.build)(&settings).expect("should evaluate disabled integration") + }); + assert!( + registration.is_none(), + "disabled {id} must not register routes or hooks" + ); + } + let mut datadome_settings = crate::test_support::tests::create_test_settings(); + datadome_settings + .integrations + .insert_config( + "datadome", + &serde_json::json!({"enabled": false, "enable_protection": true}), + ) + .expect("should insert disabled DataDome config"); + assert!( + crate::integrations::builders() + .iter() + .find(|builder| builder.id == "datadome") + .and_then(|builder| { + (builder.build)(&datadome_settings).expect("should evaluate disabled DataDome") + }) + .is_none(), + "disabled DataDome must register neither proxy hooks nor protection filters" + ); + + let mut settings = crate::test_support::tests::create_test_settings(); + settings + .integrations + .insert_config( + "prebid", + &serde_json::json!({ + "enabled": false, + "external_bundle_url": "https://assets.example/prebid/trusted-prebid.js" + }), + ) + .expect("should insert disabled Prebid config"); + let plan = crate::auction::compile_auction_plan(&settings) + .expect("should compile plan with disabled Prebid"); + assert!( + crate::integrations::prebid::register_for_plan(&settings, &plan) + .expect("should evaluate disabled Prebid") + .is_none(), + "disabled Prebid must not register" + ); + + settings + .integrations + .insert_config( + "aps", + &ApsConfig { + enabled: false, + rendering_mode: ApsRenderingMode::TrustedServer, + }, + ) + .expect("should insert disabled APS config"); + assert!( + crate::integrations::aps::register_for_plan(&settings, &plan) + .expect("should evaluate APS without an APS plan profile") + .is_none(), + "APS browser config alone must not register without an APS plan profile" + ); + let aps_plan = AuctionPlan::compile(AuctionPlanConfig { + timeout_ms: 1_000, + providers: BTreeMap::from([( + "aps-disabled-fixture" + .parse() + .expect("should parse disabled APS provider ID"), + ProviderConfig { + protocol: "openrtb-2.6".to_owned(), + profile: "aps".to_owned(), + endpoint: "https://aps.example/openrtb".to_owned(), + timeout_ms: None, + routing: RoutingMode::AllEligible, + notifications: NotificationConfig::default(), + profile_config: serde_json::json!({"account_id": "example-account"}), + }, + )]), + ..AuctionPlanConfig::default() + }) + .expect("should compile a plan containing an APS profile"); + assert!( + crate::integrations::aps::register_for_plan(&settings, &aps_plan) + .expect("should evaluate APS with a plan profile") + .is_some(), + "APS registration is plan-backed; an APS profile registers even when the independent browser config is disabled" + ); + + settings + .integrations + .insert_config( + "adserver_mock", + &serde_json::json!({ + "enabled": false, + "endpoint": "https://adserver.example/mediate" + }), + ) + .expect("should insert disabled mediator config"); + assert!( + crate::integrations::adserver_mock::register_providers(&settings) + .expect("should evaluate disabled mediator") + .is_empty(), + "disabled mediator must not register providers" + ); + } + + #[test] + fn task8_compiled_receipts_reject_duplicate_capabilities_routes_and_hooks() { + for axis in [ + "proxy_routes", + "attribute_rewriters", + "script_rewriters", + "head_injectors", + "post_processors", + "request_filters", + "providers", + ] { + assert!( + task8_unique_receipt_values(axis, ["same", "same"]).is_err(), + "compiled {axis} observations must retain and reject duplicate cardinality" + ); + } + + let capability = Task8Capability { + id: "fixture".to_owned(), + predicate: "enabled=true".to_owned(), + proxy_routes: BTreeSet::new(), + attribute_rewriters: BTreeSet::new(), + script_rewriters: BTreeSet::new(), + head_injectors: BTreeSet::new(), + post_processors: BTreeSet::new(), + request_filters: BTreeSet::new(), + providers: BTreeSet::new(), + js_mode: "none".to_owned(), + }; + let mut capabilities = BTreeSet::new(); + task8_insert_capability(&mut capabilities, capability.clone()) + .expect("should insert the first compiled capability observation"); + assert!( + task8_insert_capability(&mut capabilities, capability).is_err(), + "compiled capability observations must reject duplicate rows" + ); + + let compiled = trusted_server_js::all_module_ids() + .into_iter() + .map(str::to_owned) + .collect::>(); + let mut duplicate_bundle = compiled.clone(); + duplicate_bundle.push(compiled[0].clone()); + assert!(task8_validate_bundle_receipt(&duplicate_bundle).is_err()); + let mut missing_bundle = compiled.clone(); + missing_bundle.pop(); + assert!(task8_validate_bundle_receipt(&missing_bundle).is_err()); + let mut nonexistent_bundle = compiled; + nonexistent_bundle.push("nonexistent".to_owned()); + assert!(task8_validate_bundle_receipt(&nonexistent_bundle).is_err()); + } + + #[derive(Clone, Debug, Deserialize, Eq, Ord, PartialEq, PartialOrd)] + #[serde(deny_unknown_fields)] + struct Task8Capability { + id: String, + predicate: String, + #[serde(default)] + proxy_routes: BTreeSet, + #[serde(default)] + attribute_rewriters: BTreeSet, + #[serde(default)] + script_rewriters: BTreeSet, + #[serde(default)] + head_injectors: BTreeSet, + #[serde(default)] + post_processors: BTreeSet, + #[serde(default)] + request_filters: BTreeSet, + #[serde(default)] + providers: BTreeSet, + js_mode: String, + } + + #[derive(Deserialize)] + struct Task8Manifest { + #[serde(default)] + js_bundle_ids: Vec, + #[serde(default)] + capabilities: Vec, + #[serde(default)] + loading_modes: Vec, + } + + #[derive(Deserialize)] + struct Task8Loading { + id: String, + mode: String, + } + + fn task8_checked_manifest() -> Task8Manifest { + toml::from_str(include_str!(concat!( + env!("CARGO_MANIFEST_DIR"), + "/../../tools/docs-parity/manifests/integrations.toml" + ))) + .expect("should parse checked Task 8 integration manifest") + } + + fn task8_unique_receipt_values(axis: &str, values: I) -> Result, String> + where + I: IntoIterator, + S: AsRef, + { + let values = values + .into_iter() + .map(|value| value.as_ref().to_owned()) + .collect::>(); + let count = values.len(); + let unique = values.into_iter().collect::>(); + if unique.len() != count { + Err(format!("duplicate compiled {axis} observation")) + } else { + Ok(unique) + } + } + + fn task8_validate_bundle_receipt(expected: &[String]) -> Result<(), String> { + let compiled = trusted_server_js::all_module_ids(); + let compiled_set = task8_unique_receipt_values("JS bundle IDs", compiled.iter().copied())?; + let expected_set = task8_unique_receipt_values("checked JS bundle IDs", expected)?; + if compiled_set != expected_set { + return Err("compiled JS bundle inventory differs".to_owned()); + } + for id in expected { + if trusted_server_js::module_bundle(id).is_none() { + return Err(format!("checked JS bundle `{id}` has no emitted bundle")); + } + } + Ok(()) + } + + fn task8_insert_capability( + observations: &mut BTreeSet, + capability: Task8Capability, + ) -> Result<(), String> { + if observations.insert(capability) { + Ok(()) + } else { + Err("duplicate compiled capability observation".to_owned()) + } + } + + fn task8_observed_capabilities() -> BTreeSet { + let mut observations = BTreeSet::new(); + let empty_registry = IntegrationRegistry::empty_for_tests(); + let creative_mode = if empty_registry.js_module_ids_immediate() == ["creative"] + && empty_registry.js_module_ids_deferred().is_empty() + && trusted_server_js::module_bundle("creative").is_some() + { + "bundled" + } else { + "invalid" + }; + task8_insert_capability( + &mut observations, + Task8Capability { + id: "creative".to_owned(), + predicate: "always".to_owned(), + proxy_routes: BTreeSet::new(), + attribute_rewriters: BTreeSet::new(), + script_rewriters: BTreeSet::new(), + head_injectors: BTreeSet::new(), + post_processors: BTreeSet::new(), + request_filters: BTreeSet::new(), + providers: BTreeSet::new(), + js_mode: creative_mode.to_owned(), + }, + ) + .expect("should insert the creative capability observation"); + + for (id, config) in task8_builder_fixtures() { + if id == "didomi" { + continue; + } + task8_insert_capability( + &mut observations, + task8_builder_capability(id, &config, "enabled=true"), + ) + .expect("should insert a unique settings-builder capability observation"); + } + for capability in [ + task8_builder_capability( + "datadome", + &serde_json::json!({"enabled": true, "enable_protection": false}), + "enabled=true;enable_protection=false", + ), + task8_builder_capability( + "datadome", + &serde_json::json!({ + "enabled": true, + "enable_protection": true, + "server_side_key_secret_name": "example-key-name" + }), + "enabled=true;enable_protection=true", + ), + task8_didomi_capability(), + task8_prebid_capability(), + task8_aps_capability(ApsRenderingMode::TrustedServer), + task8_aps_capability(ApsRenderingMode::PublisherNative), + task8_mediator_capability(), + ] { + task8_insert_capability(&mut observations, capability) + .expect("should insert a unique compiled capability observation"); + } + observations + } + + fn task8_builder_fixtures() -> Vec<(&'static str, serde_json::Value)> { + vec![ + ( + "testlight", + serde_json::json!({ + "enabled": true, + "endpoint": "https://testlight.example/auction" + }), + ), + ("nextjs", serde_json::json!({"enabled": true})), + ( + "permutive", + serde_json::json!({ + "enabled": true, + "organization_id": "example-organization", + "workspace_id": "example-workspace" + }), + ), + ( + "lockr", + serde_json::json!({"enabled": true, "app_id": "example-app"}), + ), + ("didomi", serde_json::json!({"enabled": true})), + ("sourcepoint", serde_json::json!({"enabled": true})), + ("osano", serde_json::json!({"enabled": true})), + ( + "google_tag_manager", + serde_json::json!({"enabled": true, "container_id": "GTM-EXAMPLE"}), + ), + ("gpt", serde_json::json!({"enabled": true})), + ("gpt_diagnostics", serde_json::json!({"enabled": true})), + ] + } + + fn task8_builder_capability( + id: &'static str, + config: &serde_json::Value, + predicate: &str, + ) -> Task8Capability { + let mut settings = crate::test_support::tests::create_test_settings(); + settings + .integrations + .insert_config(id, config) + .expect("should insert Task 8 integration config"); + let registration = crate::integrations::builders() + .iter() + .find(|builder| builder.id == id) + .and_then(|builder| (builder.build)(&settings).expect("should run integration builder")) + .expect("enabled Task 8 integration should register"); + task8_capability(®istration, predicate) + } + + fn task8_prebid_capability() -> Task8Capability { + let default_capability = task8_prebid_capability_for_config(&serde_json::json!({ + "enabled": true, + "external_bundle_url": "https://assets.example/prebid/trusted-prebid.js" + })); + assert_eq!( + default_capability.proxy_routes, + BTreeSet::from([ + "GET /integrations/prebid/bundle.js".to_owned(), + "GET /prebid.js".to_owned(), + "GET /prebid.min.js".to_owned(), + "GET /prebidjs.js".to_owned(), + "GET /prebidjs.min.js".to_owned(), + ]), + "omitting Prebid script_patterns must register the reviewed default routes" + ); + + let mut capability = task8_prebid_capability_for_config(&serde_json::json!({ + "enabled": true, + "external_bundle_url": "https://assets.example/prebid/trusted-prebid.js", + "script_patterns": ["/task8-prebid.js", "/assets/prebid/{*rest}"] + })); + assert_eq!( + capability.proxy_routes, + BTreeSet::from([ + "GET /assets/prebid/{*rest}".to_owned(), + "GET /integrations/prebid/bundle.js".to_owned(), + "GET /task8-prebid.js".to_owned(), + ]), + "custom Prebid script_patterns must replace the default script routes" + ); + capability.predicate = "enabled=true;script_patterns=config-derived".to_owned(); + capability.proxy_routes = BTreeSet::from([ + "GET /integrations/prebid/bundle.js".to_owned(), + "GET ".to_owned(), + ]); + capability + } + + fn task8_prebid_capability_for_config(config: &serde_json::Value) -> Task8Capability { + let mut settings = crate::test_support::tests::create_test_settings(); + settings + .integrations + .insert_config("prebid", config) + .expect("should insert Task 8 Prebid config"); + let plan = crate::auction::compile_auction_plan(&settings) + .expect("should compile Task 8 Prebid plan"); + let registration = crate::integrations::prebid::register_for_plan(&settings, &plan) + .expect("should run plan-backed Prebid registration") + .expect("enabled Prebid should register"); + task8_capability(®istration, "enabled=true") + } + + fn task8_didomi_capability() -> Task8Capability { + let default_capability = task8_builder_capability( + "didomi", + &serde_json::json!({"enabled": true}), + "enabled=true", + ); + assert_eq!( + default_capability.proxy_routes, + BTreeSet::from([ + "GET /integrations/didomi/consent/*".to_owned(), + "POST /integrations/didomi/consent/*".to_owned(), + ]), + "omitting Didomi proxy_path must register the reviewed default prefix" + ); + + let mut capability = task8_builder_capability( + "didomi", + &serde_json::json!({ + "enabled": true, + "proxy_path": "task8/didomi-consent" + }), + "enabled=true", + ); + assert_eq!( + capability.proxy_routes, + BTreeSet::from([ + "GET /task8/didomi-consent/*".to_owned(), + "POST /task8/didomi-consent/*".to_owned(), + ]), + "custom Didomi proxy_path must replace the default route prefix" + ); + capability.predicate = + "enabled=true;prefix=proxy_path||/integrations/didomi/consent".to_owned(); + capability.proxy_routes = + BTreeSet::from(["GET /*".to_owned(), "POST /*".to_owned()]); + capability + } + + fn task8_aps_capability(rendering_mode: ApsRenderingMode) -> Task8Capability { + let mut settings = crate::test_support::tests::create_test_settings(); + settings + .integrations + .insert_config( + "aps", + &ApsConfig { + enabled: true, + rendering_mode, + }, + ) + .expect("should insert Task 8 APS config"); + let plan = AuctionPlan::compile(AuctionPlanConfig { + timeout_ms: 1_000, + providers: BTreeMap::from([( + "aps-fixture" + .parse() + .expect("should parse Task 8 APS provider ID"), + ProviderConfig { + protocol: "openrtb-2.6".to_owned(), + profile: "aps".to_owned(), + endpoint: "https://aps.example/openrtb".to_owned(), + timeout_ms: None, + routing: RoutingMode::AllEligible, + notifications: NotificationConfig::default(), + profile_config: serde_json::json!({"account_id": "example-account"}), + }, + )]), + ..AuctionPlanConfig::default() + }) + .expect("should compile Task 8 APS plan"); + let registration = crate::integrations::aps::register_for_plan(&settings, &plan) + .expect("should run plan-backed APS registration") + .expect("APS profile should register the renderer integration"); + task8_capability( + ®istration, + match rendering_mode { + ApsRenderingMode::TrustedServer => { + "plan.has_profile(aps);rendering_mode=trusted_server" + } + ApsRenderingMode::PublisherNative => { + "plan.has_profile(aps);rendering_mode=publisher_native" + } + }, + ) + } + + fn task8_mediator_capability() -> Task8Capability { + let mut settings = crate::test_support::tests::create_test_settings(); + settings + .integrations + .insert_config( + "adserver_mock", + &serde_json::json!({ + "enabled": true, + "endpoint": "https://adserver.example/mediate" + }), + ) + .expect("should insert Task 8 mediator config"); + let providers = task8_unique_receipt_values( + "providers", + crate::integrations::adserver_mock::register_providers(&settings) + .expect("should register Task 8 mediator") + .into_iter() + .map(|provider| provider.provider_name().to_owned()), + ) + .expect("should observe unique mediator providers"); + Task8Capability { + id: "adserver_mock".to_owned(), + predicate: "auction.mediator=adserver_mock;enabled=true".to_owned(), + proxy_routes: BTreeSet::new(), + attribute_rewriters: BTreeSet::new(), + script_rewriters: BTreeSet::new(), + head_injectors: BTreeSet::new(), + post_processors: BTreeSet::new(), + request_filters: BTreeSet::new(), + providers, + js_mode: "none".to_owned(), + } + } + + fn task8_capability( + registration: &IntegrationRegistration, + predicate: &str, + ) -> Task8Capability { + let id = registration.integration_id; + let proxy_routes = task8_unique_receipt_values( + "proxy_routes", + registration + .proxies + .iter() + .flat_map(|proxy| proxy.routes()) + .map(|route| format!("{} {}", route.method, route.path)), + ) + .expect("should observe unique integration proxy routes"); + let attribute_rewriters = task8_unique_receipt_values( + "attribute_rewriters", + registration + .attribute_rewriters + .iter() + .map(|rewriter| rewriter.integration_id()), + ) + .expect("should observe unique integration attribute rewriters"); + let script_rewriters = task8_unique_receipt_values( + "script_rewriters", + registration + .script_rewriters + .iter() + .map(|rewriter| format!("{}:{}", rewriter.integration_id(), rewriter.selector())), + ) + .expect("should observe unique integration script rewriters"); + let head_injectors = task8_unique_receipt_values( + "head_injectors", + registration + .head_injectors + .iter() + .map(|injector| injector.integration_id()), + ) + .expect("should observe unique integration head injectors"); + let post_processors = task8_unique_receipt_values( + "post_processors", + registration + .html_post_processors + .iter() + .map(|processor| processor.integration_id()), + ) + .expect("should observe unique integration post-processors"); + let request_filters = task8_unique_receipt_values( + "request_filters", + registration + .request_filters + .iter() + .map(|filter| filter.integration_id()), + ) + .expect("should observe unique integration request filters"); + let js_mode = if registration.js_disabled { + if id == "gpt_diagnostics" + && crate::integrations::gpt_diagnostics::GptDiagnosticsRequestDecision::active_for_tests() + .module_script_tag() + .is_some_and(|tag| tag.contains("tsjs-gpt_diagnostics.min.js")) + && trusted_server_js::module_bundle(id).is_some() + { + "standalone" + } else { + "none" + } + } else if registration.js_deferred { + "deferred" + } else if trusted_server_js::module_bundle(id).is_some() { + "bundled" + } else { + "none" + }; + Task8Capability { + id: id.to_owned(), + predicate: predicate.to_owned(), + proxy_routes, + attribute_rewriters, + script_rewriters, + head_injectors, + post_processors, + request_filters, + providers: BTreeSet::new(), + js_mode: js_mode.to_owned(), + } + } } diff --git a/crates/trusted-server-core/src/lib.rs b/crates/trusted-server-core/src/lib.rs index 76621baf7..043907ea6 100644 --- a/crates/trusted-server-core/src/lib.rs +++ b/crates/trusted-server-core/src/lib.rs @@ -1,23 +1,31 @@ -//! Common functionality for the trusted server. +//! Platform-neutral request handling for Trusted Server. //! -//! This crate provides shared types, utilities, and abstractions used by both -//! the Fastly edge implementation and local development/testing environments. +//! Adapter crates supply stores, outbound HTTP, geo data, and other runtime +//! services through [`platform::RuntimeServices`]. This crate owns routing, +//! policy, integrations, and response transformation without depending on a +//! specific edge provider. //! //! # Modules //! -//! - [`auth`]: Basic authentication enforcement helpers -//! - [`constants`]: Application-wide constants and configuration values -//! - [`cookies`]: Cookie parsing and generation utilities -//! - [`error`]: Error types and error handling utilities -//! - [`consent`]: Consent signal extraction and logging -//! - [`geo`]: Geographic location utilities and DMA code extraction -//! - [`models`]: Data models for ad serving and callbacks -//! - [`integrations::prebid`]: Prebid integration and real-time bidding support -//! - [`settings`]: Configuration management and validation -//! - [`streaming_replacer`]: Streaming URL replacement for large responses -//! - [`ec`]: Edge Cookie (EC) identity subsystem — ID generation, consent gating, lifecycle -//! - [`test_support`]: Testing utilities and mocks -//! - [`tester_cookie`]: Optional tester-cookie endpoint helpers +//! - Auction planning and execution: [`auction`], [`auction_config_types`]. +//! - Authentication and signing: [`auth`], [`request_signing`]. +//! - Configuration loading and validation: [`config`], [`config_payload`], +//! [`secret_resolution`], [`settings`], [`settings_data`]. +//! - Cache and privacy policy: [`cache_policy`], [`response_privacy`]. +//! - Consent, cookies, and identity: [`consent`], [`consent_config`], +//! [`cookies`], [`ec`], [`tester_cookie`]. +//! - Creative selection and rendering: [`creative`], [`creative_opportunities`], +//! [`price_bucket`]. +//! - Request and client context: [`constants`], [`geo`], [`host_header`], +//! [`http_util`], [`models`]. +//! - Integration registry and browser bundles: [`integrations`], [`tsjs`]. +//! - `OpenRTB` transport types: [`openrtb`]. +//! - Platform service contracts and persistence: [`platform`], [`storage`]. +//! - Publisher and first-party proxy routes: [`proxy`], [`publisher`]. +//! - Response transformation: [`html_processor`], [`rsc_flight`], +//! [`streaming_processor`], [`streaming_replacer`]. +//! - Shared support types and test helpers: [`error`], [`redacted`], +//! [`test_support`]. #![cfg_attr( test, diff --git a/crates/trusted-server-core/src/openrtb.rs b/crates/trusted-server-core/src/openrtb.rs index 4aded7488..a8cccfea2 100644 --- a/crates/trusted-server-core/src/openrtb.rs +++ b/crates/trusted-server-core/src/openrtb.rs @@ -1,10 +1,18 @@ +//! `OpenRTB` aliases and Trusted Server extension payloads. +//! +//! Base protocol types come from `trusted-server-openrtb`; this module adds the +//! request and response extensions used by auction orchestration. + use serde::{Deserialize, Serialize}; use serde_json::Value; use crate::auction::types::{BidRenderer, OrchestratorExt}; +/// Canonical `OpenRTB` bid-request type. pub type OpenRtbRequest = trusted_server_openrtb::BidRequest; +/// Canonical `OpenRTB` bid-response type. pub type OpenRtbResponse = trusted_server_openrtb::BidResponse; +/// Canonical `OpenRTB` bid type. pub type OpenRtbBid = trusted_server_openrtb::Bid; pub use trusted_server_openrtb::{ diff --git a/crates/trusted-server-core/src/platform/backend_naming.rs b/crates/trusted-server-core/src/platform/backend_naming.rs index 54fc176b1..ff3686f38 100644 --- a/crates/trusted-server-core/src/platform/backend_naming.rs +++ b/crates/trusted-server-core/src/platform/backend_naming.rs @@ -2,7 +2,9 @@ //! //! These policies contain no platform SDK calls. Startup validation, CLI //! validation, and runtime adapters can therefore predict the same backend -//! names before any backend registration occurs. +//! names before any backend registration occurs. The Fastly policy also keeps +//! the readable prefix, digest suffix, timeout quantization, and backend-count +//! budget in one deterministic contract. use core::fmt::Write as _; diff --git a/crates/trusted-server-core/src/platform/error.rs b/crates/trusted-server-core/src/platform/error.rs index 699eb7d48..83230fed7 100644 --- a/crates/trusted-server-core/src/platform/error.rs +++ b/crates/trusted-server-core/src/platform/error.rs @@ -1,3 +1,5 @@ +//! Errors shared by the platform-neutral service contracts. + use derive_more::Display; /// Root error type for platform service operations. diff --git a/crates/trusted-server-core/src/platform/http.rs b/crates/trusted-server-core/src/platform/http.rs index 039df014c..64d4c22c6 100644 --- a/crates/trusted-server-core/src/platform/http.rs +++ b/crates/trusted-server-core/src/platform/http.rs @@ -1,3 +1,8 @@ +//! Platform-neutral outbound HTTP requests, responses, and fan-out handles. +//! +//! Adapters translate these values to provider APIs while core request handlers +//! retain ownership of cache, streaming, and image-optimization policy. + use std::any::Any; use std::fmt; diff --git a/crates/trusted-server-core/src/platform/image_optimizer.rs b/crates/trusted-server-core/src/platform/image_optimizer.rs index 9a3f1709e..b31c07b5c 100644 --- a/crates/trusted-server-core/src/platform/image_optimizer.rs +++ b/crates/trusted-server-core/src/platform/image_optimizer.rs @@ -7,7 +7,8 @@ //! `fastly::image_optimizer::ImageOptimizerOptions`. //! //! Unsupported adapters should reject requests carrying this metadata rather -//! than silently dropping transformations. +//! than silently dropping transformations. The closed enums keep invalid +//! provider-specific options out of the shared request path. /// Platform-neutral Image Optimizer processing region. /// diff --git a/crates/trusted-server-core/src/platform/kv.rs b/crates/trusted-server-core/src/platform/kv.rs index 81d5a82d3..2735fa2b1 100644 --- a/crates/trusted-server-core/src/platform/kv.rs +++ b/crates/trusted-server-core/src/platform/kv.rs @@ -1,3 +1,5 @@ +//! Fail-closed fallback for an unavailable platform KV store. + use bytes::Bytes; use edgezero_core::key_value_store::{KvError, KvPage, KvStore as PlatformKvStore}; diff --git a/crates/trusted-server-core/src/platform/mod.rs b/crates/trusted-server-core/src/platform/mod.rs index d09f9b549..6c7516d84 100644 --- a/crates/trusted-server-core/src/platform/mod.rs +++ b/crates/trusted-server-core/src/platform/mod.rs @@ -15,13 +15,11 @@ //! - [`PlatformTemplateAssembler`] — cold-response shared-template assembly //! - [`PlatformTemplateCache`] — shared transformed-template caching //! -//! ## Platform-Agnostic Components +//! ## Platform-agnostic components //! -//! The following components were evaluated for platform-specific behavior -//! (verified 2026-03-31; see `docs/superpowers/plans/2026-03-31-pr8-content-rewriting-verification.md`) -//! and found to have a platform-agnostic rewriting pipeline. No -//! platform trait is required; future adapters (Cloudflare Workers, Axum, Spin) need not provide -//! any content-rewriting implementation: +//! Content rewriting remains in core rather than behind an adapter trait. +//! Cloudflare Workers, Axum, Spin, and Fastly all use the same processing +//! implementation: //! //! - **Content rewriting** — `html_processor`, `streaming_processor`, //! `streaming_replacer`, and `rsc_flight` modules use only standard Rust @@ -29,8 +27,7 @@ //! is accessed via [`StreamingPipeline::process`](crate::streaming_processor::StreamingPipeline::process) which //! accepts any reader that implements `std::io::Read`. //! -//! No `PlatformContentRewriter` trait exists or is needed. -//! +//! No `PlatformContentRewriter` trait exists or is needed. use std::time::Duration; diff --git a/crates/trusted-server-core/src/platform/template_assembly.rs b/crates/trusted-server-core/src/platform/template_assembly.rs index f56179f1f..acd9718a2 100644 --- a/crates/trusted-server-core/src/platform/template_assembly.rs +++ b/crates/trusted-server-core/src/platform/template_assembly.rs @@ -2,7 +2,8 @@ //! //! Core owns the cache-safety ordering and the portable byte-seam fallback. An adapter //! may provide a richer assembler for the cold response after the reader-neutral -//! template has been stored. +//! template has been stored. Unsupported adapters return a typed error rather +//! than serving unresolved ESI instructions. use core::fmt; diff --git a/crates/trusted-server-core/src/platform/template_cache.rs b/crates/trusted-server-core/src/platform/template_cache.rs index e3bbc42da..2efa58d45 100644 --- a/crates/trusted-server-core/src/platform/template_cache.rs +++ b/crates/trusted-server-core/src/platform/template_cache.rs @@ -1,7 +1,6 @@ -//! The shared transformed-template cache for the #1009 ESI validation spike. +//! Shared cache for transformed, reader-neutral publisher templates. //! -//! Three caches are in play and conflating them is what produced the original wrong -//! conclusion in the design doc, so this module names which one it is: +//! Three cache layers are intentionally distinct: //! //! | Cache | Contents | Owner | //! | ----- | --------------------------------- | ------------------------------ | @@ -9,13 +8,11 @@ //! | Template cache | post-`lol_html`, pre-assembly | **This module.** | //! | Final response | final per-user assembled response | **Must never exist.** | //! -//! The template cache holds a *shared template*: no per-user bytes, and no decisions that depend on -//! the request. What may and may not live in it is +//! The template cache holds no per-user bytes or request-dependent decisions. +//! Its safety contract is //! [§6.7 of the design doc](../../../../docs/superpowers/archive/2026-08-08-esi-cacheable-root-validation-design.md), //! and the invariant is enforced by the rendered-document byte-identity tests in //! `publisher`. -//! -//! Spike-only. Remove with the spike. use core::fmt; use std::collections::HashSet; diff --git a/crates/trusted-server-core/src/platform/traits.rs b/crates/trusted-server-core/src/platform/traits.rs index 6c31cd279..a3f89687a 100644 --- a/crates/trusted-server-core/src/platform/traits.rs +++ b/crates/trusted-server-core/src/platform/traits.rs @@ -1,3 +1,8 @@ +//! Object-safe service ports implemented by each deployment adapter. +//! +//! Store operations are synchronous because supported edge runtimes expose +//! different async models; outbound HTTP remains async through boxed futures. + use std::net::IpAddr; use error_stack::Report; diff --git a/crates/trusted-server-core/src/platform/types.rs b/crates/trusted-server-core/src/platform/types.rs index 7a3d09334..c13588b9a 100644 --- a/crates/trusted-server-core/src/platform/types.rs +++ b/crates/trusted-server-core/src/platform/types.rs @@ -1,3 +1,8 @@ +//! Shared runtime service container and request metadata. +//! +//! [`RuntimeServices`] is assembled once by an adapter and passed through core +//! handlers; its builder supplies typed unavailable services for omitted ports. + use std::fmt; use std::net::IpAddr; use std::sync::Arc; diff --git a/crates/trusted-server-core/src/price_bucket.rs b/crates/trusted-server-core/src/price_bucket.rs index 30b7430de..383700352 100644 --- a/crates/trusted-server-core/src/price_bucket.rs +++ b/crates/trusted-server-core/src/price_bucket.rs @@ -1,13 +1,24 @@ +//! Prebid-compatible CPM price bucketing. +//! +//! Buckets are computed in whole cents to avoid binary floating-point drift at +//! cent boundaries and are capped according to the selected granularity. + use serde::{Deserialize, Serialize}; +/// Named price-bucketing schedule used for ad-server targeting. #[derive(Debug, Clone, Copy, PartialEq, Eq, Deserialize, Serialize, Default)] #[serde(rename_all = "lowercase")] pub enum PriceGranularity { + /// Coarse buckets up to a low maximum CPM. Low, + /// Medium-width buckets over a broader CPM range. Medium, + /// Dense buckets; the default schedule. #[default] Dense, + /// Fine-grained buckets over a high CPM range. High, + /// Prebid's automatic mixed-width schedule. Auto, } @@ -25,6 +36,9 @@ fn cpm_to_cents(cpm: f64) -> u64 { } #[must_use] +/// Return the floor bucket for `cpm` under `granularity` as a two-decimal string. +/// +/// Non-finite and non-positive values return `"0.00"`. pub fn price_bucket(cpm: f64, granularity: PriceGranularity) -> String { // Reject NaN / Inf early so the cast in `cpm_to_cents` can never see a // non-finite value (the cast's behaviour for NaN/Inf is implementation- diff --git a/crates/trusted-server-core/src/proxy.rs b/crates/trusted-server-core/src/proxy.rs index 122e4c832..7df243064 100644 --- a/crates/trusted-server-core/src/proxy.rs +++ b/crates/trusted-server-core/src/proxy.rs @@ -1,3 +1,9 @@ +//! First-party asset and content proxying. +//! +//! Routes in this module validate signed destinations, apply origin auth, +//! enforce body limits, remove private edge-cache metadata, and stream eligible +//! responses through the rewrite pipeline. + use crate::http_util::{ RequestInfo, compute_encrypted_sha256_token, ct_str_eq, enforce_max_body_size, }; diff --git a/crates/trusted-server-core/src/publisher.rs b/crates/trusted-server-core/src/publisher.rs index 9525820ce..65906332d 100644 --- a/crates/trusted-server-core/src/publisher.rs +++ b/crates/trusted-server-core/src/publisher.rs @@ -1448,7 +1448,7 @@ pub enum PublisherResponse { response: Response, /// Origin body to be piped through the streaming pipeline. body: EdgeBody, - /// Parameters for [`process_response_streaming`]. + /// Parameters for the internal streaming-response processor. params: Box, }, /// A shared template read from template cache, to be assembled on the way out. @@ -1573,7 +1573,7 @@ fn apply_publisher_asset_cache_policy( Ok(()) } -/// Owned version of [`ProcessResponseParams`] for returning from +/// Owned version of the internal borrowed response parameters for returning from /// [`handle_publisher_request`] without lifetime issues. pub struct OwnedProcessResponseParams { /// Where to store the transformed template, or [`None`] to store nothing. @@ -6227,10 +6227,10 @@ fn page_bids_request_allowed(req: &Request) -> bool { /// Builds the `403 Forbidden` returned when the side-effecting /// `/_ts/page-bids` endpoint refuses a request — both the CORS preflight -/// (`OPTIONS`) and the GET cross-site gate ([`page_bids_request_allowed`]) +/// (`OPTIONS`) and the GET cross-site gate (`page_bids_request_allowed`) /// return this single denial shape. /// -/// The GET handler's [`page_bids_request_allowed`] gate trusts the +/// The GET handler's `page_bids_request_allowed` gate trusts the /// `X-TSJS-Page-Bids` header precisely because this endpoint never grants a /// preflight; letting `OPTIONS` fall through to the publisher origin (which may /// return permissive CORS) would defeat that, allowing a cross-site page to diff --git a/crates/trusted-server-core/src/request_signing/endpoints.rs b/crates/trusted-server-core/src/request_signing/endpoints.rs index d35e51a15..790b55c7a 100644 --- a/crates/trusted-server-core/src/request_signing/endpoints.rs +++ b/crates/trusted-server-core/src/request_signing/endpoints.rs @@ -269,7 +269,7 @@ fn validate_kid(kid: &str) -> Result<(), Report> { } /// Returns whether `kid` satisfies the create/rotate portable-KID contract -/// enforced by [`validate_kid`]. +/// enforced by the internal `validate_kid` helper. /// /// Exposed so platform adapter crates can assert their key-name encoder accepts /// every kid this validation admits, pinning the cross-adapter contract against diff --git a/crates/trusted-server-core/src/rsc_flight.rs b/crates/trusted-server-core/src/rsc_flight.rs index a9e3a40ce..8edcd6b2b 100644 --- a/crates/trusted-server-core/src/rsc_flight.rs +++ b/crates/trusted-server-core/src/rsc_flight.rs @@ -1,3 +1,8 @@ +//! Streaming URL rewriting for React Server Component Flight payloads. +//! +//! The parser preserves row framing and recomputes byte lengths for rewritten +//! length-delimited chunks without buffering an entire response. + use std::io; use crate::host_rewrite::rewrite_bare_host_at_boundaries; diff --git a/crates/trusted-server-core/src/settings.rs b/crates/trusted-server-core/src/settings.rs index 8fff7917d..5518201b4 100644 --- a/crates/trusted-server-core/src/settings.rs +++ b/crates/trusted-server-core/src/settings.rs @@ -1,3 +1,8 @@ +//! Trusted Server application configuration and validation. +//! +//! Deserialization is only the first phase: runtime preparation validates +//! cross-field contracts and compiles reusable matchers before request handling. + #[cfg(test)] use config::{Config, Environment, File, FileFormat}; use error_stack::{Report, ResultExt}; @@ -86,7 +91,7 @@ fn default_max_buffered_body_bytes() -> usize { impl Default for Publisher { /// Hand-written so `max_buffered_body_bytes` matches the serde default - /// ([`default_max_buffered_body_bytes`]) instead of `usize`'s `0`. A derived + /// (`default_max_buffered_body_bytes`) instead of `usize`'s `0`. A derived /// `Default` would set a zero-byte cap, which fails buffered post-processing /// immediately when `Publisher::default()` / `Settings::default()` are used /// programmatically (tests, helpers) rather than deserialized from TOML. @@ -620,7 +625,7 @@ impl Ec { /// # Errors /// /// Returns a validation error if the passphrase is empty or shorter - /// than [`Self::MIN_PASSPHRASE_LENGTH`] characters. + /// than the private `MIN_PASSPHRASE_LENGTH` limit. pub fn validate_passphrase(passphrase: &Redacted) -> Result<(), ValidationError> { if passphrase.expose().is_empty() { return Err(ValidationError::new("empty_passphrase")); @@ -2540,7 +2545,7 @@ pub struct DebugConfig { /// /// Note: the sanitized winning `adm` is now injected **unconditionally** for /// production inline rendering through the pbRender bridge (see - /// [`crate::publisher::build_bid_map`]); this flag no longer gates `adm`. + /// the publisher bid-map builder); this flag no longer gates `adm`. /// What it still gates is the client-side `debug_bid` signal that turns on /// the direct GAM-creative replacement (`injectAdmIntoSlot`), which bypasses /// GAM entirely — useful for validating the auction→creative pipeline while @@ -2614,11 +2619,11 @@ pub struct AuctionDebugCommentOptions { #[serde(default = "default_true")] pub include_bids: bool, - /// Subset of [`AUCTION_DEBUG_METADATA_ALLOWLIST`] to surface in + /// Subset of the fixed internal metadata allowlist to surface in /// [`AuctionDebugCommentVerbosity::Redacted`] mode. This selector cannot /// unlock provider diagnostics, and entries outside the fixed allowlist are /// rejected at config load by - /// [`validate_metadata_keys`](Self::validate_metadata_keys). + /// the private `validate_metadata_keys` validator. /// /// [`AuctionDebugCommentVerbosity::Upstream`] builds on the redacted /// metadata, so this subset still gates those three keys there; the six @@ -3682,7 +3687,7 @@ mod tests { use super::*; use regex::Regex; use serde_json::json; - use std::collections::HashSet; + use std::collections::{BTreeSet, HashSet}; use std::sync::Arc; use crate::auction::build_orchestrator; @@ -3693,6 +3698,206 @@ mod tests { use crate::redacted::Redacted; use crate::test_support::tests::{crate_test_settings_str, create_test_settings}; + #[derive(Debug, Deserialize)] + struct Task7DeserializerProbe { + #[serde(deserialize_with = "from_value_or_str")] + number: u32, + #[serde(deserialize_with = "bool_from_bool_or_str")] + enabled: bool, + #[serde(deserialize_with = "vec_from_seq_or_map")] + values: Vec, + #[serde(deserialize_with = "map_from_obj_or_str")] + headers: HashMap, + } + + #[derive(Clone, Debug, Deserialize, Eq, Ord, PartialEq, PartialOrd)] + struct Task7CompanionReceipt { + source: String, + symbol: String, + kind: String, + value: Option, + positive_probe: String, + negative_probe: String, + } + + #[derive(Deserialize)] + struct Task7CompanionManifest { + companions: Vec, + } + + macro_rules! task7_companion_receipt { + ( + $positive:ident, + $negative:ident, + $symbol:literal, + $kind:literal, + $value:expr, + $positive_body:block, + $negative_body:block + ) => {{ + fn $positive() $positive_body + fn $negative() $negative_body + $positive(); + $negative(); + let value: Option<&str> = $value; + Task7CompanionReceipt { + source: "crates/trusted-server-core/src/settings.rs".to_owned(), + symbol: $symbol.to_owned(), + kind: $kind.to_owned(), + value: value.map(str::to_owned), + positive_probe: stringify!($positive).to_owned(), + negative_probe: stringify!($negative).to_owned(), + } + }}; + } + + #[test] + fn task7_settings_companions_are_exact_and_compiled_per_record() { + let actual = BTreeSet::from([ + task7_companion_receipt!( + task7_crates_trusted_server_core_src_settings_rs_default_max_buffered_body_bytes_default_positive, + task7_crates_trusted_server_core_src_settings_rs_default_max_buffered_body_bytes_default_negative, + "default_max_buffered_body_bytes", "default", Some("16777216"), + { assert_eq!(default_max_buffered_body_bytes(), 16 * 1024 * 1024); }, + { assert_ne!(default_max_buffered_body_bytes(), 16 * 1024 * 1024 - 1); } + ), + task7_companion_receipt!( + task7_crates_trusted_server_core_src_settings_rs_from_value_or_str_deserializer_positive, + task7_crates_trusted_server_core_src_settings_rs_from_value_or_str_deserializer_negative, + "from_value_or_str", "deserializer", None, + { + let value: Task7DeserializerProbe = serde_json::from_value(json!({"number":"7","enabled":true,"values":[],"headers":{}})).expect("string number should deserialize"); + assert_eq!(value.number, 7); + }, + { serde_json::from_value::(json!({"number":[],"enabled":true,"values":[],"headers":{}})).expect_err("array number should fail"); } + ), + task7_companion_receipt!( + task7_crates_trusted_server_core_src_settings_rs_vec_from_seq_or_map_deserializer_positive, + task7_crates_trusted_server_core_src_settings_rs_vec_from_seq_or_map_deserializer_negative, + "vec_from_seq_or_map", "deserializer", None, + { + let value: Task7DeserializerProbe = serde_json::from_value(json!({"number":7,"enabled":true,"values":{"1":"second","0":"first"},"headers":{}})).expect("indexed map should deserialize"); + assert_eq!(value.values, ["first", "second"]); + }, + { serde_json::from_value::(json!({"number":7,"enabled":true,"values":false,"headers":{}})).expect_err("boolean sequence should fail"); } + ), + task7_companion_receipt!( + task7_crates_trusted_server_core_src_settings_rs_map_from_obj_or_str_deserializer_positive, + task7_crates_trusted_server_core_src_settings_rs_map_from_obj_or_str_deserializer_negative, + "map_from_obj_or_str", "deserializer", None, + { + let value: Task7DeserializerProbe = serde_json::from_value(json!({"number":7,"enabled":true,"values":[],"headers":"{\"X-Probe\":\"present\"}"})).expect("JSON string map should deserialize"); + assert_eq!(value.headers.get("X-Probe").map(String::as_str), Some("present")); + }, + { serde_json::from_value::(json!({"number":7,"enabled":true,"values":[],"headers":[]})).expect_err("array map should fail"); } + ), + task7_companion_receipt!( + task7_crates_trusted_server_core_src_settings_rs_bool_from_bool_or_str_deserializer_positive, + task7_crates_trusted_server_core_src_settings_rs_bool_from_bool_or_str_deserializer_negative, + "bool_from_bool_or_str", "deserializer", None, + { + let value: Task7DeserializerProbe = serde_json::from_value(json!({"number":7,"enabled":"true","values":[],"headers":{}})).expect("string bool should deserialize"); + assert!(value.enabled); + }, + { serde_json::from_value::(json!({"number":7,"enabled":1,"values":[],"headers":{}})).expect_err("numeric bool should fail"); } + ), + task7_companion_receipt!( + task7_crates_trusted_server_core_src_settings_rs_validate_publisher_domain_validator_positive, + task7_crates_trusted_server_core_src_settings_rs_validate_publisher_domain_validator_negative, + "validate_publisher_domain", "validator", None, + { validate_publisher_domain("publisher.example").expect("plain publisher domain should validate"); }, + { validate_publisher_domain("https://publisher.example").expect_err("publisher scheme should fail"); } + ), + task7_companion_receipt!( + task7_crates_trusted_server_core_src_settings_rs_validate_cookie_domain_validator_positive, + task7_crates_trusted_server_core_src_settings_rs_validate_cookie_domain_validator_negative, + "validate_cookie_domain", "validator", None, + { validate_cookie_domain(".publisher.example").expect("plain cookie domain should validate"); }, + { validate_cookie_domain("publisher.example; Secure").expect_err("cookie metacharacter should fail"); } + ), + task7_companion_receipt!( + task7_crates_trusted_server_core_src_settings_rs_validate_no_trailing_slash_validator_positive, + task7_crates_trusted_server_core_src_settings_rs_validate_no_trailing_slash_validator_negative, + "validate_no_trailing_slash", "validator", None, + { validate_no_trailing_slash("https://origin.example").expect("origin should validate"); }, + { validate_no_trailing_slash("https://origin.example/").expect_err("trailing slash should fail"); } + ), + task7_companion_receipt!( + task7_crates_trusted_server_core_src_settings_rs_validate_host_header_override_validator_positive, + task7_crates_trusted_server_core_src_settings_rs_validate_host_header_override_validator_negative, + "validate_host_header_override", "validator", None, + { validate_host_header_override("origin.example:443").expect("host override should validate"); }, + { validate_host_header_override("https://origin.example").expect_err("host scheme should fail"); } + ), + task7_companion_receipt!( + task7_crates_trusted_server_core_src_settings_rs_validate_redacted_not_empty_validator_positive, + task7_crates_trusted_server_core_src_settings_rs_validate_redacted_not_empty_validator_negative, + "validate_redacted_not_empty", "validator", None, + { validate_redacted_not_empty(&Redacted::new("value".to_owned())).expect("value should validate"); }, + { validate_redacted_not_empty(&Redacted::new(String::new())).expect_err("empty value should fail"); } + ), + task7_companion_receipt!( + task7_crates_trusted_server_core_src_settings_rs_ecpartner_validate_source_domain_validator_positive, + task7_crates_trusted_server_core_src_settings_rs_ecpartner_validate_source_domain_validator_negative, + "EcPartner::validate_source_domain", "validator", None, + { EcPartner::validate_source_domain("identity.example").expect("source domain should validate"); }, + { EcPartner::validate_source_domain("https://identity.example").expect_err("source scheme should fail"); } + ), + task7_companion_receipt!( + task7_crates_trusted_server_core_src_settings_rs_ec_validate_passphrase_validator_positive, + task7_crates_trusted_server_core_src_settings_rs_ec_validate_passphrase_validator_negative, + "Ec::validate_passphrase", "validator", None, + { Ec::validate_passphrase(&Redacted::new("fictional-ec-passphrase-32-bytes-ok".to_owned())).expect("strong passphrase should validate"); }, + { Ec::validate_passphrase(&Redacted::new("short".to_owned())).expect_err("short passphrase should fail"); } + ), + task7_companion_receipt!( + task7_crates_trusted_server_core_src_settings_rs_validate_path_validator_positive, + task7_crates_trusted_server_core_src_settings_rs_validate_path_validator_negative, + "validate_path", "validator", None, + { validate_path("^/articles/[a-z]+$").expect("valid regex should validate"); }, + { validate_path("[").expect_err("invalid regex should fail"); } + ), + task7_companion_receipt!( + task7_crates_trusted_server_core_src_settings_rs_validate_trusted_client_ip_validator_positive, + task7_crates_trusted_server_core_src_settings_rs_validate_trusted_client_ip_validator_negative, + "validate_trusted_client_ip", "validator", None, + { validate_trusted_client_ip(&TrustedClientIpConfig { ip_header: "fastly-client-ip".to_owned(), auth_header: "x-trusted-client-auth".to_owned(), shared_secret: Redacted::new("fictional-shared-secret-32-bytes-ok".to_owned()) }).expect("trusted client config should validate"); }, + { validate_trusted_client_ip(&TrustedClientIpConfig { ip_header: "x-same".to_owned(), auth_header: "x-same".to_owned(), shared_secret: Redacted::new("short".to_owned()) }).expect_err("unsafe trusted client config should fail"); } + ), + task7_companion_receipt!(task7_crates_trusted_server_core_src_settings_rs_ecpartner_default_openrtb_atype_default_positive, task7_crates_trusted_server_core_src_settings_rs_ecpartner_default_openrtb_atype_default_negative, "EcPartner::default_openrtb_atype", "default", Some("3"), { assert_eq!(EcPartner::default_openrtb_atype(), 3); }, { assert_ne!(EcPartner::default_openrtb_atype(), 2); }), + task7_companion_receipt!(task7_crates_trusted_server_core_src_settings_rs_ecpartner_default_batch_rate_limit_default_positive, task7_crates_trusted_server_core_src_settings_rs_ecpartner_default_batch_rate_limit_default_negative, "EcPartner::default_batch_rate_limit", "default", Some("60"), { assert_eq!(EcPartner::default_batch_rate_limit(), 60); }, { assert_ne!(EcPartner::default_batch_rate_limit(), 59); }), + task7_companion_receipt!(task7_crates_trusted_server_core_src_settings_rs_ecpartner_default_pull_sync_ttl_sec_default_positive, task7_crates_trusted_server_core_src_settings_rs_ecpartner_default_pull_sync_ttl_sec_default_negative, "EcPartner::default_pull_sync_ttl_sec", "default", Some("86400"), { assert_eq!(EcPartner::default_pull_sync_ttl_sec(), 86_400); }, { assert_ne!(EcPartner::default_pull_sync_ttl_sec(), 86_399); }), + task7_companion_receipt!(task7_crates_trusted_server_core_src_settings_rs_ecpartner_default_pull_sync_rate_limit_default_positive, task7_crates_trusted_server_core_src_settings_rs_ecpartner_default_pull_sync_rate_limit_default_negative, "EcPartner::default_pull_sync_rate_limit", "default", Some("10"), { assert_eq!(EcPartner::default_pull_sync_rate_limit(), 10); }, { assert_ne!(EcPartner::default_pull_sync_rate_limit(), 9); }), + task7_companion_receipt!(task7_crates_trusted_server_core_src_settings_rs_ec_default_pull_sync_concurrency_default_positive, task7_crates_trusted_server_core_src_settings_rs_ec_default_pull_sync_concurrency_default_negative, "Ec::default_pull_sync_concurrency", "default", Some("3"), { assert_eq!(Ec::default_pull_sync_concurrency(), 3); }, { assert_ne!(Ec::default_pull_sync_concurrency(), 2); }), + task7_companion_receipt!(task7_crates_trusted_server_core_src_settings_rs_ec_default_cluster_trust_threshold_default_positive, task7_crates_trusted_server_core_src_settings_rs_ec_default_cluster_trust_threshold_default_negative, "Ec::default_cluster_trust_threshold", "default", Some("10"), { assert_eq!(Ec::default_cluster_trust_threshold(), 10); }, { assert_ne!(Ec::default_cluster_trust_threshold(), 9); }), + task7_companion_receipt!(task7_crates_trusted_server_core_src_settings_rs_ec_default_cluster_recheck_secs_default_positive, task7_crates_trusted_server_core_src_settings_rs_ec_default_cluster_recheck_secs_default_negative, "Ec::default_cluster_recheck_secs", "default", Some("3600"), { assert_eq!(Ec::default_cluster_recheck_secs(), 3_600); }, { assert_ne!(Ec::default_cluster_recheck_secs(), 3_599); }), + task7_companion_receipt!(task7_crates_trusted_server_core_src_settings_rs_default_s3_access_key_id_default_positive, task7_crates_trusted_server_core_src_settings_rs_default_s3_access_key_id_default_negative, "default_s3_access_key_id", "default", Some("access_key_id"), { assert_eq!(default_s3_access_key_id().expose(), "access_key_id"); }, { assert_ne!(default_s3_access_key_id().expose(), "secret_access_key"); }), + task7_companion_receipt!(task7_crates_trusted_server_core_src_settings_rs_default_s3_secret_access_key_default_positive, task7_crates_trusted_server_core_src_settings_rs_default_s3_secret_access_key_default_negative, "default_s3_secret_access_key", "default", Some("secret_access_key"), { assert_eq!(default_s3_secret_access_key().expose(), "secret_access_key"); }, { assert_ne!(default_s3_secret_access_key().expose(), "access_key_id"); }), + task7_companion_receipt!(task7_crates_trusted_server_core_src_settings_rs_default_profile_param_default_positive, task7_crates_trusted_server_core_src_settings_rs_default_profile_param_default_negative, "default_profile_param", "default", Some("profile"), { assert_eq!(default_profile_param(), "profile"); }, { assert_ne!(default_profile_param(), "ar"); }), + task7_companion_receipt!(task7_crates_trusted_server_core_src_settings_rs_default_aspect_ratio_param_default_positive, task7_crates_trusted_server_core_src_settings_rs_default_aspect_ratio_param_default_negative, "default_aspect_ratio_param", "default", Some("ar"), { assert_eq!(default_aspect_ratio_param(), "ar"); }, { assert_ne!(default_aspect_ratio_param(), "profile"); }), + task7_companion_receipt!(task7_crates_trusted_server_core_src_settings_rs_default_debug_param_default_positive, task7_crates_trusted_server_core_src_settings_rs_default_debug_param_default_negative, "default_debug_param", "default", Some("_io_debug"), { assert_eq!(default_debug_param(), "_io_debug"); }, { assert_ne!(default_debug_param(), "debug"); }), + task7_companion_receipt!(task7_crates_trusted_server_core_src_settings_rs_default_default_profile_default_positive, task7_crates_trusted_server_core_src_settings_rs_default_default_profile_default_negative, "default_default_profile", "default", Some("default"), { assert_eq!(default_default_profile(), "default"); }, { assert_ne!(default_default_profile(), "standard"); }), + task7_companion_receipt!(task7_crates_trusted_server_core_src_settings_rs_default_crop_offset_x_param_default_positive, task7_crates_trusted_server_core_src_settings_rs_default_crop_offset_x_param_default_negative, "default_crop_offset_x_param", "default", Some("x"), { assert_eq!(default_crop_offset_x_param(), "x"); }, { assert_ne!(default_crop_offset_x_param(), "y"); }), + task7_companion_receipt!(task7_crates_trusted_server_core_src_settings_rs_default_crop_offset_y_param_default_positive, task7_crates_trusted_server_core_src_settings_rs_default_crop_offset_y_param_default_negative, "default_crop_offset_y_param", "default", Some("y"), { assert_eq!(default_crop_offset_y_param(), "y"); }, { assert_ne!(default_crop_offset_y_param(), "x"); }), + task7_companion_receipt!(task7_crates_trusted_server_core_src_settings_rs_default_crop_offset_buckets_default_positive, task7_crates_trusted_server_core_src_settings_rs_default_crop_offset_buckets_default_negative, "default_crop_offset_buckets", "default", Some("[10,30,50,70,90]"), { assert_eq!(default_crop_offset_buckets(), [10,30,50,70,90]); }, { assert_ne!(default_crop_offset_buckets(), [10,30,50,70]); }), + task7_companion_receipt!(task7_crates_trusted_server_core_src_settings_rs_default_tinybird_auction_dataset_default_positive, task7_crates_trusted_server_core_src_settings_rs_default_tinybird_auction_dataset_default_negative, "default_tinybird_auction_dataset", "default", Some("auction_events_raw"), { assert_eq!(default_tinybird_auction_dataset(), "auction_events_raw"); }, { assert_ne!(default_tinybird_auction_dataset(), "access_logs_raw"); }), + task7_companion_receipt!(task7_crates_trusted_server_core_src_settings_rs_default_tinybird_access_dataset_default_positive, task7_crates_trusted_server_core_src_settings_rs_default_tinybird_access_dataset_default_negative, "default_tinybird_access_dataset", "default", Some("access_logs_raw"), { assert_eq!(default_tinybird_access_dataset(), "access_logs_raw"); }, { assert_ne!(default_tinybird_access_dataset(), "auction_events_raw"); }), + task7_companion_receipt!(task7_crates_trusted_server_core_src_settings_rs_default_tinybird_max_body_bytes_default_positive, task7_crates_trusted_server_core_src_settings_rs_default_tinybird_max_body_bytes_default_negative, "default_tinybird_max_body_bytes", "default", Some("1048576"), { assert_eq!(default_tinybird_max_body_bytes(), 1024*1024); }, { assert_ne!(default_tinybird_max_body_bytes(), 1024*1024-1); }), + task7_companion_receipt!(task7_crates_trusted_server_core_src_settings_rs_default_auction_debug_metadata_keys_default_positive, task7_crates_trusted_server_core_src_settings_rs_default_auction_debug_metadata_keys_default_negative, "default_auction_debug_metadata_keys", "default", Some("[error_type,http_status,message]"), { assert_eq!(default_auction_debug_metadata_keys(), ["error_type","http_status","message"]); }, { assert_ne!(default_auction_debug_metadata_keys(), ["error_type","message"]); }), + ]); + let checked: Task7CompanionManifest = toml::from_str(include_str!(concat!( + env!("CARGO_MANIFEST_DIR"), + "/../../tools/docs-parity/manifests/settings-companions.toml" + ))) + .expect("checked Task 7 companion manifest should parse"); + let expected = checked + .companions + .into_iter() + .filter(|record| record.source == "crates/trusted-server-core/src/settings.rs") + .collect::>(); + assert_eq!(actual, expected, "compiled settings receipts must be exact"); + } + fn trusted_client_ip_toml(ip_header: &str, auth_header: &str, shared_secret: &str) -> String { format!( "{}\n[trusted_client_ip]\nip_header = \"{ip_header}\"\nauth_header = \"{auth_header}\"\nshared_secret = \"{shared_secret}\"\n", diff --git a/crates/trusted-server-core/src/settings_data.rs b/crates/trusted-server-core/src/settings_data.rs index b82ec92d4..87b096e69 100644 --- a/crates/trusted-server-core/src/settings_data.rs +++ b/crates/trusted-server-core/src/settings_data.rs @@ -1,3 +1,8 @@ +//! Loading signed application configuration from platform stores. +//! +//! Fastly's entry-size limit is handled through a checked chunk envelope; all +//! paths verify lengths and SHA-256 digests before parsing settings. + use edgezero_core::env_config::EnvConfig; use error_stack::{Report, ResultExt}; use serde::Deserialize; diff --git a/crates/trusted-server-core/src/storage/kv_store.rs b/crates/trusted-server-core/src/storage/kv_store.rs index cf8665b2a..8575dfaa6 100644 --- a/crates/trusted-server-core/src/storage/kv_store.rs +++ b/crates/trusted-server-core/src/storage/kv_store.rs @@ -6,16 +6,19 @@ //! //! # Storage layout //! -//! Each entry uses a single JSON body ([`KvConsentEntry`]) containing the raw +//! Each entry uses a single JSON body +//! ([`KvConsentEntry`](crate::storage::kv_store::KvConsentEntry)) containing the raw //! consent strings, context flags, and a fingerprint for write-on-change //! detection. //! //! # Change detection //! //! Writes only occur when consent signals have actually changed. -//! [`consent_fingerprint`] hashes the raw strings into a compact fingerprint +//! [`consent_fingerprint`](crate::storage::kv_store::consent_fingerprint) hashes +//! the raw strings into a compact fingerprint //! stored in the body's `fp` field. On the next request, the existing -//! fingerprint is compared before writing. +//! fingerprint is compared before writing. All operations use the +//! platform-neutral [`PlatformKvStore`](crate::platform::PlatformKvStore) contract. use bytes::Bytes; use serde::{Deserialize, Serialize}; diff --git a/crates/trusted-server-core/src/storage/mod.rs b/crates/trusted-server-core/src/storage/mod.rs index 0c6998b6d..dafc2c308 100644 --- a/crates/trusted-server-core/src/storage/mod.rs +++ b/crates/trusted-server-core/src/storage/mod.rs @@ -1 +1,4 @@ +//! Persistence helpers built on the platform-neutral KV contract. + +/// Consent-state persistence keyed by Edge Cookie identifier. pub mod kv_store; diff --git a/crates/trusted-server-core/src/tsjs.rs b/crates/trusted-server-core/src/tsjs.rs index e8275b814..fdf846b76 100644 --- a/crates/trusted-server-core/src/tsjs.rs +++ b/crates/trusted-server-core/src/tsjs.rs @@ -1,3 +1,8 @@ +//! Cache-busted Trusted Server JavaScript URLs and script tags. +//! +//! Hashes come from the embedded bundle crate so emitted URLs change exactly +//! when the selected module bytes change. + use trusted_server_js::{concatenated_hash, single_module_hash}; /// `/static` URL for the tsjs bundle with cache-busting hash based on diff --git a/crates/trusted-server-integration-tests/Cargo.toml b/crates/trusted-server-integration-tests/Cargo.toml index 7477fdbd1..131468724 100644 --- a/crates/trusted-server-integration-tests/Cargo.toml +++ b/crates/trusted-server-integration-tests/Cargo.toml @@ -1,6 +1,7 @@ [package] name = "trusted-server-integration-tests" description = "Cross-adapter parity and integration tests for Trusted Server." +readme = "README.md" authors = { workspace = true } edition = { workspace = true } license = { workspace = true } diff --git a/crates/trusted-server-integration-tests/README.md b/crates/trusted-server-integration-tests/README.md index e82cb8837..2d3981721 100644 --- a/crates/trusted-server-integration-tests/README.md +++ b/crates/trusted-server-integration-tests/README.md @@ -1,257 +1,40 @@ -# Integration Tests +# trusted-server-integration-tests -End-to-end tests that verify the trusted server against real frontend -containers using [Testcontainers](https://testcontainers.com/) and -[Playwright](https://playwright.dev/). +Native test package for cross-adapter parity, documentation compilation, and +end-to-end publisher behavior. It is not linked into an adapter artifact. -## Prerequisites +## Test surfaces -- **Docker** — running and accessible -- **Viceroy** — Fastly local simulator (`cargo install viceroy --version 0.17.0 --locked --force`) -- **wasm32-wasip1 target** — `rustup target add wasm32-wasip1` -- **Node.js** — version pinned in `.tool-versions`, for browser tests only +- `tests/parity.rs` calls Axum, Cloudflare, and Spin routers in process and + compares their shared route behavior. +- `tests/documentation_snippets.rs` extracts the checked integration-guide + fixture and compiles it in an isolated offline crate. +- `tests/integration.rs` exercises the Fastly/Viceroy and Axum paths against the + WordPress and Next.js fixture containers. +- `browser/` uses Playwright and Chromium to verify script loading, navigation, + rewriting, APS rendering, and GPT diagnostics in real pages. -## Quick start +The package uses the native host target. Fastly application artifacts are +compiled separately for `wasm32-wasip1`; Docker, Viceroy, Node, and Chromium +are required only by the end-to-end surfaces that invoke them. -### HTTP-level tests +## Run -```bash -./scripts/integration-tests.sh -``` - -This script handles everything: - -1. Builds the WASM binary -2. Generates Viceroy configs from the readable `trusted-server.integration.toml` - fixture -3. Builds the WordPress and Next.js Docker images -4. Runs all integration tests sequentially - -### Browser tests +From the repository root: ```bash +cargo test --manifest-path crates/trusted-server-integration-tests/Cargo.toml --test parity +cargo test --manifest-path crates/trusted-server-integration-tests/Cargo.toml --test documentation_snippets +./scripts/integration-tests.sh ./scripts/integration-tests-browser.sh ``` -This script: - -1. Builds the WASM binary and Docker images (same as above) -2. Generates the Viceroy config consumed by Playwright global setup -3. Installs Playwright and Chromium -4. Runs browser tests for Next.js and WordPress sequentially - -### Run a single test - -```bash -# HTTP-level -./scripts/integration-tests.sh test_wordpress_fastly -./scripts/integration-tests.sh test_nextjs_fastly - -# Browser — single framework after building WASM/images and generating configs -cd crates/trusted-server-integration-tests/browser -VICEROY_CONFIG_PATH=../../../target/integration-test-artifacts/configs/viceroy.toml \ -TEST_FRAMEWORK=nextjs npx playwright test -VICEROY_CONFIG_PATH=../../../target/integration-test-artifacts/configs/viceroy.toml \ -TEST_FRAMEWORK=wordpress npx playwright test -``` - -### Verbose output - -```bash -./scripts/integration-tests.sh --nocapture -``` - -## Docker images - -Two test images are built from fixtures in `fixtures/frameworks/`: - -| Image | Dockerfile | Description | -|---|---|---| -| `test-wordpress:latest` | `fixtures/frameworks/wordpress/Dockerfile` | PHP built-in server with a minimal test theme | -| `test-nextjs:latest` | `fixtures/frameworks/nextjs/Dockerfile` | Next.js 14 standalone app with 4 pages, API routes, forms, shared navigation, and deferred scripts | - -Both images include test fixtures with absolute origin URLs (`ORIGIN_HOST` env -var) so the trusted server's URL rewriting can be verified. - -### Build images manually - -```bash -docker build -t test-wordpress:latest \ - crates/trusted-server-integration-tests/fixtures/frameworks/wordpress/ - -docker build \ - --build-arg NODE_VERSION="$(grep '^nodejs ' .tool-versions | awk '{print $2}')" \ - -t test-nextjs:latest \ - crates/trusted-server-integration-tests/fixtures/frameworks/nextjs/ -``` - -## Generated Viceroy configs - -The source-controlled Viceroy template contains only local runtime resources such -as KV stores, secret stores, and JWKS config. The Trusted Server application -config is kept as readable TOML in -`fixtures/configs/trusted-server.integration.toml` and converted into an -EdgeZero `BlobEnvelope` at test setup time. - -Generate the post-cutover Viceroy config manually with: - -```bash -ARTIFACTS_DIR=target/integration-test-artifacts \ -INTEGRATION_ORIGIN_PORT=8888 \ -./scripts/generate-integration-viceroy-configs.sh -``` - -Generated output: - -| File | Purpose | -|---|---| -| `target/integration-test-artifacts/configs/viceroy.toml` | Fastly integration, EC lifecycle, and browser tests | - -Set `VICEROY_CONFIG_PATH` to the generated file when invoking `cargo test` or -Playwright directly. - -## Test scenarios - -### HTTP-level — standard (all frameworks) - -| Scenario | What it tests | -|---|---| -| `HtmlInjection` | Exactly one ` - - - -``` +The always-present `creative` module and all immediate integration modules are +served through `/static/tsjs=tsjs-unified.min.js`. Trusted Server does not +accept an arbitrary asset name from an integration registration. ### Bundle Types diff --git a/docs/guide/edgezero.md b/docs/guide/edgezero.md new file mode 100644 index 000000000..30d56856b --- /dev/null +++ b/docs/guide/edgezero.md @@ -0,0 +1,76 @@ +# EdgeZero Lifecycle and Stores + +EdgeZero is the deployment and configuration layer used by the `ts` CLI. +The repository's `edgezero.toml` declares one application, three logical +stores, and the Fastly, Axum, Cloudflare, and Spin adapter commands. + +## Lifecycle + +Use `ts config init` to create a file, edit the generated +`trusted-server.toml`, and run `ts config validate` before any platform +write. `ts config diff --adapter ADAPTER` compares the validated local +configuration with the selected platform value. `ts config push --adapter +ADAPTER` writes it. Build, serve, deploy, health-check, rollback, and active +version operations use the adapter commands declared in `edgezero.toml`. + +Configuration is startup state, not a live control plane. A successful push +does not change a running instance until the target's restart or deployment +path loads that value. Use `--staging` only with the corresponding staging +deployment flow: it writes `LOGICAL_ID_staging` in the same physical store, +while production continues to read the ordinary logical key. + +## Logical and physical stores + +The manifest declares portable logical IDs: + +| Kind | Logical ID | +| ------------- | ------------------------ | +| KV | `trusted_server_kv` | +| Configuration | `trusted_server_config` | +| Secrets | `trusted_server_secrets` | + +An adapter maps those IDs to platform resources. The runtime-facing +`StoreName` used for reads and the management-facing `StoreId` used for +writes are deliberately distinct types. Environment mappings such as +`EDGEZERO__STORES__CONFIG__TRUSTED_SERVER_CONFIG__NAME` may change the +physical store name without changing application configuration. + +Fastly binds its stores in `fastly.toml`. Axum uses isolated local state for +development. Cloudflare maps the configuration value through its Worker +binding. Spin resolves the manifest mapping and writes supported local +key-value backends. Consult the adapter guide before assuming that a declared +logical store is wired for runtime reads on every target. + +## Blob and chunk flow + +`config push` serializes validated settings into one EdgeZero +`BlobEnvelope`. Store-backed settings contain secret key names, not secret +values; the adapter resolves those keys from the logical secret store during +startup. + +When a Fastly envelope exceeds one Config Store entry, the writer stores +bounded chunks and replaces the root value with a versioned +`fastly_config_chunks` pointer. Startup verifies each chunk's declared +length and SHA-256 and then verifies the reconstructed envelope. A missing, +oversized, reordered, or modified chunk fails configuration loading. + +`ts config gc --adapter fastly` previews orphaned chunks by default. An +actual deletion requires `--yes --older-than WINDOW`. The age assertion +applies to the whole physical store, not one root key; verify the reported +store before deletion, especially when using `--store` or `--no-env`. + +## Safe operator sequence + +1. Put required secret values in the physical store mapped from + `trusted_server_secrets`. +2. Run `ts config validate`. +3. Review `ts config diff --adapter ADAPTER`; use `--no-diff` when + deliberately inline configuration must not appear in logs. +4. Run `ts config push --adapter ADAPTER`. +5. Start or deploy the adapter and verify a non-health publisher route. A + healthy endpoint alone may not prove that application configuration loaded. + +See [Configuration](/guide/configuration) for the field contract, the +[CLI guide](/guide/cli) for exact command help, and +[deployment guides](/guide/integrations-overview#adapter-support) for target +behavior. diff --git a/docs/guide/error-reference.md b/docs/guide/error-reference.md index b45cac63a..11536f243 100644 --- a/docs/guide/error-reference.md +++ b/docs/guide/error-reference.md @@ -8,7 +8,7 @@ Common errors, their causes, and solutions when working with Trusted Server. - [Runtime Errors](#runtime-errors) - [Integration Errors](#integration-errors) - [Request Signing Errors](#request-signing-errors) -- [Build & Deployment Errors](#build--deployment-errors) +- [Build & Deployment Errors](#build-deployment-errors) --- @@ -608,10 +608,10 @@ npm ci npm run build ``` -3. Check for TypeScript errors: +3. Run the TypeScript-aware ESLint checks: ```bash -npm run type-check +npm run lint ``` 4. Skip TSJS build temporarily: diff --git a/docs/guide/fastly.md b/docs/guide/fastly.md index 13f82742b..b0d27c978 100644 --- a/docs/guide/fastly.md +++ b/docs/guide/fastly.md @@ -2,6 +2,21 @@ This guide covers setting up your Fastly account and Compute service for Trusted Server. +## Support status + + + +| Adapter | Release status | Health | Startup status | Startup health | Provider fan-out | Trusted-client-IP handling | Request normalization | +| -------- | -------------- | ---------- | -------------- | -------------- | ---------------- | ------------------------------ | --------------------- | +| `fastly` | production | pre router | `500` | yes | multiple | entry-point resolve + sanitize | none | + + + +The row above is generated from the checked +[adapter-support record](./api-reference#adapter-and-startup-support). A healthy +response does not prove that configuration loaded: Fastly serves `/health` +before it constructs the application. + ## Create a Fastly Account 1. Go to [manage.fastly.com](https://manage.fastly.com) and create an account if you don't have one @@ -340,9 +355,41 @@ fastly resource-link list --service-id --version If EC sync returns `kv_unavailable` or identify responses are degraded, first check that the identity store is present and linked to the active version. Legacy partner/consent KV bindings can be removed once no deployment-specific tooling depends on them. +## Verify the complete local handoff + +Run the repository smoke from a clean shell: + +```bash +./scripts/smoke-fastly.sh +``` + +The script creates an isolated application config, applies its publisher-origin +overrides, and runs strict validation. It then executes `ts config push +--adapter fastly --local`, adds all three required entries to +`[local_server.secret_stores.ts_secrets]`, and starts `fastly compute serve` +through Viceroy. The required keys are `handler_password`, +`publisher_proxy_secret`, and `ec_passphrase`. + +The check deliberately proves both halves of startup. With no config entry, it +requires `/health` to return 200 while the publisher route returns 500 with the +missing `trusted_server_config` diagnostic. It then removes each required +secret independently and requires the corresponding setting path to fail. +Finally, the publisher request must return 200, retain the stub-origin +sentinel, rewrite an origin URL to the Fastly listener, and omit the original +URL. A green health response cannot satisfy that final assertion. + +The trap stops Viceroy and the stub origin, restores `fastly.toml` byte for +byte, restores or removes `.fastly.toml.edgezero-lock` according to its initial +state, and removes the isolated temporary directory. The synthetic credentials +exist only in that local configuration. For a deployed service, provision and +link the stores described above and write the three secrets through Fastly's +secret-store interface. + ## Next Steps - Return to [Getting Started](/guide/getting-started) to continue setup - See [Configuration](/guide/configuration) for detailed configuration options - See [EC Setup Guide](/guide/ec-setup-guide) for end-to-end EC verification - See [Request Signing](/guide/request-signing) for setting up cryptographic signing +- Compare the [Cloudflare](./cloudflare), [Spin](./spin), and [Axum](./axum-dev) + adapter journeys diff --git a/docs/guide/index.md b/docs/guide/index.md index e69de29bb..8a2c00d04 100644 --- a/docs/guide/index.md +++ b/docs/guide/index.md @@ -0,0 +1,30 @@ +# Guide + +Use these guides to understand Trusted Server, configure a deployment, and +contribute to the project. + +## Start here + +- [What is Trusted Server?](/guide/what-is-trusted-server) +- [Getting Started](/guide/getting-started) + +## Core concepts + +- [Edge Cookies](/guide/edge-cookies) +- [GDPR Compliance](/guide/gdpr-compliance) +- [Ad Serving](/guide/ad-serving) +- [First-Party Proxy](/guide/first-party-proxy) +- [Asset Routes](/guide/asset-routes) + +## Development + +- [Architecture](/guide/architecture) +- [Configuration](/guide/configuration) +- [CLI](/guide/cli) +- [Testing](/guide/testing) +- [Integration Guide](/guide/integration-guide) + +## Reference + +- [API Reference](/guide/api-reference) +- [Error Reference](/guide/error-reference) diff --git a/docs/guide/integration-guide.md b/docs/guide/integration-guide.md index cf7da71f6..f127d4207 100644 --- a/docs/guide/integration-guide.md +++ b/docs/guide/integration-guide.md @@ -1,380 +1,223 @@ -# Integration Guide +# Integration Development -This document explains how to integrate a new integration module with the Trusted Server runtime. The workflow mirrors the built-in `testlight` sample in `crates/trusted-server-core/src/integrations/testlight.rs`. +Trusted Server integrations are platform-neutral registrations assembled by +`IntegrationRegistry`. Adapter crates provide I/O through +`RuntimeServices`; integration code must not import Fastly, Cloudflare, +Axum, or Spin SDK types. -## Architecture Overview +## Choose the narrowest hook -| Component | Purpose | -| ----------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| `crates/trusted-server-core/src/integrations/registry.rs` | Defines the `IntegrationProxy`, `IntegrationAttributeRewriter`, `IntegrationScriptRewriter`, and `IntegrationHeadInjector` traits and hosts the `IntegrationRegistry`, which drives proxy routing, HTML/text rewrites, and head injection. | -| `Settings::integrations` (`crates/trusted-server-core/src/settings.rs`) | Free-form JSON blob keyed by integration ID. Use `IntegrationSettings::insert_config` to seed configs; each module deserializes and validates (`validator::Validate`) its own config and exposes an `enabled` flag so the core settings schema stays stable. | -| Fastly entrypoint (`crates/trusted-server-adapter-fastly/src/main.rs`) | Instantiates the registry once per request, routes `/integrations//…` requests to the appropriate proxy, and passes the registry to the publisher origin proxy so HTML rewriting remains integration-aware. | -| `html_processor.rs` | Applies first-party URL rewrites, injects the Trusted Server JS shim, and lets integrations override attribute values (for example to swap script URLs). | +- `IntegrationProxy` owns explicit method/path endpoints and receives + `Settings`, `RuntimeServices`, and an EdgeZero-neutral request. +- `IntegrationAttributeRewriter` inspects selected HTML attributes. +- `IntegrationScriptRewriter` handles one declared selector. +- `IntegrationHeadInjector` inserts deterministic head markup. +- `IntegrationHtmlPostProcessor` is for bounded whole-document work that + cannot be performed during streaming. +- `IntegrationRequestFilter` makes an early request decision. -## Step-by-Step Integration +Build one `IntegrationRegistration` with only the hooks the feature needs. +Use `with_deferred_js()` only for a separately loaded integration module and +`without_js()` when another asset path owns delivery. Proxy routes should be +namespaced and bounded; do not introduce a general outbound proxy. -### 1. Define Integration Configuration +## Compiling core-neutral fixture -Add a `trusted-server.toml` block and any environment overrides under `TRUSTED_SERVER__INTEGRATIONS____*`. Configuration values are exposed to your module via `Settings::integration_config()`. +The fixture below registers an attribute rewriter and constructs every +required `RuntimeServices` service without an adapter dependency. The +documentation test extracts this exact fence and compiles it as an isolated +crate. -```toml -[integrations.my_integration] -endpoint = "https://example.com/api" -timeout_ms = 1000 -rewrite_scripts = true -``` - -### 2. Create the Integration Module - -Add a module under `crates/trusted-server-core/src/integrations//mod.rs` (see `crates/trusted-server-core/src/integrations/testlight.rs` for reference) and expose it in `crates/trusted-server-core/src/integrations/mod.rs`. - -Key pieces: + ```rust -#[derive(Deserialize, Validate)] -struct MyIntegrationConfig { - #[serde(default = "default_enabled")] - enabled: bool, - // … -} - -impl IntegrationConfig for MyIntegrationConfig { - fn is_enabled(&self) -> bool { self.enabled } -} - -pub struct MyIntegration { - config: MyIntegrationConfig, -} - -pub fn build(settings: &Settings) -> Option> { - let config = settings - .integration_config::("my_integration") - .ok() - .flatten()?; - Some(Arc::new(MyIntegration { config })) -} - -// Tests or scaffolding code can seed configs without hand-writing JSON: -settings - .integrations - .insert_config( - "my_integration", - &serde_json::json!({ - "enabled": true, - "endpoint": "https://example.com/api" - }), - )?; -``` - -`Settings::integration_config::` automatically deserializes the raw JSON blob, runs [`validator`](https://docs.rs/validator/latest/validator/) on the type, and drops configs whose `is_enabled` returns `false`. Always derive/implement `Validate` for schema enforcement and implement `IntegrationConfig` (typically wrapping a `#[serde(default)] enabled` flag) so operators can toggle integrations without code changes. - -### 3. Return an IntegrationRegistration +use std::net::IpAddr; +use std::sync::Arc; + +use error_stack::Report; +use trusted_server_core::integrations::{ + AttributeRewriteAction, IntegrationAttributeContext, + IntegrationAttributeRewriter, IntegrationRegistration, +}; +use trusted_server_core::platform::{ + BackendNamingPolicy, ClientInfo, GeoInfo, PlatformBackend, + PlatformBackendSpec, PlatformConfigStore, PlatformError, PlatformGeo, + PlatformSecretStore, RuntimeServices, StoreId, StoreName, + UnavailableHttpClient, UnavailableKvStore, +}; + +struct ReadOnlyStore; + +impl PlatformConfigStore for ReadOnlyStore { + fn get( + &self, + _store: &StoreName, + _key: &str, + ) -> Result> { + Err(Report::new(PlatformError::Unsupported)) + } -Each integration registers itself via a `register` function that returns an `IntegrationRegistration`. This object describes which HTTP proxies and HTML rewrites the integration exposes: + fn put( + &self, + _store: &StoreId, + _key: &str, + _value: &str, + ) -> Result<(), Report> { + Err(Report::new(PlatformError::Unsupported)) + } -```rust -pub fn register(settings: &Settings) -> Option { - let integration = build(settings)?; - Some( - IntegrationRegistration::builder("my_integration") - .with_proxy(integration.clone()) - .with_attribute_rewriter(integration.clone()) - .with_script_rewriter(integration.clone()) - .with_head_injector(integration) - .with_asset("my_integration") - .build(), - ) + fn delete( + &self, + _store: &StoreId, + _key: &str, + ) -> Result<(), Report> { + Err(Report::new(PlatformError::Unsupported)) + } } -``` - -Any combination of the vectors may be populated. Modules that only need HTML rewrites can skip the `proxies` field altogether, and vice versa. The registry automatically iterates over the static builder list in `crates/trusted-server-core/src/integrations/mod.rs`, so adding the new `register` function is enough to make the integration discoverable. - -### 4. Implement IntegrationProxy for Endpoints -Implement the trait from `registry.rs` when your integration needs its own HTTP entrypoint: - -```rust -#[async_trait(?Send)] -impl IntegrationProxy for MyIntegration { - fn integration_name(&self) -> &'static str { - "my_integration" +impl PlatformSecretStore for ReadOnlyStore { + fn get_bytes( + &self, + _store: &StoreName, + _key: &str, + ) -> Result, Report> { + Err(Report::new(PlatformError::Unsupported)) } - fn routes(&self) -> Vec { - vec![ - self.post("/auction"), - self.get("/status"), - ] + fn create( + &self, + _store: &StoreId, + _key: &str, + _value: &str, + ) -> Result<(), Report> { + Err(Report::new(PlatformError::Unsupported)) } - async fn handle( + fn delete( &self, - settings: &Settings, - req: Request, - ) -> Result> { - // Parse/generate EC IDs, forward upstream, and return the response. + _store: &StoreId, + _key: &str, + ) -> Result<(), Report> { + Err(Report::new(PlatformError::Unsupported)) } } -``` - -::: tip Route Helpers -Use the provided helper methods to automatically namespace your routes under `/integrations/{integration_name()}/`. Available helpers: `get()`, `post()`, `put()`, `delete()`, and `patch()`. This lets you define routes with just their relative paths (e.g., `self.post("/auction")` becomes `"/integrations/my_integration/auction"`). -::: - -Routes are matched verbatim in `crates/trusted-server-adapter-fastly/src/main.rs`, so stick to stable paths and register whichever HTTP methods you need. **New integrations should namespace their routes under `/integrations/{INTEGRATION_NAME}/`** using the helper methods for consistency, but you can define routes manually if needed (e.g., for backwards compatibility). - -The shared context already injects Trusted Server logging, headers, and error handling; the handler only needs to deserialize the request, call the upstream endpoint, and stamp integration-specific headers. - -#### Proxying Upstream Requests - -Use the shared helper in `crates/trusted-server-core/src/proxy.rs` to forward requests so you automatically get the same header copying, redirect handling, HTML/CSS rewrite behavior, and EC ID handling the first-party proxy uses: -```rust -use crate::proxy::{proxy_request, ProxyRequestConfig}; -use fastly::http::{header, HeaderValue}; - -let payload = serde_json::to_vec(&my_body)?; -let response = proxy_request( - settings, - req, - ProxyRequestConfig::new(&self.config.endpoint) - .with_body(payload) - .with_header(header::CONTENT_TYPE, HeaderValue::from_static("application/json")) - .with_streaming(), // stream passthrough; disable if you need HTML rewrites -) -.await?; -``` - -Set `forward_ec_id` to `false` if the upstream should not receive the caller's EC ID (`Testlight` does this), and disable `follow_redirects` if you need to surface redirects directly to the caller. - -**Streaming passthrough example:** - -```rust -let response = proxy_request( - settings, - req, - ProxyRequestConfig::new("https://example.com/pixel") - .with_streaming() // no HTML/CSS rewrites; preserves origin compression -); -``` +struct FixtureBackend; -::: info When to Use Streaming -Use streaming when the upstream response is binary or large and you do not need creative rewrites. Keep the default (non-streaming) mode when you want HTML/CSS content rewritten through the existing creative pipeline. -::: - -### 5. Implement HTML Rewrite Hooks (Optional) - -If the integration needs to rewrite script/link tags or inject HTML, implement `IntegrationAttributeRewriter` for attribute mutation and `IntegrationScriptRewriter` for inline `"#, - ctx.request_host - )] +impl PlatformGeo for FixtureGeo { + fn lookup( + &self, + _client_ip: Option, + ) -> Result, Report> { + Ok(None) } } -``` - -`html_processor.rs` calls `head_inserts` once per HTML response when the `` element is first encountered. The returned snippets are concatenated before the unified script tag, so ordering between integrations is not guaranteed — keep snippets self-contained. - -::: tip When to Use Head Injection -Use `IntegrationHeadInjector` when you need to emit configuration, inline scripts, or `` tags that must appear early in ``. For attribute or script content changes on existing elements, prefer `IntegrationAttributeRewriter` or `IntegrationScriptRewriter` instead. -::: - -### 6. Register the Module - -Add the module to `crates/trusted-server-core/src/integrations/mod.rs`'s builder list. The registry will call its `register` function automatically. Once registered: - -- `crates/trusted-server-adapter-fastly/src/main.rs` automatically exposes the declared route(s). -- `handle_publisher_request` receives the same registry so HTML responses get integration shims without further code changes. -- `IntegrationRegistry::registered_integrations()` exposes a machine-readable summary of hooks for tests, tooling, or diagnostics. -- Declared assets are injected automatically into ``; the runtime emits `