Skip to content

Add managed LiveRamp RampID integration - #1054

Open
prk-Jr wants to merge 56 commits into
mainfrom
issue-355-liveramp-integration
Open

Add managed LiveRamp RampID integration#1054
prk-Jr wants to merge 56 commits into
mainfrom
issue-355-liveramp-integration

Conversation

@prk-Jr

@prk-Jr prk-Jr commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Add managed LiveRamp RampID support to Prebid so operators can configure IdentityLink centrally while preserving publisher User ID modules.
  • Carry opaque liveramp.com EIDs through the existing auction, consent, edge-cookie, and EC/KV paths, with generated-artifact coverage for exact default TCF behavior.
  • Keep ATS Direct audience segments and server-side identity resolution out of scope; live validation remains credential-gated.

Status

Code complete; live LiveRamp validation pending IABTechLab/uid2-optout#385.

Changes

File Change
.cargo/config.toml Add host-target clippy aliases for the CLI and OpenRTB codegen crates.
CLAUDE.md Include the new host-target aliases in the documented CI gates.
crates/trusted-server-cli/src/prebid_bundle.rs Reject LiveRamp bundle configuration that explicitly omits identityLinkIdSystem.
crates/trusted-server-core/src/consent/mod.rs Cover source-agnostic EID consent gating with multiple identity sources.
crates/trusted-server-core/src/ec/prebid_eids.rs Verify opaque LiveRamp envelopes survive bounded edge-cookie persistence.
crates/trusted-server-core/src/integrations/prebid.rs Add typed LiveRamp configuration, validation, defaults, and browser injection.
crates/trusted-server-js/lib/build-prebid-external.mjs Bundle IdentityLink and tcfControl with generated Prebid artifacts.
crates/trusted-server-js/lib/src/integrations/prebid/index.ts Install and protect the operator-managed IdentityLink entry while preserving publisher entries.
crates/trusted-server-js/lib/test/build-prebid-external.test.mjs Cover generated bundle metadata and explicit IdentityLink selection.
crates/trusted-server-js/lib/test/integrations/prebid/index.test.ts Cover ownership, queue ordering, API guards, EID transport, privacy, and idempotence.
crates/trusted-server-js/lib/test/integrations/prebid/user_id_modules.test.ts Cover the LiveRamp module mapping and default preset.
crates/trusted-server-js/lib/test/prebid-artifact-integration.test.mjs Exercise the real bundle and shim, including partial userSync updates and auction EID output.
crates/trusted-server-js/lib/test/prebid-consent-enforcement.test.mjs Verify Purpose 1, Purposes 3/4, and LiveRamp vendor consent independently against generated artifacts.
docs/guide/configuration.md Document LiveRamp settings, defaults, and validation constraints.
docs/guide/integrations/prebid.md Document setup, ownership limits, exact TCF defaults, network/storage effects, failure modes, and live validation.
docs/superpowers/plans/2026-08-21-liveramp-integration.md Record the reviewed implementation, correction, and verification plan.
docs/superpowers/specs/2026-08-21-liveramp-integration-design.md Specify architecture, consent behavior, security boundaries, tests, and excluded ATS Direct scope.
trusted-server.example.toml Add an example managed LiveRamp configuration.

Closes

Closes #355

Test plan

  • cargo test-fastly && cargo test-axum && cargo test-cloudflare && cargo test-spin
  • cargo clippy-fastly && cargo clippy-axum && cargo clippy-cloudflare && cargo clippy-cloudflare-wasm && cargo clippy-spin-native && cargo clippy-spin-wasm && cargo clippy-cli && cargo clippy-codegen
  • cargo fmt --all -- --check
  • JS tests: cd crates/trusted-server-js/lib && npx vitest run
  • JS build: cd crates/trusted-server-js/lib && node build-all.mjs
  • JS lint/format: cd crates/trusted-server-js/lib && npm run lint && npm run format
  • CLI tests: ./scripts/test-cli.sh
  • Docs format: cd docs && npm run format
  • WASM build: cargo build --package trusted-server-adapter-fastly --release --target wasm32-wasip1
  • Manual testing via fastly compute serve
  • Other: generated Prebid artifact tests cover managed configuration and the independent TCF matrix.

Hardening note

This change does not add config-derived regular-expression or pattern compilation. Invalid enabled LiveRamp configuration is rejected by typed validation and covered by Rust and CLI regression tests.

Checklist

  • Changes follow CLAUDE.md conventions.
  • No new unwrap() calls were introduced in production code.
  • No direct stdout logging was introduced in production code.
  • New behavior has automated coverage.
  • No secrets, credentials, live Placement IDs, or RampID envelope values are committed.

Acceptance boundary

RampID identity envelopes traverse the existing Prebid auction path; ATS Direct audience segments are not passed by this implementation.

@prk-Jr prk-Jr self-assigned this Aug 21, 2026
prk-Jr and others added 8 commits August 21, 2026 16:57
The bundle imported consentManagementTcf, consentManagementGpp and
consentManagementUsp, which only retrieve the consent signal. The activity
controls that act on it live in Prebid's tcfControl module, which was not
bundled, so a denied purpose changed nothing.

Reproduced with the real bundle and shim in JSDOM against a CMP stub denying
Purpose 1 and vendor 97: identityLink still issued a credentialed GET to
LiveRamp's envelope endpoint, still wrote idl_env and its bookkeeping cookies,
and the envelope still reached the ts-eids cookie. The run was identical to the
consent-granted one. Server-side gating stops EID forwarding only, not the
browser's collection and storage.

Import tcfControl alongside the consent collectors and lock the behavior with a
regression test that evaluates both production artifacts: denied purposes must
produce no vendor call and no storage, granted purposes must still resolve.

Note the scope: tcfControl enforces for every bidder and User ID module in the
bundle, not only LiveRamp. Regenerating a bundle is a manual step, so this
reaches a publisher only when `ts prebid bundle` is re-run.
`[integrations.prebid.liveramp]` combined with a pinned
`bundle.user_id_modules` list that lacks identityLinkIdSystem is valid config
that builds a bundle unable to serve it. The managed identityLink entry the
server injects has no submodule to drive, and the only signal is a browser-side
diagnostic.

Fail in `ts prebid bundle`, which already reads both keys from the same file. An
absent list still takes the generator's default preset, which includes the
module.
Three fixes to the setConfig and mergeConfig guards:

Skip seeding when getConfig is missing. `hasPrebidJsApi` only requires
registerBidAdapter, so getConfig can be absent. The previous fallback read an
empty entry list and then seeded the managed entry alone, silently dropping
every publisher module already configured. Log and leave the wrappers installed
instead, so a later publisher userIds call still gets the managed entry.

Build the managed entry per call. Prebid retains what it receives as
`submodule.config` for the life of the page, so one shared instance let any
mutation there leak into every later normalization.

Keep the normalization guard but report the error. Publisher configuration is
arbitrary page data and a throwing accessor must not break the publisher's own
setConfig call, so the guard stays; it now logs what failed instead of
discarding it. The redundant hasUserIdsPath check is dropped since
withManagedLiveRampUserId already performs it.
merge_auction_eids_deduplicates_liveramp_and_prefers_resolved_metadata was
merge_auction_eids_prefers_server_resolved_metadata_on_conflict with the source
string changed. merge_auction_eids is source-agnostic, so it added no coverage.

gate_eids_by_consent is all-or-nothing across sources. Rename its test and give
it a second source so the name no longer implies per-source gating that does not
exist.
The inline closure that mapped PrebidLiveRampConfig into its camel-cased
injection struct pushed rustfmt into indenting the whole payload literal an
extra level. A From implementation keeps the mapping next to the struct it
builds and restores the surrounding formatting.
The degraded-behavior table said only that no EID is forwarded when consent is
absent, which read as though nothing happened. Separate the TCF case, where
tcfControl now blocks resolution outright, from the US-state case, where the
server's forwarding gate is what applies.

Add the browser-side footprint the previous text omitted: the credentialed
request the page makes to LiveRamp's envelope endpoint, that Trusted Server does
not proxy it, the connect-src allowance a strict CSP needs, and every storage
entry IdentityLink writes. Note that the submodule prefers an ATS library
already on the page, and that this is therefore not a first-party replacement
for LiveRamp recognition.

Also record that the environment override cannot introduce the subsection, that
`ts prebid bundle` rejects a module list without identityLinkIdSystem, that
regenerating a bundle now enforces TCF for every bidder and User ID module, and
that KV persistence needs a matching [[ec.partners]] entry. Correct the design
document's consent section, which assumed the consent-management modules were
themselves the enforcement point, and flag the remaining GPP gap.
prk-Jr added 7 commits August 24, 2026 14:05
CI lints trusted-server-cli and trusted-server-openrtb-codegen explicitly
because no adapter-scoped alias covers them, but both invocations pin
x86_64-unknown-linux-gnu. There was no command a developer could run locally to
reproduce them, and CLAUDE.md's CI gate list named only the six adapter aliases,
so running the documented gates locally passed while CI failed on a
clippy::doc_markdown error in the CLI.

Add clippy-cli and clippy-codegen. Both omit --target so they build for the
host, and record them in the gate list.
CodeQL flagged js/incomplete-url-substring-sanitization on both assertions.
These are not sanitizers — the URLs come from the test's own fetch spy — so the
alert is not a vulnerability, but the substring test is still too loose for an
assertion: it would also match an unrelated host carrying api.rlcdn.com in its
name or query string, letting the granted-consent case count the wrong request.

Compare the parsed hostname instead. Verified the test is still non-vacuous:
dropping the tcfControl import fails two of its three cases.
@aram356 aram356 modified the milestones: 202608, 202609 Aug 31, 2026
@prk-Jr
prk-Jr marked this pull request as ready for review September 7, 2026 10:55
prk-Jr added a commit that referenced this pull request Sep 7, 2026

@ChristianPavilonis ChristianPavilonis left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Requesting changes for two consent-enforcement regressions and one managed User ID validation issue.

if (
!managedUserIds?.length ||
typeof window === 'undefined' ||
typeof tcfApi !== 'function' ||

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Do not seed managed IDs before a late-loading CMP is discoverable

TCF activation performs a one-time check for window.__tcfapi. If an asynchronous CMP installs it after the deferred shim runs, managed IDs are seeded with Prebid’s GDPR handler disabled. Using the generated artifacts, installing a Purpose-1-denying CMP immediately after the shim still produced a request to api.rlcdn.com without TCF parameters. Defer managed-ID initialization until CMP discovery completes, or reconfigure/fail closed when the API appears.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed and fixed in d269ff8.

Managed entries now stay out of every configuration Prebid sees until CMP discovery concludes, rather than seeding and hoping to reconfigure later. Discovery resolves on whichever comes first:

  • window.__tcfapi is already callable at shim time — activate and seed synchronously, as before.
  • A CMP assigns window.__tcfapi later — watchForLateTcfApi installs a configurable accessor, restores the plain property on the first function assignment, then activates and seeds. If a CMP redefines the property outright, the descriptor check leaves it alone and the fallback below still reads the live API.
  • No CMP by the first requestBids — a conforming CMP installs its stub before vendor tags request bids, so that is the last useful moment to conclude none is coming. Managed entries seed there with no TCF activation, which is the correct posture for a page that genuinely has no CMP.

While discovery is pending, normalizePublisherConfig passes publisher config through untouched; managed entries are merged onto the effective configuration once it resolves, so nothing the publisher set is lost.

Two regression tests cover it (defers managed User ID seeding until a late CMP installs __tcfapi, seeds managed User IDs at the first auction when no CMP ever appears), and prebid-artifact-integration.test.mjs now asserts the deferral against the real generated bundle before settling it. Both fail against the previous behaviour.

const activation: ManagedTcfConsentActivation = { acceptCmpEvents: true };
try {
const effectiveConsent = effectiveConsentManagement ?? {};
if (Object.prototype.hasOwnProperty.call(effectiveConsent, 'gdpr')) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Treat Prebid’s legacy consent shape as publisher-owned

Prebid 10.26 supports legacy TCF configuration directly under consentManagement, such as { cmpApi: "static", consentData: ... }. Because this check only recognizes an own gdpr property, the shim appends gdpr: { cmpApi: "iab" }; Prebid then interprets the object as the modern shape and ignores the publisher’s top-level static consent. A generated-artifact reproduction with legacy static consent denying Purpose 1 and __tcfapi granting it made the LiveRamp request. Recognize legacy TCF configuration here and in publisherClaimsGdprOwnership.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed and fixed in d269ff8.

Verified against the pinned artifact: modules/consentManagementTcf.ts:200 reads the config as config.gdpr || config.usp || config.gpp ? config.gdpr : config, so a legacy object carries no namespace key and appending gdpr demotes every legacy setting.

Ownership detection now mirrors that rule rather than probing for an own gdpr property, and is shared by activateManagedUserIdTcfConsent and publisherClaimsGdprOwnership:

  • A truthy own gdpr, usp, or gpp selects the namespaced shape; the publisher owns TCF only when gdpr itself is truthy. A usp-only or gpp-only config still gets the automatic collector, since Prebid would leave TCF unconfigured there.
  • Any other non-empty object is a legacy top-level TCF config and is left alone.
  • Namespace probing uses hasOwnProperty deliberately, so a consentManagement object whose descriptors cannot be read still throws into the existing fail-safe path instead of being silently treated as empty.

Recognizing the legacy shape exposed a second defect on the retirement path that I fixed alongside it: mergeConfig deep-merges onto the current config, so retiring with gdpr: { enabled: false } survives a legacy publisher merge and Prebid then reads { enabled: false } and disables the TCF module entirely — fail-open, and worse than the original bug. removeAutomaticGdprNamespace now drops the key with a setConfig topic replacement after the merge, so the publisher’s legacy config governs.

Three regression tests: leaves a legacy top-level TCF consent configuration to the publisher, retires automatic IAB consent when a late setConfig uses the legacy TCF shape, drops the automatic gdpr namespace when a merge uses the legacy TCF shape. All three fail against the previous behaviour. activates managed GDPR consent alongside a publisher USP namespace guards the non-regression.

.collect::<Vec<_>>();
candidates.sort();
candidates.dedup();

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Reject multiple aliases resolving to the same User ID module

Each managed name is accepted independently even when multiple names resolve to one module. The registry maps both sharedId and pubCommonId to sharedIdSystem, so configuring both passes validation. Prebid registers one submodule with those name/alias values and selects the first matching configuration, silently ignoring the other managed entry. Reject duplicate resolved module_name values or canonicalize aliases before generation.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed and fixed in d269ff8.

The checked-in registry has exactly one such module today: sharedIdSystem <- sharedId, pubCommonId. resolve_managed_user_id_modules now runs reject_managed_user_id_module_collisions over the resolved set and fails before generation, naming both managed names, the shared module, and the registry path. Rejecting rather than canonicalizing, since collapsing two entries would silently pick one operator configuration over the other — the same ambiguity, just moved earlier.

Two existing tests encoded the old behaviour and are now rejection tests:

  • managed_names_resolve_aliases_to_registered_modules -> split into managed_name_resolves_an_alias_to_its_registered_module (single alias still resolves) and managed_names_sharing_one_module_are_rejected.
  • run_bundle_accepts_two_aliases_backed_by_one_module -> run_bundle_rejects_two_aliases_backed_by_one_module, which also asserts nothing is generated and the configured bundle hash and SRI are left untouched.

Docs updated in docs/guide/integrations/prebid.md and docs/guide/configuration.md.

Main split the Prebid configuration into a browser-only
`PrebidIntegrationConfig` and a test-only `LegacyPrebidServerConfig`, so
move `managed_user_ids` onto the live browser config and map it through
the legacy conversion. Left on the legacy struct the field would have
compiled but never reached a running service.

Also un-gate the `HashSet` import now that managed User ID name
validation runs in production builds, and update the script-breakout
assertion for main's stronger JSON escaping, which escapes every `<`
rather than only `</script>`.
TCF activation read `window.__tcfapi` once, but managed User ID seeding
ran regardless. A CMP that installed itself after the deferred shim left
managed modules seeded with Prebid's GDPR handler disabled, and the
module fired its vendor request with no TCF parameters. No later
reconfiguration can recall that request, so hold managed entries out of
every configuration Prebid sees until CMP discovery concludes: at once
when `__tcfapi` is already callable, on a watched late assignment, or at
the first `requestBids`, since a conforming CMP installs its stub before
vendor tags request bids.

Prebid 10.26 reads `consentManagement` as
`config.gdpr || config.usp || config.gpp ? config.gdpr : config`, so a
legacy top-level TCF configuration carries no namespace key. Recognizing
only an own `gdpr` property let the shim append `gdpr: { cmpApi: "iab" }`
to such an object, which flipped Prebid to the namespaced shape and
discarded the publisher's static consent. Mirror Prebid's own rule in
both ownership checks. Retirement then needs a matching path: a
`mergeConfig` deep merge carries `gdpr: { enabled: false }` forward and
would disable the TCF module entirely, so remove the namespace instead of
leaving it disabled.

`ts prebid bundle` accepted each managed name independently even when
several resolved to one module. The registry maps both `sharedId` and
`pubCommonId` to `sharedIdSystem`; Prebid registers one submodule for a
module's name and its aliases and selects the first matching entry, so
the second managed configuration was silently ignored. Reject duplicate
resolved module names before generation.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Investigate and document LiveRamp integration

6 participants