feat: trace post processing and Valkey sessions

[seonghobae]

Summary

• install opt-in OpenTelemetry API/SDK/OTLP HTTP export in LineageWeave and the contextual-orchestrator image
• propagate the existing post-scoped session as X-LineageWeave-Session-Id
• trace HTTP LLM/VISION/embedding calls, Valkey stream producers, and worker reads without body, actor, source identifier, credential, or response content
• preserve the existing worker metadata reconstruction and add ADR/doctoring traceability against the organization GRC ADR 0009

Stack

This PR is stacked on #329 (fix/buyer-safe-image-captions). The image pins contextual-orchestrator PR #802 head 7b8f7e3a44a16b5eb305d7c3cac671100ed0932f.

Validation

• uv run --with pytest --with httpx pytest tests/test_observability.py tests/test_http_client.py -q -> 13 passed
• uv run --with pytest --with httpx --with asyncpg --with redis --with fastapi pytest tests/test_observability.py tests/test_http_client.py tests/test_activity_stream.py tests/test_post_content_normalization.py -q -> 39 passed
• uv run --with pytest --with httpx --with asyncpg --with redis --with fastapi pytest tests/test_post_content_worker.py tests/test_analysis_run_worker.py -q -> 11 passed
• docker compose build orchestrator -> passed; OTel packages installed in image
• targeted Ruff and git diff --check -> passed

---

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 8a634348-08d6-4d32-a2a1-78e7fd80a635

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[seonghobae]

Additional local verification for the current head 8af20c90fb3520626ad82ca3214d7d951d00c531:

• Full Python suite against an isolated PostgreSQL 18 test volume/port: 757 passed, 16 skipped, 4 warnings.
• The default real-data PostgreSQL volume was not used for fixture creation; its existing data was preserved.
• Compose orchestrator image build and container import check passed with OpenTelemetry API/SDK/OTLP HTTP exporter installed.

[seonghobae]

Addressed the Devin finding at current head 66dfa0c:

• normalize OTEL_EXPORTER_OTLP_ENDPOINT base URLs to the explicit /v1/traces signal path, without duplicating an already-specified signal path.

Evidence: LineageWeave focused HTTP, observability, and Valkey worker suite 51 passed; changed observability files Ruff checks passed.

[seonghobae]

Current-head audit and root-cause repair (2026-08-21):

• Exact head: 4932c6518880d6d9f780dfacd453f31f2741ff57
• Exact base: fix/buyer-safe-image-captions at 9b7ef93a4708993c7d5af28c67c28b227f59437a
• Fixed the remaining provider-error persistence leak in post_content_worker: exception text is no longer written to the ingestion ledger; the stable retry action is persisted while the full exception remains server-side logging only.
• Regression assertion now rejects the raw provider message in persisted transition arguments.
• Full backend: 758 passed, 16 skipped, 4 warnings; focused worker/observability: 14 passed; CodeGraph healthy; compileall and git diff --check passed.

Merge remains gated: REVIEW_REQUIRED, required hosted Checks queued, and no independent formal approval at this exact head.

[seonghobae]

Restacked on the repaired parent #329 without force-push.

• Exact head: bdfa3bc6d702ca6deb067114ee99fa31b92484bf
• Exact base: fix/buyer-safe-image-captions at c04f228a31e2b6ca1acb9e01a0b621b2d5e209e9
• Inherited the verified Vision SSRF fail-closed boundary from fix: keep internal image instructions out of buyer evidence #329 and retained the provider-detail ledger fix in this PR.
• Full backend on this exact stack: 762 passed, 16 skipped, 4 warnings; CodeGraph healthy and diff checks passed. Frontend 142 passed, lint/build/Storybook passed on the unchanged inherited frontend tree.

Required hosted Checks are queued and formal independent approval is still absent; do not merge yet.

[seonghobae]

Restacked normally onto repaired parent #329 exact head 29a606b8; child exact head is b751c1ef. The parent now closes raw provider-error leakage and the child retains stable durable post-content failure details.

Verification on exact stack: full backend suite 762 passed, 16 skipped, 4 warnings; git diff --check passed. The PR remains blocked pending independent review and terminal hosted Checks.

[seonghobae]

Current-head OTEL evidence for e308ce9239b33effd1d0319aff09f23286ca722f:

• Added standard W3C trace-context injection inside the existing lineageweave.http.post_json span.
• Preserved the existing bounded X-LineageWeave-Session-Id and post metadata correlation; no prompt, body, provider response, secret, or raw identity enters the span.
• Regression coverage: tests/test_observability.py passed (4 passed); compileall passed.
• This satisfies the LineageWeave side of GRC ADR 0012/runbook propagation; remote required Checks are still queued and the PR remains blocked pending independent review.