Skip to content

fix(analytics): preserve validated referring hostnames - #64

Merged
scttbnsn merged 1 commit into
devfrom
fix/analytics-referrers
Sep 13, 2026
Merged

scttbnsn merged 1 commit into
devfrom
fix/analytics-referrers

Conversation

@scttbnsn

@scttbnsn scttbnsn commented Sep 13, 2026

Copy link
Copy Markdown
Collaborator

The production analytics clients disable referrer capture and discard the referring-domain property when rebuilding events. This enables collection and retains validated bare hostnames or the direct-traffic marker, while dropping full URLs, malformed values, and campaign data. Cookieless storage settings stay in force.

Regression tests first reproduced the missing field and disabled configuration, then passed with the fix. Existing analytics checks and production website builds pass. Production ingestion verification follows deployment.

Summary by CodeRabbit

  • New Features

    • Analytics now saves referring-page information when available.
    • Valid referring domains are included in event data and normalized to lowercase.
    • Direct visits continue to be represented consistently.
  • Privacy

    • Invalid or malformed referring-domain values are excluded.
    • Full referrer URLs and campaign-source values remain excluded from event data.
  • Tests

    • Added coverage for valid, invalid, normalized, and excluded referring information across tracked events.

@coderabbitai

coderabbitai Bot commented Sep 13, 2026

Copy link
Copy Markdown

Review Change StackReview Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Advanced

Run ID: 552402c9-dd64-4c5f-824c-459da835b9c6

📥 Commits

Reviewing files that changed from the base of the PR and between b620263 and 17b935d.

📒 Files selected for processing (4)
  • frontend/instrumentation-client.ts
  • frontend/lib/posthog-privacy.ts
  • frontend/test/posthog-source.test.mjs
  • frontend/test/posthog.test.ts

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.


Walkthrough

The PostHog configuration now saves referrers. Event processing validates referring domains, lowercases valid domains, preserves $direct, and excludes invalid values. Tests cover configuration presence and sanitization across pageview, pageleave, and web-vitals events.

Changes

Referrer privacy handling

Layer / File(s) Summary
Referring-domain sanitization
frontend/lib/posthog-privacy.ts, frontend/test/posthog.test.ts
Valid referring domains are lowercased and included in common event properties. Invalid, padded, overlong, or URL-like values are excluded. $referrer and utm_source remain excluded.
Referrer capture configuration
frontend/instrumentation-client.ts, frontend/test/posthog-source.test.mjs
PostHog initialization sets save_referrer to true. The source test checks that this option is present.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix

Merge Risk: 🔵 Low · up to 17b93

Full referrer URLs are not sent with rebuilt events, but are briefly retained in browser memory after enabling referrer capture. This is bounded because durable persistence remains disabled, yet should be explicitly accepted or avoided before merging.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 3 functions across 4 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the primary change: preserving validated referring hostnames in analytics.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
  • Fix all pre-merge checks with AI
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/analytics-referrers

Warning

Some tools did not complete. Review the errors below.

🔧 Biome (2.5.10)
frontend/instrumentation-client.ts

Biome could not lint this file: nested root configuration. Check the repository's Biome configuration and plugins.

frontend/lib/posthog-privacy.ts

Biome could not lint this file: nested root configuration. Check the repository's Biome configuration and plugins.

frontend/test/posthog-source.test.mjs

Biome could not lint this file: nested root configuration. Check the repository's Biome configuration and plugins.

  • 1 others

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@vercel

vercel Bot commented Sep 13, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
codeswhat-website Ready Ready Preview Sep 13, 2026 2:55pm UTC

@scttbnsn

Copy link
Copy Markdown
Collaborator Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Sep 13, 2026

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@biggest-littlest biggest-littlest left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the hostname allowlist and regression coverage. Full URLs and campaign parameters remain excluded; website checks pass.

@ALARGECOMPANY ALARGECOMPANY left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified the configuration and sanitizer agree, with passing privacy regression tests and production build.

@scttbnsn
scttbnsn merged commit 2d3ccab into dev Sep 13, 2026
4 checks passed
scttbnsn added a commit that referenced this pull request Sep 13, 2026
* fix(analytics): forward $raw_user_agent and $host for cookieless ingestion (#52)

PostHog's cookieless server-hash step reads $raw_user_agent and $host
straight off event.properties and drops the event with a
cookieless_missing_user_agent/cookieless_missing_host ingestion warning
if either is absent. createCommonProperties rebuilt an allowlisted
properties object that dropped both, so every event was silently
discarded at ingestion. Forward them through; never add $ip, which
PostHog's capture service fills in server-side from the connection.

Co-authored-by: scttbnsn <80784472+scttbnsn@users.noreply.github.com>

* chore(config): remove the stale Cursor rules (#54)

* fix(analytics): promote cookieless ingestion fix to production (#53)

PostHog's cookieless server-hash step reads $raw_user_agent and $host
straight off event.properties and drops the event with a
cookieless_missing_user_agent/cookieless_missing_host ingestion warning
if either is absent. createCommonProperties rebuilt an allowlisted
properties object that dropped both, so every event was silently
discarded at ingestion. Forward them through; never add $ip, which
PostHog's capture service fills in server-side from the connection.

Co-authored-by: biggest-littlest <zap_inane.2p@icloud.com>

* chore(config): drop the stale Cursor rules folder

* docs(config): drop dangling .cursorrules references

---------

Co-authored-by: biggest-littlest <zap_inane.2p@icloud.com>

* docs(readme): describe what this repo is and how it deploys (#55)

* chore(gitignore): ignore the root .vercel link and history-backup bundles

* feat(analytics): capture $pageleave and send $pathname (#60)

Measured over the shared PostHog project, 208 of 432 sessions across
the five instrumented sites record zero duration, and PostHog's
built-in Web analytics Page/Entry page/Exit page tables return zero
rows. capture_pageleave was false, so a session's last recorded
timestamp is its last pageview, and a five-minute read of one page
scores as zero seconds.

Flipping the option alone fixes nothing: sanitizeEvent allowlisted
only $pageview, cta activated, and $web_vitals, so every $pageleave
posthog-js emitted would have been dropped silently with no error and
no ingestion warning. This adds a $pageleave branch that rebuilds the
event the same way $pageview does. capture_pageview is false here
(pageviews are captured by hand), so posthog-js's
_shouldCapturePageleave gate needs an explicit true rather than the
default.

$pathname is the property PostHog's page tables actually key off, and
it was never sent. It's bound to the already-sanitized `path` value,
never the raw pathname, so it can't carry a route outside
ALLOWED_ROUTES and adds no information the event wasn't already
sending. A regression test asserts the two never diverge.

No privacy option changes: cookieless_mode, person_profiles,
persistence, disable_persistence, respect_dnt, save_referrer, and
save_campaign_params are untouched.

Part of X16 in the ops execution plan.

* fix(seo): repair JSON-LD logo 404 and double-slash base URLs

- fix(seo): point Organization.logo at /icon-512x512.png; the referenced
  /logos/codeswhat-logo-green.png never existed, so crawlers got a 404
- fix(seo): strip trailing slashes from BASE_URL and reuse it in robots.ts
  and sitemap.ts, so a NEXT_PUBLIC_SITE_URL set with a trailing slash can't
  emit //sitemap.xml-style URLs
- chore(seo): disallow /studio/ in robots.txt; the capture pages already
  404 in production but the exclusion shouldn't depend on that guard
- chore(seo): 308 the stable *.vercel.app production aliases to
  codeswhat.com instead of serving duplicate content
- fix(api): stop forwarding EmailOctopus error detail to subscribe clients;
  log it server-side and return a fixed message

* build(deps): bump next to ^16.2.11 to clear all 35 Dependabot alerts

One-line range bump; npm resolves next 16.3.3, which also pulls the
patched transitive versions: postcss 8.5.23, nanoid 3.3.18, sharp 0.35.4.
npm audit now reports zero vulnerabilities. No code changes needed: the
app has no middleware, rewrites, server actions, CSP nonces, or
next/image usage, so none of the fixed CVEs required app-side work.

* build(deps): regenerate next-env.d.ts for next 16.3

- build(deps): pick up the root-params.d.ts reference next 16.3 adds
- ci(hooks): pass --no-errors-on-unmatched to the biome pre-commit job so
  committing only biome-ignored files (like next-env.d.ts) doesn't fail

* docs(roadmap): track web-analytics table coverage follow-ups (ops X37)

* chore: ignore .claude/ with a tracked line (#59)

It was covered only by .git/info/exclude, which protects one clone and nobody else's. Without a tracked line, `git add -A` in the parent stages a nested worktree as an embedded gitlink and `git clean -ffd` deletes it.

* docs(roadmap): point acquisition-data item at the ops analytics standard

* docs(roadmap): pageleave ratio is structural; note the bot-detection canary caveat

* chore(config): add .planning/ to .gitignore (#62)

* fix(analytics): preserve validated referring hostnames (#64)

---------

Co-authored-by: biggest-littlest <zap_inane.2p@icloud.com>
@scttbnsn
scttbnsn deleted the fix/analytics-referrers branch September 13, 2026 15:13
scttbnsn added a commit that referenced this pull request Sep 13, 2026
* fix(analytics): forward $raw_user_agent and $host for cookieless ingestion (#52)

PostHog's cookieless server-hash step reads $raw_user_agent and $host
straight off event.properties and drops the event with a
cookieless_missing_user_agent/cookieless_missing_host ingestion warning
if either is absent. createCommonProperties rebuilt an allowlisted
properties object that dropped both, so every event was silently
discarded at ingestion. Forward them through; never add $ip, which
PostHog's capture service fills in server-side from the connection.

Co-authored-by: scttbnsn <80784472+scttbnsn@users.noreply.github.com>

* chore(config): remove the stale Cursor rules (#54)

* fix(analytics): promote cookieless ingestion fix to production (#53)

PostHog's cookieless server-hash step reads $raw_user_agent and $host
straight off event.properties and drops the event with a
cookieless_missing_user_agent/cookieless_missing_host ingestion warning
if either is absent. createCommonProperties rebuilt an allowlisted
properties object that dropped both, so every event was silently
discarded at ingestion. Forward them through; never add $ip, which
PostHog's capture service fills in server-side from the connection.

Co-authored-by: biggest-littlest <zap_inane.2p@icloud.com>

* chore(config): drop the stale Cursor rules folder

* docs(config): drop dangling .cursorrules references

---------

Co-authored-by: biggest-littlest <zap_inane.2p@icloud.com>

* docs(readme): describe what this repo is and how it deploys (#55)

* chore(gitignore): ignore the root .vercel link and history-backup bundles

* feat(analytics): capture $pageleave and send $pathname (#60)

Measured over the shared PostHog project, 208 of 432 sessions across
the five instrumented sites record zero duration, and PostHog's
built-in Web analytics Page/Entry page/Exit page tables return zero
rows. capture_pageleave was false, so a session's last recorded
timestamp is its last pageview, and a five-minute read of one page
scores as zero seconds.

Flipping the option alone fixes nothing: sanitizeEvent allowlisted
only $pageview, cta activated, and $web_vitals, so every $pageleave
posthog-js emitted would have been dropped silently with no error and
no ingestion warning. This adds a $pageleave branch that rebuilds the
event the same way $pageview does. capture_pageview is false here
(pageviews are captured by hand), so posthog-js's
_shouldCapturePageleave gate needs an explicit true rather than the
default.

$pathname is the property PostHog's page tables actually key off, and
it was never sent. It's bound to the already-sanitized `path` value,
never the raw pathname, so it can't carry a route outside
ALLOWED_ROUTES and adds no information the event wasn't already
sending. A regression test asserts the two never diverge.

No privacy option changes: cookieless_mode, person_profiles,
persistence, disable_persistence, respect_dnt, save_referrer, and
save_campaign_params are untouched.

Part of X16 in the ops execution plan.

* fix(seo): repair JSON-LD logo 404 and double-slash base URLs

- fix(seo): point Organization.logo at /icon-512x512.png; the referenced
  /logos/codeswhat-logo-green.png never existed, so crawlers got a 404
- fix(seo): strip trailing slashes from BASE_URL and reuse it in robots.ts
  and sitemap.ts, so a NEXT_PUBLIC_SITE_URL set with a trailing slash can't
  emit //sitemap.xml-style URLs
- chore(seo): disallow /studio/ in robots.txt; the capture pages already
  404 in production but the exclusion shouldn't depend on that guard
- chore(seo): 308 the stable *.vercel.app production aliases to
  codeswhat.com instead of serving duplicate content
- fix(api): stop forwarding EmailOctopus error detail to subscribe clients;
  log it server-side and return a fixed message

* build(deps): bump next to ^16.2.11 to clear all 35 Dependabot alerts

One-line range bump; npm resolves next 16.3.3, which also pulls the
patched transitive versions: postcss 8.5.23, nanoid 3.3.18, sharp 0.35.4.
npm audit now reports zero vulnerabilities. No code changes needed: the
app has no middleware, rewrites, server actions, CSP nonces, or
next/image usage, so none of the fixed CVEs required app-side work.

* build(deps): regenerate next-env.d.ts for next 16.3

- build(deps): pick up the root-params.d.ts reference next 16.3 adds
- ci(hooks): pass --no-errors-on-unmatched to the biome pre-commit job so
  committing only biome-ignored files (like next-env.d.ts) doesn't fail

* docs(roadmap): track web-analytics table coverage follow-ups (ops X37)

* chore: ignore .claude/ with a tracked line (#59)

It was covered only by .git/info/exclude, which protects one clone and nobody else's. Without a tracked line, `git add -A` in the parent stages a nested worktree as an embedded gitlink and `git clean -ffd` deletes it.

* docs(roadmap): point acquisition-data item at the ops analytics standard

* docs(roadmap): pageleave ratio is structural; note the bot-detection canary caveat

* chore(config): add .planning/ to .gitignore (#62)

* fix(analytics): preserve validated referring hostnames (#64)

* fix(site): restore hydration and reject invalid signup bodies (#66)

---------

Co-authored-by: biggest-littlest <zap_inane.2p@icloud.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants