Conversation
A daily pg_cron job (pgflow_telemetry.report) aggregates yesterday's pgflow.* activity into coarse, identifier-free buckets (active-day, worker versions, run/worker counts, flow shapes, feature adoption, durations) and posts one small JSON payload via pg_net to a Cloudflare Worker backed by Workers Analytics Engine. Edge workers stamp their package version into pgflow.workers at registration.
Design decisions: the cron.job row is the only switch — the migration schedules it exactly once and nothing re-schedules it, so disable() stays permanent across upgrades. pgflow_telemetry.sent_reports doubles as user-auditable payload log and per-day dedup marker. report() gates run in order: inactive day, already reported, is_local(); one pg_net attempt, 5s statement timeout, no retries, no response inspection. Metric names, bucket strings, and count buckets are closed allowlists identical across SQL (0133), the ingest worker (apps/telemetry-worker), and docs; anything else never leaves the database. Local supabase-start stacks never send.
Compatibility fixes found during implementation: Deno 2.1.4 requires the JSON import attribute ('with { type: json }') and a default import for package.json version lookup, so jsr.json publish.includes package.json and the e2e vendor script copies it; bucket_steps takes bigint because steps_per_flow passes count(*); 0136 uses $disable$/$enable$ dollar-quote tags because the plan's nested $$ would terminate enable()'s body early. Plan test bugs corrected with rationale in the tests: 90s lands in 1-4.9m not 1-9.9s; bucket_count(2) is '2-3'; the 65-contribution body always exceeds the 2KB cap so 413 fires before the count guard.
Verification: 4 new pgTAP files (red/green, migration replay with deps), telemetry-worker vitest 10, full pgTAP suite 314 files/1606 tests, nx affected build+test, e2e 13/13, live report() returns skipped: local with zero audit rows, live worker registration stamps 0.17.0, website build. Ingest worker deployment (wrangler deploy + smoke) is a manual follow-up and is not part of this commit.
🦋 Changeset detectedLatest commit: 4d10e7f The changes in this PR will be included in the next version bump. This PR includes changesets to release 5 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
View your CI Pipeline Execution ↗ for commit 4f7f279
💡 Verify your cache is correct by running tasks in a sandbox. Read docs ↗ ☁️ Nx Cloud last updated this comment at |
CI build-and-test failed on @pgflow/telemetry-worker:typecheck with TS5069: the @nx/js plugin infers 'tsc --build --emitDeclarationOnly' for every root-tsconfig project, but this wrangler-deployed worker's tsconfig sets noEmit and never emits declarations. A package.json 'typecheck' script overrides the inferred target with tsc --noEmit — the same per-project override pattern pkgs/client uses (project.json noEmit target) instead of touching nx.json. Verified: focused typecheck exit=0; nx affected -t lint typecheck test green except the known unrelated demo Groq tests (sandbox GROQ_API_KEY, decommissioned llama-3.1-8b-instant).
![graphite-app[bot]](https://avatars.githubusercontent.com/in/158384?s=60&v=4){kind=small}
Comment on lines
+120
to
+126
| for (const c of contributions) { | ||
| env.PGFLOW_TELEMETRY.writeDataPoint({ | ||
| indexes: [c.metric as string], | ||
| blobs: [c.bucket as string], | ||
| doubles: [1], | ||
| }); | ||
| } |
Contributor
There was a problem hiding this comment.
The count field from contributions is validated (lines 113-117) but never used when writing data points. All contributions are written with doubles: [1] regardless of their actual count value. This means aggregated count data (like "count": "8-15") is completely lost.
Fix:
for (const c of contributions) {
env.PGFLOW_TELEMETRY.writeDataPoint({
indexes: [c.metric as string],
blobs: [c.bucket as string, c.count as string ?? ''],
doubles: [1],
});
}Or if count should affect the double value, the SQL queries building these payloads (like line 27 in 0133_function_build_telemetry_payload.sql) would need to be updated to not include the count field at all since it cannot be meaningfully transmitted in the current schema.
Suggested change
| for (const c of contributions) { | |
| env.PGFLOW_TELEMETRY.writeDataPoint({ | |
| indexes: [c.metric as string], | |
| blobs: [c.bucket as string], | |
| doubles: [1], | |
| }); | |
| } | |
| for (const c of contributions) { | |
| env.PGFLOW_TELEMETRY.writeDataPoint({ | |
| indexes: [c.metric as string], | |
| blobs: [c.bucket as string, c.count as string ?? ''], | |
| doubles: [1], | |
| }); | |
| } | |
Spotted by Graphite
Is this helpful? React 👍 or 👎 to let us know.
Contributor
🔍 Preview Deployment: Website✅ Deployment successful! 🔗 Preview URL: https://pr-684.pgflow.pages.dev 📝 Details:
_Last updated: _ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Anonymous opt-out telemetry
Summary
Ships the researched daily anonymous telemetry (plan:
Plans/2026-09-18-anonymous-telemetry.md, spec:Runs/anonymous-telemetry-research/research.md).pgflow_telemetryschema:sent_reportsaudit/dedup table, bucket helpers,build_payload()covering all 26 metrics,preview()(never sends),report()with gates (inactive day → already reported → local), onepg_netattempt with 5 s timeout, no retries, andenable()/disable()where thecron.jobrow is the switch. The Atlas migration schedules the daily job exactly once; later migrations never re-schedule it.pgflow.workers.pgflow_versioncolumn; edge workers stamp their package version at registration (Deno/JSR-compatible JSON import, vendored for e2e).apps/telemetry-worker: strict allowlist validation (metric names, bucket strings, semver), one Analytics Engine data point per contribution, no cookies, no body logging, 2 KB / 64-contribution caps.pgflow_telemetry.sent_reports;preview()shows any day's exact bytes; local dev stacks never send./reference/telemetry/page, sidebar + reference card + redirect for the renamed news article (combined 0.17.1), changeset (@pgflow/core,@pgflow/edge-worker: patch).Deferred (manual follow-up, not in this PR):
wrangler deployof the ingest worker and the live endpoint smoke test. The SQL endpoint constant ishttps://pgflow-telemetry.workers.dev; if the deployed subdomain differs, it needs a follow-up change per the plan's Task 8 Step 5.Checks
telemetry/{workers_version,schema,payload,report}.test.sqlnx affected -t build test— pass, exceptdemo:test: 4 Groq-API tests fail only when aGROQ_API_KEYis present in the environment (modelllama-3.1-8b-instantdecommissioned); demo sources untouched by this PR and CI has no such keyverify-migrations,gen-types,verify-gen-types— passpreview()shows real contributions;report()returnsskipped: local, zero audit rows; worker registration stampspgflow_version = 0.17.0