Skip to content

Say which write path the process took, because the fallback is silent - #144

Merged
ralyodio merged 1 commit into
mainfrom
worktree-write-path-visible
Aug 19, 2026
Merged

Say which write path the process took, because the fallback is silent#144
ralyodio merged 1 commit into
mainfrom
worktree-write-path-visible

Conversation

@ralyodio

Copy link
Copy Markdown
Contributor

Why

REDIS_URL has never been set on either production service. connect() falls back to the in-process queue when it is absent — deliberately, as what a Redis outage should degrade to — so every write has been serialized per process while the Redis queue (#132) and the folding built for it (#140) sat dormant.

Nothing was broken and nothing was logged. "We have a write queue" and "the write queue is running" looked identical from outside until somebody read the environment. That is a bad property for the component that decides the crawler's throughput.

What changed

connect() now announces its write path and the reason:

[db] write path: in-process (REDIS_URL is not set)
[db] write path: redis (one writer per cluster)

The reason is the point — in-process alone cannot distinguish a deliberate local run from a production service missing its broker. Announced once per path per process, since the web app calls connect() per request.

The decision moves into an exported writePath() that returns the path and reason and opens nothing.

A note on the test

The first version asserted against connect() directly and hung the suite — the client it returns keeps the event loop alive. That was a signal about the design, not just the test: the decision is the part worth testing, the client is not. Extracting writePath() made it a pure-function test that runs in milliseconds.

Full suite: 1,146 tests, 0 failures.

Follow-up

This is the observability half. Actually enabling BullMQ needs a Redis service and REDIS_URL set on both services — the Railway CLI goes interactive to create one, so that step is manual. Once it is set, this line is how you confirm it engaged.

Worth being clear that Redis will not multiply write throughput: SQLite permits one writer regardless. What it buys is one writer per cluster instead of one per process (poller and web currently contend), plus durability and retries on writes.

`REDIS_URL` has never been set on either production service. `connect()` falls
back to the in-process queue when it is absent -- deliberately, as what a Redis
outage should degrade to -- so every write has been serialized per process while
the Redis queue (#132) and the folding built for it (#140) sat dormant.

Nothing was broken and nothing was logged. "We have a write queue" and "the
write queue is running" looked identical from outside until somebody read the
environment, which is a bad property for the component that decides the
crawler's throughput.

So `connect()` now says which path it took and *why*: the reason is the point,
since "in-process" alone cannot distinguish a deliberate local run from a
production service missing its broker. Announced once per path per process,
because the web app calls `connect()` per request.

The decision moves into an exported `writePath()` that returns the path and the
reason and opens nothing. That is what made this testable: the first attempt
asserted against `connect()` itself and hung the suite, because the client it
returns keeps the event loop alive. The decision is the part worth testing; the
client is not.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@ralyodio
ralyodio merged commit 0dbe4ef into main Aug 19, 2026
3 checks passed
@ralyodio
ralyodio deleted the worktree-write-path-visible branch August 19, 2026 21:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant