publish when the live outage state moves, not only at 03:10

[tiXor-code]

Why

The site is static and was rebuilt once a night, so a newly announced outage, or a revised restore estimate, could sit invisible for up to 24h. A visitor said exactly that on 2026-08-31:

"Nu este actualizat la data de 31.08.2026" — /punct-termic/pt-1-c5-1

The data was already in this repo. Only the publish lagged.

What

The scrape dispatches Nightly publish when the live state changes. Nightly keeps its own 03:10 schedule for the historical rebuild; this only adds intraday publishes, and nightly's existing concurrency: nightly group serialises any overlap.

The gate is the canonical hash over page A's parsed records (pipeline.parse.content_hash), not the raw bytes:

• a revised remediere_raw does trigger, which matters because the restore time is the number people are asking for
• reordered rows and markup churn do not — verified, whitespace and attribute churn yields an identical hash
• the affected-street list is not in the key tuple, so a change only to which streets a PT lists will not publish. Accepted and documented in the script

Page A lists only currently-active outages (records vanish on resolve), so a change to it is by definition a change to the live state.

A measurement that corrects the plan this was built from

I proposed this as the "tighter" option, expecting it to cut publishes to a handful a day. It does not. Across the last 12 snapshots of page A the live hash changed on 11 of 11 transitions. The filter is barely tighter than "any data change".

It still earns its place as protection against cosmetic churn, but not as a way to reduce build count. Expect roughly 8 publishes/day — bounded not by this filter but by how often GitHub actually runs the schedule, measured at ~8/day despite the */15 cron, because GitHub throttles frequent schedules on public repos.

Cost: both repos are public, so Actions minutes are free. Vercel Hobby runs one build at a time at ~6 min each, so ~8/day serialise comfortably.

Failure mode

The detector fails open: any read, git or parse error reports changed=true. A broken detector should degrade to publishing too often, never to going silently stale — which is the exact failure this change exists to fix.

It runs before the commit step, because it diffs the working tree against HEAD.

Checks

45 passed, 15 skipped (the documented expected state without a local db/termo.db). Script exercised against real history in this repo: correctly reported changed=false on an unchanged tree, and detected all 11 real transitions.

🤖 Generated with Claude Code

[claude]

Solid design (fail-open, hash on parsed records not raw bytes, running before the commit step) and the claims in the description check out against pipeline/parse.py — Record.key_tuple() does exclude streets and include remediere_raw, and nightly.yml does have workflow_dispatch + concurrency: group: nightly as described. One correctness bug blocks merge though.

EmptyState isn't distinguished from a real parse failure, so the detector fires on every scrape while the outage list is empty, not just on the empty↔non-empty transition.

• pipeline/parse.py:179 raises EmptyState when CMTEB renders the "Nu exista inregistrari" banner — a routine, recurring state, not an error. pipeline/backfill.py:65 already special-cases it separately from ParseFailure for exactly this reason.
• scripts/live_changed.py:78-84 wraps both live_hash(previous) and live_hash(current) in one broad except Exception, which also swallows EmptyState. So whenever page A is empty either before or after (a common, routine state, not just a rare edge case), the call raises before any hash is computed, and the script falls into the "parse failed – failing open" branch and unconditionally emits changed=true.
• Net effect: during any calm period with zero active outages, every 15-minute scrape will dispatch nightly.yml, not just the one transition into/out of that state. That directly contradicts the PR's own "~8 publishes/day" measurement and risks backing up the Vercel Hobby one-concurrent-build queue — the exact waste this design is meant to avoid.
• Fix: catch EmptyState explicitly (in live_hash or around each call) and treat it as zero records, e.g. content_hash([]), so two empty scrapes hash equal. Leave ParseFailure and anything else on the current fail-open path.

Secondary, non-blocking: subprocess.run(..., check=True) at scripts/live_changed.py:59 only catches CalledProcessError; an unexpected OSError (e.g. git unavailable) would propagate uncaught, failing the "Did the live outage state change?" step itself. Since that step has no continue-on-error, the job would stop there and the Commit if changed step would be skipped entirely — the opposite of "fails open." Wrapping main() in a top-level try/except would make the fail-open guarantee actually total.

[claude]

@claude please address this review:

• Bug: scripts/live_changed.py catches EmptyState (pipeline/parse.py:179, a routine "no active outages" state, already special-cased from ParseFailure in pipeline/backfill.py:65) inside the same broad except Exception used for real parse failures. This makes the detector emit changed=true on every scrape while the outage list is empty, not just on the transition — defeating the throttling this PR is built around. Fix by catching EmptyState explicitly and treating it as zero records (e.g. content_hash([])) so two empty scrapes compare equal.
• Minor: wrap main() in a top-level try/except so an uncaught OSError from subprocess.run (only CalledProcessError is currently caught) can't fail the "Did the live outage state change?" step itself, which would skip the commit step entirely rather than failing open.

[tiXor-code]

Both findings were real and are fixed in 4a8e0e0-onwards on this branch.

1. EmptyState conflated with parse failure — confirmed, and it defeated the throttle.

Reproduced before fixing, against a synthetic page carrying the exact banner parse.py:179 looks for:

	shipped	fixed
two consecutive EMPTY scrapes	changed=true (publishes every scrape)	changed=false
empty -> outage appears	—	changed=true
outage -> empty (all resolved)	—	changed=true

live_hash now catches EmptyState and returns content_hash([]), so zero records is a legitimate state rather than a failure. Worth adding: no empty-state snapshot exists in the last 400 commits of data/functionare.html, so this had not bitten yet — it would have arrived quietly in summer, publishing hardest exactly when there was nothing to publish.

2. Uncaught exception in the detect step — confirmed, and worse than a missed publish.

The step runs before Commit if changed, so a crash would fail the step, skip the commit and lose the scraped snapshot. Fixed belt and braces: main() now catches everything, and the step carries continue-on-error: true so even a hard crash cannot stop the commit.

That change makes the step's outputs possibly unset, so the publish condition moved from == 'true' to != 'false' — only an explicit "no change" suppresses a publish, which preserves the fail-open intent when the detector itself dies.

Added tests/test_live_changed.py (4 tests) pinning empty-vs-empty, empty-vs-real, real-vs-real and markup churn. Suite: 49 passed, 15 skipped.

[claude]

Verified the fix for the prior blocking bug: live_hash (scripts/live_changed.py:56-70) now explicitly catches EmptyState and maps it to content_hash([]), matching the existing precedent in pipeline/backfill.py:65. Confirmed against pipeline/parse.py: parse_page raises EmptyState only for the routine empty-records banner (line 179), distinct from ParseFailure, so a real parse failure still falls through to the fail-open path as intended. New tests in tests/test_live_changed.py cover the empty/empty, empty/real, and markup-churn cases.

The secondary non-blocking suggestion (make the fail-open guarantee total) is also addressed: main() is now wrapped in a top-level except BaseException (scripts/live_changed.py:104-110) and the workflow step has continue-on-error: true (.github/workflows/scrape.yml:32), so even an OSError from a missing git binary degrades to changed=true rather than skipping the commit step.

Workflow logic checked: actions: write permission is the minimal scope needed for gh workflow run, the publish step is correctly gated on both committed=='true' and changed!='false', and no secrets/injection issues (subprocess uses list args, no shell=True).