Skip to content

feat(p2p): gate slashing, proposing, health, and sendTx on p2p connectivity - #25185

Merged
spalladino merged 7 commits into
merge-train/spartan-v5from
spl/p2p-no-peers-gates
Aug 14, 2026
Merged

feat(p2p): gate slashing, proposing, health, and sendTx on p2p connectivity#25185
spalladino merged 7 commits into
merge-train/spartan-v5from
spl/p2p-no-peers-gates

Conversation

@spalladino

@spalladino spalladino commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Gates p2p-dependent operations on actual gossip connectivity, so a node running with a dead libp2p stack (zero peers) can no longer file false slashing accusations, burn its proposer slots, or silently blackhole txs. Follows an incident where a validator whose TCP listener failed to bind kept running peerless and voted DATA_WITHHOLDING against entire committees.

One commit per concern (squashed on merge):

  1. feat(p2p): expose p2p connectivity — new getP2PConnectivity(): { enabled, connectedPeers } on the p2p service, client, and node RPC API. The dummy (p2p-disabled) implementation reports enabled: false, so sandbox and single-node setups are vacuously healthy for every gate below.
  2. fix(slasher): do not file data-withholding offenses while peerless — the data-withholding watcher infers offenses from the absence of txs in the local pool, which is invalid evidence without gossip connectivity. It now permanently skips (never backfills) slots probed while p2p is enabled with zero peers — after reconnecting, mined txs may already be evicted from the pool, so a late probe would still false-positive.
  3. fix(sequencer): skip proposing when node has no connected peers — a peerless proposer broadcasts into the void (burns the slot) or, with enough own committee seats, publishes a checkpoint whose data nobody has. It now skips building below minPeersToPropose (default 1, SEQ_MIN_PEERS_TO_PROPOSE, 0 disables) while keeping L1 duties (votes, prune, invalidation) via the existing fallback.
  4. feat(node): report per-component health on GET /status/status now returns { ok, components: { p2p: { healthy, enabled, connectedPeers }, ... } }. The p2pHealthMinPeers floor (P2P_HEALTH_MIN_PEERS) defaults to 0 (report-only) so fresh-network bootstrap nodes aren't failed by default; operators opt in.
  5. fix(node): reject sendTx when node has no peers — instead of accepting a tx and gossiping it to nobody, the node rejects immediately so callers can retry against a healthy node.
  6. test: keep single-node mock-gossip e2e workingregisterPhantomGossipPeer test helper so the one single-node e2e test on the in-memory mock gossip bus presents one connected peer to the gates.
  7. fix(slasher): do not file data-withholding offenses when p2p is disabled — a node with no p2p stack at all sees strictly less than a peerless one, so it must not slash either. The runtime gate drops enabled (any node with zero peers skips), and the watcher isn't constructed when p2pEnabled is false. Unlike the other gates, this one infers guilt from absence, which is never valid without gossip; the propose//status/sendTx gates keep treating enabled: false as vacuously fine because they gate the ability to act, which a deliberately p2p-less node has by design.

Strongly suggest to review commit by commit.

Related PRs (independent, same incident): #25177 (fail node startup when the p2p service fails to start), #25183 (periodic zero-peers warning).

Part of A-1701.

// those checkpoints may already have been evicted from the pool as mined, so a late probe would
// still report them missing. An unknown must not turn into an offense.
const connectivity = await this.p2p.getP2PConnectivity();
if (connectivity.enabled && connectivity.connectedPeers === 0) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if connectivity is disabled? looks like you'd still try to slash?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch — fixed in 7301a30. The gate was inconsistent: enabled: false means the node runs no p2p stack at all, so it sees strictly less than a peerless one, yet the gate did not fire and it would happily slash. That has not bitten us only because every p2p-disabled setup today is single-node (the node proposed the txs itself, so they are always in its own pool) — an accident of topology, not a safety argument.

Two changes:

  • the runtime gate is now just connectedPeers === 0, ignoring enabled;
  • the watcher is not constructed at all when p2pEnabled is false — a watcher whose only evidence is gossip should not exist on a node with no gossip.

The other three gates (propose, /status, sendTx) keep treating enabled: false as vacuously fine on purpose: those gate the ability to act, which a deliberately p2p-less node has by design (it is its own network). This one gates an inference from absence, which is never valid without gossip.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And yeah, Opus answered the above

spalladino added a commit that referenced this pull request Aug 12, 2026
When the p2p client starts while there are blocks to sync, the libp2p
service start is deferred to
`startServiceIfSynched`, which runs inside the block stream event
handler. That handler catches and
only logs errors, so a TCP bind failure (e.g. `ERR_NO_VALID_ADDRESSES`
after a restart) was swallowed.
On top of that, the sync promise awaited by node creation was resolved
before the service was started,
so startup succeeded regardless and the node kept running with a dead
p2p stack and zero peers.

The service is now started before the sync promise settles, and a
failure logs at fatal level and
rejects the stored sync promise (rather than throwing, which the block
stream would eat) so node
creation fails and the process exits, letting the orchestrator restart
it.

Part of A-1701.


---

Related PRs from the same incident (all independent, all targeting
`merge-train/spartan-v5`): #25177 (fail startup when the p2p service
fails to start), #25185 (connectivity signal +
slasher/proposer/health/sendTx gates), #25183 (periodic zero-peers
warning).
spalladino added a commit that referenced this pull request Aug 12, 2026
)

A validator could run for hours with a dead libp2p stack - zero peers
and zero
gossip - without anything in its logs flagging the condition. The
failure was
only reconstructable after the fact from the absence of activity, rather
than
from any explicit signal.

The peer manager heartbeat now tracks consecutive zero-peer heartbeats.
After 3
of them (so startup and transient dips stay quiet) it logs a warning
that the
node has no connected peers and can neither gossip nor propagate txs,
with the
zero-peer heartbeat count, the heartbeat interval, time since a peer was
last
connected, and the cached peer count. The warning then repeats about
once a
minute, derived from peerCheckIntervalMS, instead of on every heartbeat.
When
peers come back, a single info log reports connectivity restored and the
new
peer count.

Connected peers are counted via libp2p getPeers(), matching what
getP2PConnectivity reports. A peer count gauge already existed
(PEER_MANAGER_PEER_COUNT, recorded in discover()), so no metric was
added -
this change is logs only.

Part of A-1701.


---

Related PRs from the same incident (all independent, all targeting
`merge-train/spartan-v5`): #25177 (fail startup when the p2p service
fails to start), #25185 (connectivity signal +
slasher/proposer/health/sendTx gates), #25183 (periodic zero-peers
warning).
@spalladino
spalladino force-pushed the spl/p2p-no-peers-gates branch from 7301a30 to b87865a Compare August 14, 2026 13:37
Comment on lines +524 to +530
const connectivity = await this.p2pClient.getP2PConnectivity();
if (connectivity.enabled && connectivity.connectedPeers === 0) {
this.metrics.receivedTx(timer.ms(), false);
this.log.warn(`Rejecting tx ${txHash}: node has no connected peers`, { txHash });
throw new Error('Cannot accept tx: node has no connected peers to propagate it');
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice!

log?.warn(`Health check failed for ${failed.join(', ')}`, { components });
}
return true;
return { healthy: failed.length === 0, details: { components } };

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, this is actually going to be super helpful!

Adds `getP2PConnectivity()`, returning `{ enabled, connectedPeers }`, so downstream subsystems can tell
whether the libp2p stack is alive and how many peers it is talking to. Until now a node running with a dead
p2p stack (zero peers) looked identical to a healthy one from the outside.

- `P2PService` declares the method synchronously, matching its neighbouring `getPeers` / `getGossipMeshPeerCount`.
- `LibP2PService` returns `enabled: true` and counts peers reported as connected by the peer manager, so
  dialing and cached peers are excluded.
- `DummyP2PService` (and the TXE dummy client) return `enabled: false` with zero peers, encoding "p2p is
  disabled" for sandbox and single-node setups; consumers can treat disabled p2p as vacuously healthy instead
  of as a stack with no peers.
- `P2PClient` forwards to the service, and `P2PApi` exposes it over JSON-RPC with a `P2PConnectivity` zod
  schema, so the node API surfaces it too.

No behavior is gated on this yet; it is the foundation for gating slashing, proposing, and sendTx on p2p
connectivity in follow-up changes.

Part of A-1701.
The data-withholding watcher infers an offense from the absence of a
checkpoint's txs in the local pool. That is only valid evidence if the node
could have received those txs in the first place: a node whose p2p stack is
down sees every checkpoint's txs as missing and accuses entire committees of
withholding data.

The watcher now checks p2p connectivity once per tick, and while p2p is
enabled with zero connected peers it skips the slots that would otherwise be
probed. The skip is permanent — those slots are marked as checked and never
backfilled — because after peers return, txs from those checkpoints may
already have been evicted from the pool as mined, so a late probe would still
report them missing. An unknown must not become an offense.

The gate is vacuous when p2p is disabled by configuration (sandbox and
single-node setups report `enabled: false`), so behavior there is unchanged.
The degraded-state warning is logged only on the transition into and out of
the state, since the watcher ticks several times per slot.

Part of A-1701.
A validator running with a dead p2p stack (zero connected peers) still took its
turn as proposer. Two failure modes follow: the checkpoint proposal is broadcast
into the void, so committee attestations can never be collected and the slot is
burned; and an HA node holding enough committee seats to reach quorum with its
own attestations publishes a checkpoint whose tx data was never gossiped to
anyone, which is indistinguishable from data withholding and gets it slashed.

The proposer now checks p2p connectivity before entering the build path and
skips building when peerless, while still performing its L1 duties: governance
and slashing votes, prune, and invalidation all run via the existing
vote-and-prune fallback. The skipped slot is marked as attempted so the gate is
not re-evaluated on every work-loop tick, and the skip is recorded on the
checkpoint precheck metric with a `no_peers` reason.

The threshold is the new `minPeersToPropose` sequencer option
(`SEQ_MIN_PEERS_TO_PROPOSE`), defaulting to 1; setting it to 0 disables the gate.
Setups that disable p2p by config (sandbox, single node, automine) report
`enabled: false` and are unaffected, so they keep proposing with no peers.

Part of A-1701.
…tatus

A validator ran for hours with a dead p2p stack and zero peers while `GET /status` kept answering a plain 200
with an empty body, so neither operators nor k8s probes had anything to key off. The endpoint now reports the
health of each registered RPC namespace.

`GET /status` returns a JSON body: `{ "ok": true, "components": { "<namespace>": { "healthy": true, ... } } }`,
where each component entry carries its healthy flag plus whatever details its health check reports. The 200/500
semantics are unchanged: 500 when any component is unhealthy, 200 otherwise. Servers with no component-level
checks answer with just `{ "ok": true }`. Health check functions may keep returning a plain boolean, which is
read as `{ healthy: <boolean> }`, so existing checks are unaffected. `SafeJsonRpcServer.isHealthy()` keeps its
boolean contract; the detailed shape is available through the new `getStatus()`.

The node registers a health check for its `p2p` namespace that reports `enabled` and `connectedPeers` from
`getP2PConnectivity()`. It is healthy when p2p is disabled by configuration, or when the connected peer count is
at least the new `p2pHealthMinPeers` p2p option (`P2P_HEALTH_MIN_PEERS`). That floor defaults to 0, so out of the
box the peer count is reported but never fails the check: the first node of a fresh network legitimately runs
with no peers, and failing readiness by default would prevent bootstrapping the network. Operators who want the
check to fail on a peerless node opt in by raising the floor.

Part of A-1701.
A node whose p2p stack is up but has zero connected peers used to accept txs
over RPC and "gossip" them to nobody: the tx sat in the local pool forever
while the caller believed it had been submitted. The failure was silent — no
error, no receipt, nothing actionable.

`#sendTx` now checks p2p connectivity before doing any work and rejects with a
clear error when p2p is enabled and no peers are connected, so callers fail
immediately and can retry against a healthy node. The check runs ahead of
`isValidTx` so a peerless node does not pay for full tx validation before
rejecting, and the rejection is counted in the received-tx metric like the
other failure paths.

Nodes running without a p2p stack (sandbox, single-node setups) report
`enabled: false` and are unaffected — they keep accepting txs, since their
local sequencer mines straight from the local pool.

Part of A-1701.
`single-node/misc/missed_l1_slot` is the only test that runs a single node on the
in-memory mock gossip bus (it needs the bus so the proposer's own broadcasts route
through the local proposal handler). With one member on the bus, `DummyPeerManager`
filters the node itself out and reports zero peers, so the node ends up looking
peerless to every connectivity gate: `sendTx` now rejects because there is nobody
to propagate to, and a proposer under `minPeersToPropose: 1` skips its slot.

Adds `registerPhantomGossipPeer` to the p2p test helpers: it registers a bare
`MockGossipSubService` on the network and returns its peer id. The phantom peer
subscribes to no topics and drops anything delivered to it, so it changes nothing
about message flow — it only makes the node count one connected peer. The test
registers one right after setup.

Test-infra only; the production gates are unchanged.

Part of A-1701.
The data-withholding gate only fired when p2p was enabled with zero peers, so a node
running with P2P_ENABLED=false would still slash — despite having strictly less
visibility than a peerless one, since its only evidence is the absence of gossiped txs
in the local pool.

Drop `enabled` from the runtime gate so any node with zero connected peers skips, and
do not construct the watcher at all when p2p is disabled.
@spalladino
spalladino force-pushed the spl/p2p-no-peers-gates branch from b87865a to ae3c5dc Compare August 14, 2026 19:16
@spalladino
spalladino merged commit a6ce449 into merge-train/spartan-v5 Aug 14, 2026
12 checks passed
@spalladino
spalladino deleted the spl/p2p-no-peers-gates branch August 14, 2026 19:49
aminsammara added a commit that referenced this pull request Aug 17, 2026
Promotes `v5-next` onto `v5` for the **v5.2.0** release.

Frozen at `ee5d2d367e` — the `v5-next` tip at cut time. Both
`merge-train/spartan-v5` and `merge-train/fairies-v5` are drained to
that same commit, so nothing staged is left behind.

### Testnet validation

This branch was cut at `a4db216abf`, which is byte-identical to the
`v5.2.0-nightly.20260815` tag — same commit, same tree
(`0b22572eace2419ede6bbab173514d6965d4e0d0`). That nightly's CI3 run is
green, its artifacts are published to npm and Docker Hub, and it has
been **running healthily on testnet since 2026-08-15**:

| Component | Image / reported version |
|---|---|
| validators, prover node, prover broker, prover agents (`testnet` ns) |
`aztecprotocol/aztec:5.2.0-nightly.20260815` |
| `v5.testnet.rpc.aztec-labs.com` | `nodeVersion=5.2.0-nightly.20260815`
|
| `canonical.testnet.rpc.aztec-labs.com` |
`nodeVersion=5.2.0-nightly.20260815` |

Chain advancing normally, no prunes or reorgs, no WARN/ERROR across node
pods since rollout.

The branch has since been fast-forwarded to `ee5d2d367e` to pick up
#25242 (configurable RPC server HTTP timeouts and CORS allowed-headers).
That is the only delta from the soaked tree — 8 files, +131/−10, no
nightly covers it yet. Its config defaults were checked against Node's
built-ins (`keepAliveTimeout` 5000 ms, `headersTimeout` 60000 ms) and
match exactly, and the CORS default path resolves to the same `cors()`
call as before, so a node that sets none of the new env vars behaves
identically. It touches no circuits, protocol contracts, or generated
constants.

### Manifest

`.release-please-manifest.json` reads `5.2.0` on this branch, which is
the released version — correct as-is, no change needed in this PR.
`v5-next` moves to `5.3.0` separately in #25240; this branch is frozen
and cannot pick that up, so the two can merge in either order.

`v5` is a strict ancestor of `v5-next` this cycle, so there was no
manifest conflict to pre-resolve.

### Scope

139 commits (86 non-merge, 23 PR-level) spanning 2026-07-14 to
2026-08-17.

| Area | Theme |
|---|---|
| `prover-node` / `prover-client` | Epoch-proving robustness:
retry-to-converge with failure declared only at submission-window
expiry, ticker-driven expiry, per-checkpoint post-mortem upload,
checkpoint-only re-proving, prune-induced fault handling (#24678,
#24982, #24983, #24990, #25027) |
| `pxe` | Sync performance: hash-pinned node read cache (#24969),
anchor-bounded tag log caching (#25074), note/event validation from
cached tx data (#25076), constrained tag sync (#24275), sender tagging
finalization from log blocks (#25045) |
| `p2p` / `validator-client` | Gossip tx validation no longer stalls
behind tx-pool finalization (#25148); startup fails when p2p fails to
start (#25177); slashing/proposing/health/`sendTx` gated on p2p
connectivity (#25185); duplicate time-sensitive proposal validation
removed (#25207); `ValidatedProposal` branding (#25222) |
| `ethereum` / `aztec-node` | L1 watchers poll `getLogs` instead of
`eth_newFilter` (#25176); block stream sync no longer resolves against
an earlier pass (#25206) |
| `stdlib` / `foundation` | Deserialization bounds hardening (#25026,
#25028, #25029, #25109); checkpoint block-shape and block-count
validation (#25229); JSON-RPC cookies (#25231) |
| `archiver` | Removed-block cleanup and ownership-checked tx-effect
deletes (#24765); L2→L1 witness from a single store snapshot (#24754) |
| `aztec` | Declarative deployment framework at `@aztec/aztec/deploy`
(#24685) |
| `slasher` | Own-validator slash-target warnings and metrics (#25058) |
| telemetry | JSON-RPC metrics (#25159) |
| JSON-RPC server | Configurable HTTP keep-alive / headers timeouts and
CORS allowed-headers, defaults preserving current behaviour (#25242) |
| toolchain | Noir bumped to `v1.0.0-beta.25` (#24907) |

**One breaking change**, inherited from the Noir bump: note types
declared directly inside a `contract` module must now be `pub`.
Everything else is additive or internal.

### Protocol constants

Built from source and compared against `v5` — a cache-free rebuild of
every protocol circuit with each ref's own nargo and bb, then
regeneration of `vk_tree.ts` and `protocol_contract_data.ts`:

- `vkTreeRoot` =
`0x2b3b6ea4412b9c8f6457a37f91a2870306f8641e07e16a49b68bda6f8bc02892` —
unchanged from v5.1.0
- `protocolContractsHash` =
`0x2c075866eafc88a1f6f9addc7e337c6e64e45d1cb7fd7c0d612ebcec72aab2ca` —
unchanged from v5.1.0

The Noir beta.24 → beta.25 bump does not reach the circuits: the release
build consumes the committed `pinned-build.tar.gz`, which is
bit-identical between v5.1.0 and this commit (blob `3bedcb1fd1…`), so
the protocol-circuit bytecode is frozen rather than recompiled. The 47
verification keys were recomputed locally from that pinned bytecode with
the cache disabled, and all 47 `check_pinned_vk` checks passed.
`protocolContractsHash` is likewise backed by the new
`pinned-protocol-contracts.tar.gz`, whose three artifacts were
byte-compared against the build. Both values were also confirmed inside
the published `@aztec/protocol-contracts` and
`@aztec/noir-protocol-circuits-types` packages for
`5.2.0-nightly.20260815`.

That makes v5.2.0 a drop-in upgrade against the current rollup rather
than a coordinated one.

✅ **Gate re-run against `ee5d2d367e` (the current head) and passed** —
both values reproduced exactly from a build at this commit, with
`vkTreeRoot` numerically evaluated rather than inferred.
`29556326ce..ee5d2d3` touches no `noir-projects/**`,
`l1-contracts/src/**`, `ConstantsGen.sol`, `constants.gen.ts` or
`constants.nr`. Scope of the check: it verifies that the pinned circuit
bytecode plus locally recomputed VKs agree with the pin — not that a
from-source recompile reproduces v5.1.0's bytecode.

<details>
<summary>Commits added after the original constants check at
2955632</summary>

`#25207` `#25222` `#25206` `#25185` `#25176` `#25163` `#25229` `#25159`
`#25162` `#25231` `#25224` `#25228` `#25230` `#25242` plus their merge
commits.
</details>

### Known gap

Migration notes carry entries under `## TBD` that arrived with the docs
baseline backport (#25017) and describe `next`-line changes not present
on the v5 line — protocol contracts removed from
`@aztec/noir-contracts.js`, and the `at(wallet)` → `withWallet(wallet)`
deprecation. Neither exists on this branch. There is also no `## 5.1.0`
heading, and the entries that ship in v5.2.0 sit under `## TBD` rather
than a version heading.

This ships in the release docs, so it is worth correcting on `v5-next`
and backporting to `v5` before the tag, rather than after.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants