Description
Problem
publicProofsOfIndexing (and proofOfIndexing) take a deployment hash and resolve it to the
active site. When several copies of the same deployment exist, there is no way to ask for
the PoI of a specific copy: the non-active one is unreachable through the status API, even
though its data is fully present and its PoI is perfectly well-defined.
This matters because verifying a PoI is inherently a comparison between two datasets, and
the API only exposes one of them at a time.
Why this matters
PoI divergence investigation. When a deployment's PoI disagrees with the rest of the
network, the way to narrow down where the divergence was introduced is to reindex the
deployment and compare the result against the suspect data, block by block, until they part
ways. That requires reading both — while keeping the suspect copy intact and untouched, since
it is the evidence. Today the only way to read the second copy is to make it the active one,
which moves production query traffic onto it and mutates the very state being investigated.
The two copies can never be compared as they stand.
Silent corruption detection. Two independently indexed copies of the same deployment must
agree on every PoI. Comparing them is a cheap and powerful check, and it is the only one that
surfaces storage-level corruption or nondeterminism that no other mechanism catches — a
deployment stays healthy throughout. Again, it requires reading both copies, which the API
does not allow.
Secondary, but worth noting: the same limitation means a copy produced by graphman copy
cannot be checked against its source before graphman copy activate routes traffic to it.
Current workarounds and why they fall short
1. Flip the active flag temporarily (graphman copy activate <hash> <shard>, query, flip
back). This mutates production routing state, has propagation delay, and cannot distinguish two
copies living in the same shard. For divergence investigation it is worse than inconvenient: it
alters the deployment under examination.
2. Compute the PoI outside graph-node, reading the deployment's tables directly. This means
reimplementing the PoI hasher and depending on graph-node's internal schema. The Fast hasher
is reproducible with effort, but the Legacy hasher (Blake3 SetHasher with derived sequence
numbers) is not reasonably reimplementable, so pre-0.0.6 deployments are out of reach entirely.
More importantly, such an implementation would diverge silently from graph-node whenever either
the hashing or the schema changes — a poor foundation for a tool whose entire purpose is to
establish that two datasets are identical.
Request
Provide a supported way to obtain the Proof of Indexing of a specific deployment copy, rather
than only of whichever copy is currently active.
The public PoI (indexer address 0x00…0) is sufficient for the use cases above. Beyond that,
the design is entirely up to the maintainers.
Are you aware of any blockers that must be resolved before implementing this feature? If so, which?
None known.
Related but distinct: #4730 and #2092 concern publicProofsOfIndexing returning null when the
block hash cannot be resolved from the chain store — same resolver, different problem.
Some information to help us out
Description
Problem
publicProofsOfIndexing(andproofOfIndexing) take a deployment hash and resolve it to theactive site. When several copies of the same deployment exist, there is no way to ask for
the PoI of a specific copy: the non-active one is unreachable through the status API, even
though its data is fully present and its PoI is perfectly well-defined.
This matters because verifying a PoI is inherently a comparison between two datasets, and
the API only exposes one of them at a time.
Why this matters
PoI divergence investigation. When a deployment's PoI disagrees with the rest of the
network, the way to narrow down where the divergence was introduced is to reindex the
deployment and compare the result against the suspect data, block by block, until they part
ways. That requires reading both — while keeping the suspect copy intact and untouched, since
it is the evidence. Today the only way to read the second copy is to make it the active one,
which moves production query traffic onto it and mutates the very state being investigated.
The two copies can never be compared as they stand.
Silent corruption detection. Two independently indexed copies of the same deployment must
agree on every PoI. Comparing them is a cheap and powerful check, and it is the only one that
surfaces storage-level corruption or nondeterminism that no other mechanism catches — a
deployment stays
healthythroughout. Again, it requires reading both copies, which the APIdoes not allow.
Secondary, but worth noting: the same limitation means a copy produced by
graphman copycannot be checked against its source before
graphman copy activateroutes traffic to it.Current workarounds and why they fall short
1. Flip the active flag temporarily (
graphman copy activate <hash> <shard>, query, flipback). This mutates production routing state, has propagation delay, and cannot distinguish two
copies living in the same shard. For divergence investigation it is worse than inconvenient: it
alters the deployment under examination.
2. Compute the PoI outside graph-node, reading the deployment's tables directly. This means
reimplementing the PoI hasher and depending on graph-node's internal schema. The
Fasthasheris reproducible with effort, but the
Legacyhasher (Blake3SetHasherwith derived sequencenumbers) is not reasonably reimplementable, so pre-0.0.6 deployments are out of reach entirely.
More importantly, such an implementation would diverge silently from graph-node whenever either
the hashing or the schema changes — a poor foundation for a tool whose entire purpose is to
establish that two datasets are identical.
Request
Provide a supported way to obtain the Proof of Indexing of a specific deployment copy, rather
than only of whichever copy is currently active.
The public PoI (indexer address
0x00…0) is sufficient for the use cases above. Beyond that,the design is entirely up to the maintainers.
Are you aware of any blockers that must be resolved before implementing this feature? If so, which?
None known.
Related but distinct: #4730 and #2092 concern
publicProofsOfIndexingreturning null when theblock hash cannot be resolved from the chain store — same resolver, different problem.
Some information to help us out