Skip to content

Fetch the user's profile from the whole swarm at once - #161

Open
mpretty-cyro wants to merge 3 commits into
clientfrom
feat/fan-out-user-profile-fetch
Open

mpretty-cyro wants to merge 3 commits into
clientfrom
feat/fan-out-user-profile-fetch

Conversation

@mpretty-cyro

Copy link
Copy Markdown
Collaborator

Why

An ordinary poll asks one swarm member, once every poll interval, and the storage server does not promise a config has reached every member of a swarm. Measured against mainnet through a client bridge: an account created minutes earlier, restored on a second device, took 66 seconds for its display name to come back — several members sampled one at a time before one of them had it.

Both mobile clients already work around exactly this and say so in their own comments. Session Desktop does too. So the ordinary poll was never any client's design for this moment; it was just what a bridge without the workaround falls back to.

The case it matters for is a restore: nothing local can answer "what is this account called", and there is a person watching a progress indicator with a fifteen-second budget.

What

Core::fetch_user_profile(done) asks every swarm member concurrently for the UserProfile namespace, and is deliberately asymmetric about what it concludes:

  • the first answer carrying a config ends the fetch;
  • an empty answer never does, however many members give it — having the config is not a majority property, and a member that has not caught up is the condition being routed around rather than evidence against the one that has;
  • only every member having answered concludes there is nothing to find.

No agreement is required of the answer that wins. A config is merged rather than assigned, so a stale one taken from a member behind its swarm is corrected by the next ordinary poll; requiring a second member to say the same thing buys protection against something that already repairs itself, and costs a round trip in front of somebody waiting.

Every member is asked the same question — no last_hash — which is both what makes the answers comparable and what stops two members agreeing on a subset while a third, further along, is still answering.

Measured

Same account, same method, against mainnet:

time for the display name
ordinary poll 66s
fan-out, requiring a second member to agree 17s
fan-out, first member that has it 5s

Also confirmed end to end on an iOS simulator through a client bridge: the restore lands on the conversation list rather than falling through to "pick a new display name".

Refactor that came with it

Two pieces of the poll path are now shared rather than duplicated:

  • decode_retrieved takes no node and touches no database, deliberately. The retrieve cursor is kept per (namespace, node), and a decoder that knew about nodes is the shape in which one member's cursor gets written from another's answer — silently, because each cursor is individually plausible, and showing up much later as a namespace that re-fetches for ever or one that skips messages.
  • Core::_record_swarm_cursor takes the node as a parameter for the same reason.

Draft because

The consuming client change is not up yet, and this wants a review from someone who knows the swarm layer better than the caller does — in particular whether the fan-out should be bounded to a subset of the swarm rather than all of it on a metered connection.

An ordinary poll asks one swarm member, once every poll interval, and the storage
server does not promise a config has reached every member of a swarm. Measured against
mainnet through a client bridge: an account created minutes earlier, restored on a
second device, took 66 seconds for its display name to come back -- several members
sampled one at a time before one of them had it. Both mobile clients work around
exactly this and say so in their own comments.

`fetch_user_profile` asks every member concurrently. It is deliberately asymmetric
about what it concludes from what they say: having the config is not a majority
property, so one member holding it is the whole answer and the others not having it yet
is the condition being routed around rather than evidence against it. An answer
carrying the config seconded by a second member ends the fetch early; an empty answer
never does, however many members give it; and when everyone has answered, any member
that had it still wins. Concluding there is no config is the expensive direction and
waits for the whole swarm.

Every member is asked the same question -- no `last_hash` -- which is both what makes
the answers comparable and what stops two members agreeing on a subset while a third,
further along, is still answering.

Two pieces of the poll path are now shared rather than duplicated. `decode_retrieved`
takes no node and touches no database, deliberately: the retrieve cursor is kept per
(namespace, node), and a decoder that knew about nodes is the shape in which one
member's cursor gets written from another's answer -- silently, because each cursor is
individually plausible. `_record_swarm_cursor` takes the node as a parameter for the
same reason.

Measured after: 17 seconds, against 66 before.
Waiting for a second member to return the same answer bought protection against
something that repairs itself. A config is merged rather than assigned, so a stale one
taken from a member behind its swarm is corrected by the next ordinary poll; the
seconding only delayed the answer, and it delayed it in front of somebody watching a
progress indicator.

Measured on mainnet, same account, same method: 66s through the ordinary poll, 17s with
seconding, 5s without it. The cost of the confirmation was most of the remaining wait.

The asymmetry that matters stays: an empty answer still never settles the fetch, however
many members give it, and only every member having answered concludes there is nothing
to find.
@mpretty-cyro
mpretty-cyro marked this pull request as ready for review September 15, 2026 06:30
@mpretty-cyro mpretty-cyro self-assigned this Sep 15, 2026
`NamespaceIndentation: Inner` indents an anonymous namespace nested inside
`session`, and the decoder moved into that namespace without being re-indented.
Whitespace and comment rewrapping only: `./utils/format.sh`'s own output, which
is what the lint stage runs.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant