listeners: preserve distinct sockets across graceful upgrades - #990
Open
torinnd wants to merge 1 commit into
Open
listeners: preserve distinct sockets across graceful upgrades#990torinnd wants to merge 1 commit into
torinnd wants to merge 1 commit into
Conversation
Add ListenerConfig::fd_transfer_id so listeners sharing a configured address, including repeated port-zero and SO_REUSEPORT listeners, use distinct fd-table keys and retain their sockets across same-version graceful upgrades. Fixed TCP, Unix, and unidentified listeners retain their existing keys. Validate IDs during listener construction and return BindError for empty, whitespace-containing, or duplicate IDs. Track keys registered by the current process so an unidentified duplicate address is a bind error rather than a second owner of the same raw fd; no upstream test or example relies on that sharing. Mixed-version upgrades may rebind identified listeners because old binaries do not understand the new keys. Tests cover build-time validation, distinct routing, duplicate IDs, legacy keys, and a real SCM_RIGHTS round trip.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The listener fd table used by graceful upgrade is keyed by the configured address string. When two listeners in one process share a configured address, the second one finds the first one's fd in the table and wraps it with
from_raw_fd. Both listeners then own the same descriptor: accepts are split between them, and shutdown closes the fd twice.This is easy to hit with
127.0.0.1:0. Two services that each want their own ephemeral port end up on one socket. It also affectsSO_REUSEPORTlisteners on the same fixed address.This PR adds
ListenerConfig::fd_transfer_id. When set, the transfer key becomes<addr>#id=<id>instead of<addr>, so listeners with the same configured address get separate table entries and keep their own sockets across a same-version upgrade. Listeners without an ID keep their existing keys, so the wire format and old configurations are unchanged.Fdsnow tracks which keys the current process registered. A second listener that hits a key this process already owns gets aBindErrorinstead of a secondfrom_raw_fdon the same descriptor. Empty or whitespace IDs are also aBindError, since the transfer metadata is whitespace-delimited.One behavior change: two listeners in one process with the same fixed address and no ID used to share an fd silently. They now fail to bind. I didn't find anything in the repo that relies on the sharing, but if there are downstream users who do,
fd_transfer_idgives them separate keys. Mixed-version upgrades will rebind listeners that have an ID, since the old binary does not know the new key.Tests cover distinct binding and routing for two
127.0.0.1:0listeners, a realSCM_RIGHTSround trip that preserves both assigned ports, duplicate and invalid IDs, and unchanged keys for fixed TCP, UDS, and unidentified port-0 listeners.Related: #988