Exercise ownership through real partitions and owner VM loss - #35
Open
jeregrine wants to merge 1 commit into
Open
Exercise ownership through real partitions and owner VM loss#35jeregrine wants to merge 1 commit into
jeregrine wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
In-process storage faults and a connected peer do not establish what happens when a real owner VM disappears or loses its quorum. The suite lacked direct checks that acknowledged state survives those failures, that a minority cannot acknowledge a durable write, and that old ownership tokens remain fenced after healing.
Fix
Add a focused, opt-in three-node EKV-backed ownership lane:
Peer control runs over stdio independently of Erlang distribution. Per-node cookies prevent accidental reconnection during partitions; no host network rules are modified. Nodes and worktree-local data directories are uniquely named and cleaned up on exit. A dedicated permanent counter keeps the recovery contract explicit.
Supporting information
These are narrow, black-box failure scenarios, not a general history/linearizability checker or a deterministic simulator. An interrupted call is recorded as indeterminate rather than assumed uncommitted. This does not replace future generated operation histories or broader fault schedules.
The lane is tagged
distributedandintegration, keeping it out of the default suite. Its storage is entirely local EKV, but main's global test helper still contacts LocalStack; the independent fixture-isolation change in #33 removes that unrelated dependency. CI must explicitly select the distributed lane to exercise it.Only tests, test support, and documentation change; no production behavior is relaxed.