fix(count_paths): walk an explicit stack so deep graphs do not overflow - #815
Open
tachsin wants to merge 1 commit into
Open
fix(count_paths): walk an explicit stack so deep graphs do not overflow#815tachsin wants to merge 1 commit into
tachsin wants to merge 1 commit into
Conversation
`cached_count_paths` recursed once per node, so the depth of the recursion was the length of the longest path. A chain of 200 000 nodes overflows the stack, and that graph has exactly one path and no loop anywhere in it, although the documentation implies a loop-free graph is safe. Walk an explicit stack instead. The nodes are held in an `FxIndexMap` that the stack refers to by index, which keeps `T: Clone` off the bounds and makes the node's state part of the same lookup that reads its count. That state also settles what a loop does now. There is no stack left to overflow, so an undetected loop would instead grow the working set until memory ran out, which is a worse failure than the documented one. A node reached again while its own count is still unknown lies on a loop by definition, so it is recognised at no extra cost and reported as a panic naming the problem. The documentation changes accordingly, from a note that loops overflow the stack to a `# Panics` section. Counting itself is unchanged: each node is still expanded once and its count reused, which the added test pins down by counting the calls.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #814.
cached_count_pathsrecursed once per node, so the depth of the recursion was the length of the longest path. A chain of 200 000 nodes overflows the stack, although that graph has exactly one path and no loop in it anywhere.Walks an explicit stack instead. The nodes are held in an
FxIndexMapthat the stack refers to by index, which keepsT: Cloneoff the bounds and folds the node's state into the same lookup that reads its count.Worth reviewing
This is the decision raised in #814. Once the recursion is gone there is no stack left to overflow, so an undetected loop would grow the working set until memory ran out — a worse failure than the one documented today. A node reached again while its own count is still unknown lies on a loop by definition, so it is recognised at no extra cost and reported as a panic naming the problem.
The documentation changes accordingly: the note that loops overflow the stack becomes a
# Panicssection. If you would rather it stayed a crash, or returned aResult, say the word and I will change it.Checking
Counting is unchanged — each node is still expanded once and its count reused. Three tests cover it:
C(38, 19)and against an independent dynamic-programming count, asserting thatsuccessorsis called at most once per node rather than once per pathThe existing
gridtest and both Advent of Code tests that usecount_pathsare unaffected.