gh-154969: Record stop-the-world pause durations in pystats - #155069
Open
overlorde wants to merge 1 commit into
Open
gh-154969: Record stop-the-world pause durations in pystats#155069overlorde wants to merge 1 commit into
overlorde wants to merge 1 commit into
Conversation
The free-threaded build counts stop-the-world pauses but never times them, so a 1 us pause and a 5 ms pause are indistinguishable in the stats output. Record the elapsed time from the moment a pause is requested until threads are about to be resumed, and report both the total and the longest single pause. Also fix merge_ft_stats(), which assigned rather than accumulated when folding a thread's stats into the interpreter's. With more than one thread, every free-threading counter reported whichever thread happened to merge last instead of the sum. All the other merge helpers already use +=.
overlorde
requested review from
FFY00,
ZeroIntensity and
ericsnowcurrently
as code owners
August 2, 2026 06:17
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This covers both problems described in the issue.
The first is that stop-the-world pauses are counted but never timed.
world_stops: 97says nothing about whether those pauses cost microseconds or milliseconds, so there is no way to tell a healthy workload from one that is spending real time frozen.stop_the_world()now takes a monotonic timestamp when a pause is requested, andstart_the_world()reads the clock again just before threads are released, recording both the total and the longest single pause. The window deliberately covers the whole time the application was denied parallelism, not just the time spent bringing threads to a halt. The new field instruct _stoptheworld_stateand both clock reads sit under#ifdef Py_STATSinside the existingPy_GIL_DISABLEDblock, so every other configuration compiles unchanged.The second is a bug I ran into while measuring the first.
merge_ft_stats()assigned where every other merge helper in the file accumulates:Threads fold their stats into the interpreter's one after another, so with assignment each counter ended up holding whichever thread merged last instead of the sum. Stop-the-world requests come mostly from the main thread, and the main thread merges last, so
world_stopshappened to look right the whole time. That is probably why this went unnoticed. The mutex and QSBR counters did not fare as well.Eight threads appending 200k items each to one shared list, same binary, with only
Python/pystats.cdiffering between the runs:mutex_sleepsqsbr_pollsworld_stopsworld_stop_total_nsworld_stop_max_nsworld_stopsis unchanged, as expected from the above. The mutex count swings between runs because the contention itself does, but it is now in the right range rather than pinned to the main thread's 14.The duration figures show why a count on its own is not much use here. Roughly 13 ms of total pause across 97 stops, but a single worst stop of 4.4 ms, so one pause accounts for a third of all the time spent frozen. A bare count cannot express that and an average would bury it, which is the reason for tracking the maximum separately.
One question for review: the merge fix is a bugfix against counters that already shipped, while the duration counters are new. If those want to go to different branches I am happy to split them into two PRs.