Skip to content

gh-157217: Fix dir() race when merging from a mappingproxy - #157279

Open
KingLizard1020 wants to merge 7 commits into
python:mainfrom
KingLizard1020:gh-157217-fix
Open

gh-157217: Fix dir() race when merging from a mappingproxy#157279
KingLizard1020 wants to merge 7 commits into
python:mainfrom
KingLizard1020:gh-157217-fix

Conversation

@KingLizard1020

@KingLizard1020 KingLizard1020 commented Sep 10, 2026

Copy link
Copy Markdown

dir(), dict(vars(cls)), and {**vars(cls)} go through dict_merge() on a mappingproxy of the class dict. The generic merge path called PyMapping_Keys() without holding the wrapped dict's critical section. On the free-threaded build, a concurrent insert (for example the first access to __annotations__, which stores __annotations_cache__) raised RuntimeError: dictionary changed size during iteration.

Unwrap types.MappingProxyType when it wraps a dict so the locked dict-to-dict path is used. Py_BEGIN_CRITICAL_SECTION2 already treats a self-merge (a == source) as a single mutex.

Tests:

  • Lib/test/test_dict_mappingproxy.py for dict(), {**view}, and update from a mappingproxy (including a UserDict wrap, which still uses the generic path)
  • test_dir_racing_class_dict_insert in Lib/test/test_free_threading/test_type.py (free-threaded only). This failed with many RuntimeErrors without the C change and passed with it.

Fixes #157217

Comment thread Include/internal/pycore_descrobject.h Outdated
* The struct layout matches mappingproxyobject in Objects/descrobject.c.
*/
static inline PyObject *
_PyDictProxy_GetMapping(PyObject *op)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you need it in the headers?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't — the helper was only used from dict_merge(). I dropped _PyDictProxy_GetMapping from the header and unwrapped the mapping locally there (Py_IS_TYPE, since mappingproxy is not a base type).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

dir() can raise RuntimeError: dictionary changed size during iteration on the free-threaded build

2 participants