FT: Data race on `Task.task_coro()` in the eager-task path

[deadlovelll]

Bug report

Bug description:

In FT build task_eager_start() clears task_coro outside CS, while Task.get_coro() reads it under one. This causes data race

Reproducer:

import asyncio
import threading


async def child():
    total = 0
    for i in range(2000):
        total += i


def watch(loop):
    while True:
        for task in asyncio.all_tasks(loop):
            task.get_coro()


async def main():
    loop = asyncio.get_running_loop()
    loop.set_task_factory(asyncio.eager_task_factory)
    for _ in range(3):
        threading.Thread(target=watch, args=(loop,), daemon=True).start()
    for _ in range(8000):
        loop.create_task(child())


asyncio.run(main())

Output:

==================
WARNING: ThreadSanitizer: data race (pid=72293)
  Read of size 8 at 0x00010d0a04a0 by thread T2:
    #0 _asyncio_Task_get_coro _asynciomodule.c.h:1472 (python.exe:arm64+0x1003b9dac)
    #1 _PyEval_EvalFrameDefault generated_cases.c.h:4330 (python.exe:arm64+0x100299b88)
    #2 _PyEval_Vector ceval.c:2172 (python.exe:arm64+0x10028f0ec)
    #3 _PyFunction_Vectorcall call.c (python.exe:arm64+0x1000889b0)
    #4 _PyObject_VectorcallPrepend call.c:855 (python.exe:arm64+0x10008a14c)
    #5 method_vectorcall classobject.c:55 (python.exe:arm64+0x10008d650)
    #6 context_run context.c:731 (python.exe:arm64+0x1002df018)
    #7 method_vectorcall_FASTCALL_KEYWORDS descrobject.c:422 (python.exe:arm64+0x1000a03a4)
    #8 PyObject_Vectorcall call.c:327 (python.exe:arm64+0x100088350)
    #9 _Py_VectorCallInstrumentation_StackRefSteal ceval.c:768 (python.exe:arm64+0x10028fc48)
    #10 _PyEval_EvalFrameDefault generated_cases.c.h:1906 (python.exe:arm64+0x100294ca8)
    #11 _PyEval_Vector ceval.c:2172 (python.exe:arm64+0x10028f0ec)
    #12 _PyFunction_Vectorcall call.c (python.exe:arm64+0x1000889b0)
    #13 _PyObject_VectorcallPrepend call.c:855 (python.exe:arm64+0x10008a14c)
    #14 method_vectorcall classobject.c:55 (python.exe:arm64+0x10008d650)
    #15 _PyObject_Call call.c:348 (python.exe:arm64+0x100088610)
    #16 PyObject_Call call.c:373 (python.exe:arm64+0x100088688)
    #17 thread_run _threadmodule.c:388 (python.exe:arm64+0x100457d44)
    #18 pythread_wrapper thread_pthread.h:234 (python.exe:arm64+0x10038ef50)

  Previous write of size 8 at 0x00010d0a04a0 by main thread:
    #0 task_eager_start _asynciomodule.c:3501 (python.exe:arm64+0x1003bac24)
    #1 _asyncio_Task___init__ _asynciomodule.c.h:956 (python.exe:arm64+0x1003b8994)
    #2 type_call typeobject.c:2415 (python.exe:arm64+0x10019508c)
    #3 _PyObject_Call call.c:361 (python.exe:arm64+0x1000885d0)
    #4 PyObject_Call call.c:373 (python.exe:arm64+0x100088688)
    #5 _PyEval_EvalFrameDefault generated_cases.c.h:3161 (python.exe:arm64+0x100297678)
    #6 gen_send_ex genobject.c:374 (python.exe:arm64+0x1000bea2c)
    #7 PyGen_am_send genobject.c:381 (python.exe:arm64+0x1000be624)
    #8 PyIter_Send abstract.c:2936 (python.exe:arm64+0x10005df38)
    #9 task_step_impl _asynciomodule.c:3092 (python.exe:arm64+0x1003af17c)
    #10 task_step _asynciomodule.c:3434 (python.exe:arm64+0x1003aebc8)
    #11 TaskStepMethWrapper_call _asynciomodule.c:2100 (python.exe:arm64+0x1003ae768)
    #12 _PyObject_MakeTpCall call.c:242 (python.exe:arm64+0x100087738)
    #13 context_run context.c:731 (python.exe:arm64+0x1002df0a4)
    #14 cfunction_vectorcall_FASTCALL_KEYWORDS methodobject.c:465 (python.exe:arm64+0x10012ffd0)
    #15 _PyObject_Call call.c:348 (python.exe:arm64+0x100088610)
    #16 PyObject_Call call.c:373 (python.exe:arm64+0x100088688)
    #17 _PyEval_EvalFrameDefault generated_cases.c.h:3161 (python.exe:arm64+0x100297678)
    #18 PyEval_EvalCode ceval.c:679 (python.exe:arm64+0x10028ec54)
    #19 run_eval_code_obj pythonrun.c:1406 (python.exe:arm64+0x100369f28)
    #20 run_mod pythonrun.c:1509 (python.exe:arm64+0x100369c58)
    #21 _PyRun_SimpleFile pythonrun.c:544 (python.exe:arm64+0x100364e0c)
    #22 _PyRun_AnyFile pythonrun.c:92 (python.exe:arm64+0x1003645b8)
    #23 Py_RunMain main.c:891 (python.exe:arm64+0x1003a74c8)
    #24 pymain_main main.c:921 (python.exe:arm64+0x1003a7918)
    #25 Py_BytesMain main.c:945 (python.exe:arm64+0x1003a79ec)
    #26 main python.c:15 (python.exe:arm64+0x1000020fc)

  Thread T2 (tid=34353080, running) created by main thread at:
    #0 pthread_create <null>:93330096 (libclang_rt.tsan_osx_dynamic.dylib:arm64e+0x309d8)
    #1 do_start_joinable_thread thread_pthread.h:281 (python.exe:arm64+0x10038e148)
    #2 PyThread_start_joinable_thread thread_pthread.h:323 (python.exe:arm64+0x10038df80)
    #3 ThreadHandle_start _threadmodule.c:475 (python.exe:arm64+0x100457b00)
    #4 do_start_new_thread _threadmodule.c:1919 (python.exe:arm64+0x10045750c)
    #5 thread_PyThread_start_joinable_thread _threadmodule.c:2042 (python.exe:arm64+0x100456544)
    #6 cfunction_call methodobject.c:564 (python.exe:arm64+0x100130c88)
    #7 _PyObject_MakeTpCall call.c:242 (python.exe:arm64+0x100087738)
    #8 PyObject_Vectorcall call.c:327 (python.exe:arm64+0x1000883f0)
    #9 _Py_VectorCall_StackRefSteal ceval.c:726 (python.exe:arm64+0x10028f4c4)
    #10 _PyEval_EvalFrameDefault generated_cases.c.h (python.exe:arm64+0x1002986a4)
    #11 gen_send_ex genobject.c:374 (python.exe:arm64+0x1000bea2c)
    #12 PyGen_am_send genobject.c:381 (python.exe:arm64+0x1000be624)
    #13 PyIter_Send abstract.c:2936 (python.exe:arm64+0x10005df38)
    #14 task_step_impl _asynciomodule.c:3092 (python.exe:arm64+0x1003af17c)
    #15 task_step _asynciomodule.c:3434 (python.exe:arm64+0x1003aebc8)
    #16 TaskStepMethWrapper_call _asynciomodule.c:2100 (python.exe:arm64+0x1003ae768)
    #17 _PyObject_MakeTpCall call.c:242 (python.exe:arm64+0x100087738)
    #18 context_run context.c:731 (python.exe:arm64+0x1002df0a4)
    #19 cfunction_vectorcall_FASTCALL_KEYWORDS methodobject.c:465 (python.exe:arm64+0x10012ffd0)
    #20 _PyObject_Call call.c:348 (python.exe:arm64+0x100088610)
    #21 PyObject_Call call.c:373 (python.exe:arm64+0x100088688)
    #22 _PyEval_EvalFrameDefault generated_cases.c.h:3161 (python.exe:arm64+0x100297678)
    #23 PyEval_EvalCode ceval.c:679 (python.exe:arm64+0x10028ec54)
    #24 run_eval_code_obj pythonrun.c:1406 (python.exe:arm64+0x100369f28)
    #25 run_mod pythonrun.c:1509 (python.exe:arm64+0x100369c58)
    #26 _PyRun_SimpleFile pythonrun.c:544 (python.exe:arm64+0x100364e0c)
    #27 _PyRun_AnyFile pythonrun.c:92 (python.exe:arm64+0x1003645b8)
    #28 Py_RunMain main.c:891 (python.exe:arm64+0x1003a74c8)
    #29 pymain_main main.c:921 (python.exe:arm64+0x1003a7918)
    #30 Py_BytesMain main.c:945 (python.exe:arm64+0x1003a79ec)
    #31 main python.c:15 (python.exe:arm64+0x1000020fc)

SUMMARY: ThreadSanitizer: data race _asynciomodule.c.h:1472 in _asyncio_Task_get_coro

Proposed fix:

Wrap task clear with CS block

    Py_BEGIN_CRITICAL_SECTION(task);
    if (task->task_state != STATE_PENDING) {
        // This seems to really help performance on pyperformance benchmarks
        clear_task_coro(task);
    }
    Py_END_CRITICAL_SECTION();

I have a fix ready

CPython versions tested on:

CPython main branch

Operating systems tested on:

macOS

Linked PRs

• gh-154734: Fix data race on Task.get_coro() with eager tasks in FT build #154735