Skip to content

Preserve concurrent task changes when triggers report results - #72849

Draft
hussein-awala wants to merge 1 commit into
apache:mainfrom
hussein-awala:codex/guard-trigger-task-transitions
Draft

Preserve concurrent task changes when triggers report results#72849
hussein-awala wants to merge 1 commit into
apache:mainfrom
hussein-awala:codex/guard-trigger-task-transitions

Conversation

@hussein-awala

Copy link
Copy Markdown
Member

Late trigger results could overwrite a concurrent task transition because event and failure submission selected deferred tasks without locking them. A task cleared, failed, resumed, or attached to a replacement trigger after that read could be scheduled again, or receive stale terminal callbacks and XComs.

Lock matching task instances in a consistent order and refresh their cached values before applying an event or failure. Both submission paths share the selection logic. Locks cover only task instances and wait for competing transactions, allowing the database to recheck the deferred state and trigger ID before dispatch.

Validation through Breeze with Python 3.10:

  • All 18 new regression cases fail against the original source.
  • Trigger model suite: 62 passed on PostgreSQL 14 and 62 passed on MySQL 8.
  • Trigger model and triggerer suites on SQLite: 157 passed, with 17 row-locking cases skipped.
  • Concurrent tests cover task failure, clearing, resumption, and replacement triggers; late terminal results produce no callbacks or XComs.
  • Regular and manual prek checks passed, including Ruff and airflow-core mypy.

Selective checks request the full core CI matrix, which was not run locally.

Related to #71391, which retries transient deadlocks; this change prevents stale task updates.


Was generative AI tooling used to co-author this PR?
  • Yes — Codex (GPT-6)

Generated-by: Codex (GPT-6) following the guidelines

A late trigger event or failure could overwrite a task that another transaction had cleared, failed, resumed, or attached to a different trigger. Terminal events could also send callbacks and XComs for a transition that had already lost the race.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The correctness lock is silently disabled when scheduler row-level locking is configured off.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Prevents late trigger results from overwriting concurrent task transitions.

Changes:

  • Locks and refreshes deferred task instances before processing trigger results.
  • Adds regression coverage for locking, cache refresh, and concurrent transitions.
File summaries
File Description
airflow-core/src/airflow/models/trigger.py Centralizes locked deferred-task selection.
airflow-core/tests/unit/models/test_trigger.py Adds concurrency and locking tests.
Review details
  • Files reviewed: 2/2 changed files
  • Comments generated: 1
  • Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

.execution_options(populate_existing=True)
)
# Wait for concurrent transitions; skipping a locked row would lose this result.
return session.scalars(with_row_locks(query, session, of=TaskInstance))

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good point!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants