Skip to content

fix: ValueError "Token was created in a different context" - #661

Open
aticie wants to merge 1 commit into
taskiq-python:masterfrom
aticie:fix/opentelemetry-context-issues
Open

aticie wants to merge 1 commit into
taskiq-python:masterfrom
aticie:fix/opentelemetry-context-issues

Conversation

@aticie

@aticie aticie commented Aug 24, 2026

Copy link
Copy Markdown

Fixes an issue that happens with OpenTelemetryMiddleware when a message is requeued with Context.requeue().

Context.requeue() uses self.broker.kick to kick a message to the broker which skips the Middleware pre-send and post-send invocations that kicker does here:

taskiq/taskiq/kicker.py

Lines 160 to 162 in ced1909

for middleware in self.broker.middlewares:
if middleware.__class__.pre_send != TaskiqMiddleware.pre_send:
message = await maybe_awaitable(middleware.pre_send(message))

taskiq/taskiq/kicker.py

Lines 168 to 170 in ced1909

for middleware in reversed(self.broker.middlewares):
if middleware.__class__.post_send != TaskiqMiddleware.post_send:
await maybe_awaitable(middleware.post_send(message))

Messages that are requeued with the same context variables will raise an error on post_save's .detach() because they are now running in a different async context.

The message's lifecycle on OpenTelemetryMiddleware will be:

  • pre_send -> message.labels are injected with context here
  • post_send
  • pre_execute -> message.labels are extracted here and re-used
  • post_execute
  • requeue() happens -> skipping pre_send and post_send, therefore not renewing the context.
  • pre_execute -> context inferred from message.labels again
  • post_execute
  • post_save -> Detaching the stale context here. Raises ValueError: Token was created in a different context

This change gets rid of post_save on OpenTelemetryMiddleware to detach the context on post_execute instead. Since, we are not actually doing anything database save related on the current post_save method, it seems fair to move everything under post_execute as this is where the task execution actually ends.

@danfimov

danfimov commented Sep 15, 2026

Copy link
Copy Markdown
Member

@soapun Can you help me with review here? I remeber that you built opentelemetry integration. Maybe you have an opinion on this change. Thank you)

@codecov

codecov Bot commented Sep 15, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 85.71429% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.90%. Comparing base (ef92b9e) to head (12aa51e).

Files with missing lines Patch % Lines
taskiq/middlewares/opentelemetry_middleware.py 85.71% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #661      +/-   ##
==========================================
- Coverage   82.91%   82.90%   -0.01%     
==========================================
  Files          69       69              
  Lines        2756     2755       -1     
==========================================
- Hits         2285     2284       -1     
  Misses        471      471              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@s3rius s3rius left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This fix for the situation is actually correct.

The only 1 concern I have is that we will skip recording save events. In context of the system it might be okay.

@s3rius

s3rius commented Sep 17, 2026

Copy link
Copy Markdown
Member

@aticie, but can you please run prek locally and fix issues? I rebased your PR onto new changes.

@s3rius
s3rius force-pushed the fix/opentelemetry-context-issues branch from be0d3d1 to 12aa51e Compare September 17, 2026 16:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants