Skip to content

fix(voice): deadline-based user_away_timeout ignores transcript-less noise (#6030) - #6499

Closed
dorukdumlu wants to merge 1 commit into
livekit:mainfrom
dorukdumlu:fix/user-away-deadline
Closed

fix(voice): deadline-based user_away_timeout ignores transcript-less noise (#6030)#6499
dorukdumlu wants to merge 1 commit into
livekit:mainfrom
dorukdumlu:fix/user-away-deadline

Conversation

@dorukdumlu

@dorukdumlu dorukdumlu commented Jul 21, 2026

Copy link
Copy Markdown

Summary

Test plan

  • test_noise_user_state_flips_do_not_defer_away_deadline
  • test_final_transcript_and_agent_activity_refresh_away_deadline
  • test_away_recovery_transcript_does_not_arm_during_agent_speech
  • test_conversation_user_away_ignores_user_speaking
  • test_conversation_user_away_still_cancels_on_agent_speaking
  • Existing test_final_transcript_resets_away_timer_when_not_speaking

devin-ai-integration[bot]

This comment was marked as resolved.

self._user_away_timer: asyncio.TimerHandle | None = None
# absolute deadline for "away"; preserved across transcript-less user
# speaking↔listening flips so telephony noise cannot defer the timeout
self._user_away_deadline: float | None = None

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can add an option to only cancel the user away timer when agent speaking or user input transcribed. right now it's cancelled when agent or user speaking that may too sensitive to the noises.

dorukdumlu added a commit to dorukdumlu/agents that referenced this pull request Jul 22, 2026
Address review on livekit#6499: drop redundant away-recovery timer re-arm (can fire during agent speech), and add user_away_on=conversation so VAD speaking no longer pauses the away countdown. Default stays presence with deadline preservation.
@dorukdumlu

Copy link
Copy Markdown
Author

@longcw thanks — agreed the cancel policy was the missing piece (deadline alone only fixes flip-storms, not continuous noise stuck in speaking).

Pushed:

  1. user_away_on"presence" (default) keeps current cancel-on-user-speaking + deadline preservation; "conversation" cancels only while the agent is active and ignores user VAD speaking. Telephony/user_away_timeout getting triggered for SIP participant on telephony calls even when user is not speaking and taking background noise and conversations as user state speaking. #6030 should use user_away_on="conversation".
  2. Devin bug — removed the redundant away-recovery reset=True that could arm the timer mid-agent-speech.

Happy to rename the option or flip the default if you'd rather.

devin-ai-integration[bot]

This comment was marked as resolved.

@dorukdumlu

Copy link
Copy Markdown
Author

Follow-up for the Devin finding on stale remaining=0 after long genuine speech in presence mode:

conversation mode unchanged.

devin-ai-integration[bot]

This comment was marked as resolved.

@dorukdumlu

Copy link
Copy Markdown
Author

Follow-up for Devin's conversation-mode stall:

When the agent returns to listening while the user is stuck in VAD speaking (noise), _update_agent_state now re-arms the away timer in user_away_on=""conversation"" — presence mode unchanged (still requires user listening).

Added test_conversation_rearms_away_after_agent_while_user_speaking + presence regression guard.

@dorukdumlu

Copy link
Copy Markdown
Author

Rebased onto current main (resolved conflicts with transcription_timeout / session options).

user_away_on + deadline-based tracking still in place on top of #6478's final-transcript refresh. Away unit tests green locally.

@longcw ready for another look whenever you have a minute.

@dorukdumlu
dorukdumlu requested a review from longcw August 6, 2026 19:54
devin-ai-integration[bot]

This comment was marked as resolved.

@dorukdumlu

Copy link
Copy Markdown
Author

Addressed Devin's conversation-mode finding:

Final non-empty transcripts now refresh the away deadline mid-utterance when user_away_on=""conversation"" (agent idle). Prevents a long answer from flipping to away before EOS while VAD still says speaking.

Added test_conversation_final_transcript_refreshes_away_during_speech. Away suite: 10 passed.

…kit#6030)

Noise-triggered speaking no longer defers away. user_away_on=conversation
(longcw) cancels only while the agent is active, which is the telephony
path. Presence mode still pauses on user VAD; a final transcript refreshes
the window so a real utterance cannot expire mid-sentence.
@dorukdumlu

Copy link
Copy Markdown
Author

@longcw rebased onto current main and squashed the follow-ups.

Your cancel-policy request is user_away_on: presence (default) still pauses on user/agent speaking; conversation ignores user VAD speaking and only pauses while the agent is active. Telephony / #6030 should use user_away_on=conversation.

Deadline is preserved across transcript-less noise flips. Final transcripts refresh the window (including mid-utterance in conversation mode). Ready for another look.

@longcw

longcw commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

closing in favor of #6880

@longcw longcw closed this Aug 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants