fix(voice): deadline-based user_away_timeout ignores transcript-less noise (#6030) - #6499
fix(voice): deadline-based user_away_timeout ignores transcript-less noise (#6030)#6499dorukdumlu wants to merge 1 commit into
Conversation
| self._user_away_timer: asyncio.TimerHandle | None = None | ||
| # absolute deadline for "away"; preserved across transcript-less user | ||
| # speaking↔listening flips so telephony noise cannot defer the timeout | ||
| self._user_away_deadline: float | None = None |
There was a problem hiding this comment.
I think we can add an option to only cancel the user away timer when agent speaking or user input transcribed. right now it's cancelled when agent or user speaking that may too sensitive to the noises.
Address review on livekit#6499: drop redundant away-recovery timer re-arm (can fire during agent speech), and add user_away_on=conversation so VAD speaking no longer pauses the away countdown. Default stays presence with deadline preservation.
|
@longcw thanks — agreed the cancel policy was the missing piece (deadline alone only fixes flip-storms, not continuous noise stuck in Pushed:
Happy to rename the option or flip the default if you'd rather. |
|
Follow-up for the Devin finding on stale
|
|
Follow-up for Devin's conversation-mode stall: When the agent returns to Added |
|
Addressed Devin's conversation-mode finding: Final non-empty transcripts now refresh the away deadline mid-utterance when Added |
…kit#6030) Noise-triggered speaking no longer defers away. user_away_on=conversation (longcw) cancels only while the agent is active, which is the telephony path. Presence mode still pauses on user VAD; a final transcript refreshes the window so a real utterance cannot expire mid-sentence.
|
@longcw rebased onto current main and squashed the follow-ups. Your cancel-policy request is Deadline is preserved across transcript-less noise flips. Final transcripts refresh the window (including mid-utterance in conversation mode). Ready for another look. |
995d722 to
45b1ca7
Compare
|
closing in favor of #6880 |
Summary
_user_away_deadlineforuser_away_timeoutso transcript-less userspeaking ↔ listeningflips re-arm with remaining time (telephony noise can no longer defer "away" indefinitely) — user_away_timeout getting triggered for SIP participant on telephony calls even when user is not speaking and taking background noise and conversations as user state speaking. #6030.user_away_on:"presence"(default, backcompat) vs"conversation"(ignore user VAD speaking; only pause while the agent is active / refresh on final transcript). Preferconversationfor noisy SIP +turn_detection="stt"._set_user_away_timer(reset=True)that could arm the timer while the agent is speaking (Devin review).Test plan
test_noise_user_state_flips_do_not_defer_away_deadlinetest_final_transcript_and_agent_activity_refresh_away_deadlinetest_away_recovery_transcript_does_not_arm_during_agent_speechtest_conversation_user_away_ignores_user_speakingtest_conversation_user_away_still_cancels_on_agent_speakingtest_final_transcript_resets_away_timer_when_not_speaking