PHASE A: Add device routing and better session management to C# Unity Platform Audio - #378
Open
MaxHeimbrock wants to merge 53 commits into
Open
PHASE A: Add device routing and better session management to C# Unity Platform Audio#378MaxHeimbrock wants to merge 53 commits into
MaxHeimbrock wants to merge 53 commits into
Conversation
On iOS the WebRTC ADM ran the shared AVAudioSession in automatic mode, so joining a call rerouted other app audio to the earpiece and hanging up deactivated the session out from under Unity (ambient audio died). Put RTCAudioSession into manual mode and have the app own the session: hold one permanent activation (so WebRTC's per-call setActive:NO never deactivates it), set PlayAndRecord + VideoChat (loudspeaker by default), and gate the VPIO unit via isAudioEnabled around connect/disconnect. Also restore the session on PlatformAudio.Dispose (previously dead code). Validated: Meet sample compiles for iOS (Unity 6000.3.10f1). Manual-mode audio behavior needs on-device validation. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Snapshot the app's audio session category/mode/options the first time LiveKit configures the session, and on the last PlatformAudio dispose (Interlocked instance counter) restore that snapshot and reactivate the session with setActive:YES so Unity audio output resumes. Previously the restore path hardcoded the Ambient category and left the session deactivated, which killed Unity audio at dispose time. Also removes the now-fixed README known issue and updates stale VoiceChat references in doc comments (the session uses VideoChat mode). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Backgrounding interrupts the shared AVAudioSession and WebRTC stops its VPIO unit. On foreground RTCAudioSession restarts the unit exactly once, with no retry, while Unity/FMOD restarts its own audio around the same moment and can reconfigure the shared session underneath it (observed on Unity 6; Unity 2022 happens to win the race). In manual audio mode nothing else ever restarts the unit, leaving calls with no audio output and no mic input. Observe UIApplicationDidBecomeActive and interruption-ended in the plugin and, after Unity's delayed restart has settled, re-assert LiveKit's category/mode/options, reactivate the session, and cycle isAudioEnabled to force a clean rebuild of the audio unit. Logs the session state before the re-assert so device tests can see who won the focus race. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Device-verified: with the foreground recovery in place, platform audio survives background/foreground on Unity 6, and the change is backwards compatible with Unity 2022.3. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Define the phase-A routing API on PlatformAudio, purely in C# and shaped identically to the planned FFI-backed implementation so app code written against it survives the plumbing swap: - AudioOutputKind enum (values mirror the planned FFI proto enum 1:1) - AudioDevice.Kind / AudioDevice.IsSelected - OutputPreference ranked policy (default BT > wired > speaker > earpiece) - IsSpeakerOutputPreferred as documented sugar over the list order - SelectOutput / ClearOutputOverride sticky override - DevicesChanged event (playout, recording) on the Unity main thread - internal IRouteController seam with desktop (FFI enumeration/GUID selection) and unsupported-mobile implementations; Android/iOS backends plug into the seam in follow-up work Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Implement the PAR-019 routing seam on Android as an SDK-owned backend, hardened from the device-validated sample hotfix (PR #364): - AndroidRouteController (API 31+) pins the communication device to the sticky SelectOutput override while its device is available, else the highest-ranked available kind per OutputPreference; kinds missing from the ranking are never auto-selected (pin released, OS default applies) - session-scoped MODE_IN_COMMUNICATION with save/restore of the prior mode; optional audio-focus request (internal, default off) - change detection via OnCommunicationDeviceChangedListener plus a 1.5 s poll thread for the trace-verified transitions that fire no OS event (device added while pinned; BT headset leaving the available list ~10 s after the route already fell back); no-op re-pin guard stops the feedback loop from our own setCommunicationDevice - GetDevices playout list and DevicesChanged now report real Android communication devices with Kind and IsSelected; recording stays the FFI default-input placeholder - StartRecording re-asserts the policy: since Android 13 the mode request is only honored while voice-communication capture is active - pre-API-31 stays a documented unsupported placeholder, matching the hotfix gate; platform notes on the public API updated accordingly Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… into max/par-021-unity-ios-session-features
The audio session config is now derived from a small state machine (idle / playout-only / recording) driven from C#, where PlatformAudio knows the recording state, plus a speaker-vs-earpiece preference expressed through the session mode (VideoChat/VoiceChat) — never overrideOutputAudioPort, so external devices always win. Non-recording states drop to mode Default + MixWithOthers to keep Unity audio unprocessed between calls; PlayAndRecord stays because the fork's ADM supports playout-only via an input-disabled VPIO unit but nothing guarantees VPIO under the Playback category. Every apply is mirrored into WebRTC's RTCAudioSessionConfiguration snapshot (reflected, no link-time dependency) so ADM-driven restarts re-apply the same config. Route changes are observed via AVAudioSessionRouteChangeNotification and forwarded to the new IosRouteController, which implements the PAR-019 seam: the playout list is the session's current output route with real Kind/IsSelected, DevicesChanged is raised on route changes (marshalled to the Unity main thread by PlatformAudio), OutputPreference reduces to the Speaker/Earpiece relative order, and SelectOutput throws the documented NotSupportedException pointing at AVRoutePickerView. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The ADM skips the VPIO rebuild on route changes that keep the hardware sample rate (HandleValidRouteChange -> HandleSampleRateChange no-ops), so a live VoiceChat -> VideoChat switch left the unit calibrated for the receiver — device-observed as an attenuated loudspeaker after an earpiece -> speaker toggle. Cycle isAudioEnabled after a mode change (when call audio is wanted) to force a clean rebuild against the new route, the same mechanism the foreground recovery already uses. The recovery and configure paths pass NO: the former cycles itself, the latter runs before the unit exists. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…11-samples-migration # Conflicts: # Runtime/Scripts/Audio/PlatformAudio.cs # Runtime/Scripts/Audio/RouteController.cs
…ration # Conflicts: # Samples~/Meet/Assets/Runtime/MeetManager.cs
…011) With PAR-019/020/021 the SDK owns output routing (OutputPreference policy, sticky SelectOutput, DevicesChanged, communication-mode ownership on Android), so the sample-level reimplementation is deleted: RouteRank, the AudioManager JNI plumbing (setMode/setCommunicationDevice, the communication-device change listener) and the route watchdog poll, plus their call sites in MeetManager and LiveKitAgentSession. The controller now only demonstrates the API: it relies on the default OutputPreference ranking (customizing it is shown as a one-liner) and logs DevicesChanged with device kind and selection state. The Android capture lifecycle policy stays: capture starts at call begin and keeps running while muted, since Android 13 only honors the communication-mode request — and with it the SDK's route pin — while the app has active voice-communication capture. Both PlatformAudioController copies (Meet + Agents) stay byte-identical. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds an "Audio Output Routing" section to the platform-audio docs: the default OutputPreference order, the IsSpeakerOutputPreferred convenience toggle, sticky SelectOutput/ClearOutputOverride, DevicesChanged, and per-platform behavior (Android 12+/older Android/iOS/desktop), including the MODIFY_AUDIO_SETTINGS requirement and the Android 13 active-capture constraint. The Meet sample README points at it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The Android routing backend took MODE_IN_COMMUNICATION and pinned the output route in its constructor and held both until Dispose, so an app that creates PlatformAudio at startup to keep one ADM alive sat in call mode from launch to quit. SetSessionAudioEnabled — the switch iOS already uses for this — was a documented no-op on Android. It is now plumbed through the routing backends: the Android backend takes the session on enable and hands it back on disable (pin cleared, replaced mode restored per transition, never an unconditional MODE_NORMAL), while enumeration, the change listener and the poll thread stay alive in both states so GetDevices and DevicesChanged keep working while idle. Every re-evaluation path — listener, poll, and the StartRecording re-assert — is observation-only while disabled, so none of them can resurrect a released session. The documented default (enabled at creation, uniform with iOS) is unchanged; the samples disable it right after creating PlatformAudio and MeetManager re-enables it for the duration of a call. The samples also recover Unity's own audio across output route changes: it does not follow a route change on its own, so the controller reopens the audio engine with AudioSettings.Reset, driven by Unity's OnAudioConfigurationChanged and by the SDK's DevicesChanged (logging both, since which one Android delivers is device behavior), coalesced so one route change resets once. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Device testing contradicted them: Unity audio kept playing over a Bluetooth headset both while idle and alongside an active call, so the docs must not assert that holding the call session forces a headset onto a call link instead of A2DP media. What the SDK actually does — request communication mode, pin the route, and release both when session audio is disabled — is stated instead; how a given device carries call and media audio concurrently is left to the device observation the card still owes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…eoff Device testing (Pixel 8a / Android 16, classic BR/EDR headset) established what actually breaks Unity's audio engine: a device being added or removed, not the route moving between the devices already connected. Joining a call leaves game audio playing, including when the platform moves media from the headset's A2DP link onto its call link. Resetting on every route change would therefore have restarted the game's audio at each join and hang-up for nothing, so the sample now triggers on the device set and resumes each source at the position it reached instead of from the start of the clip. The same run answered what the platform does with media during a call on a classic Bluetooth headset: dumpsys shows STREAM_MUSIC on bt_sco_hs while the call is active and back on bt_a2dp afterwards, with A2DP suspended for the duration — media is carried by the call link, not diverted or dropped. The README documents that as the observed behavior on that device, and as a further reason to hold the session only while a call is in progress. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Device testing (Pixel 8a / Android 16): disconnecting a Bluetooth headset does raise AudioSettings.OnAudioConfigurationChanged, but with deviceWasChanged=false, so the flag cannot separate a device change from any other reconfiguration and the recovery never ran. The sample now reacts to the callback either way; the callback AudioSettings.Reset raises itself always lands inside the coalescing window, so the recovery still cannot feed itself. This also makes Unity's callback the fast path rather than the SDK's event: a powered-off headset can stay in the platform's device list for seconds after the route moved, so the device-set signal trails the disconnect. The reset now reports how many sources came back and on which output, so a run can tell a failed reopen from a successful one that stayed silent. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
adb traces of a Bluetooth connect on a Pixel 8a (Android 16) show why the
recovery left the app silent: Unity stops every AudioSource when it reinitializes
its engine and raises OnAudioConfigurationChanged 25 ms afterwards, so the
handler's snapshot of "what is playing" was always empty and it restored nothing.
34.920 AudioTrack stop(11092): called with 92104 frames delivered
34.945 Unity audio configuration changed (deviceWasChanged=False, ...)
34.947 Reopening Unity's audio output (...), resuming 0 source(s)
The same trace shows the two events are ordered the other way round than assumed:
a headset's call profile appears ~650 ms before its media profile takes over
(32.912 SCO available, 33.576 setA2dpActiveDevice), so the SDK's DevicesChanged
arrives while the engine is still healthy and Unity's callback arrives after the
damage. They are now used accordingly — DevicesChanged remembers what is audible
and touches nothing, Unity's callback puts it back — instead of both racing to
reopen the engine, which is what made the outcome depend on timing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
adb traces of a mid-call Bluetooth connect on a Pixel 8a (Android 16) show
AudioSettings.Reset is not merely unnecessary but actively harmful. Reinitializing
the engine makes Unity claim the headset's call link through the deprecated
AudioManager.startBluetoothSco(), which evicts the setCommunicationDevice pin the
routing backend holds and leaves the platform unable to bring SCO up again:
03.281 setCommunicationRouteForClient … bt_sco_hs addr:…E0:03 (setCommunicationDevice, ours)
03.928 [AudioSettings.Reset]
04.060 setCommunicationRouteForClient … null (stopBluetoothSco, Unity)
04.083 setCommunicationRouteForClient … bt_sco addr: (startBluetoothSco, Unity)
04.783 AS.BtHelper: requestScoState: failed to connect in state 1
06.295 … and on every 1.5 s re-pin thereafter
Call audio and game audio both stayed on the loudspeaker for the rest of the
session as a result. Unity has already reopened its output by the time it raises
OnAudioConfigurationChanged, so the recovery only has to restart the app's own
sources — which it now does, idempotently, leaving anything Unity kept running
alone. The README's advice to reset is replaced with the reason not to.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Selecting a Bluetooth device is not synchronous: setCommunicationDevice starts an
SCO negotiation and the platform keeps reporting the previous communication device
until it completes. The 1.5 s poll took that for a dropped pin and re-issued the
request into its own pending activation, which the platform refuses — and the
refusal aborts the activation, so the route never arrived at all. Every failure in
the trace lands on the poll's cadence, with nothing else in the process asking for
SCO:
10:41:38.685 setCommunicationDevice() -> updateCommunicationRoute,
preferredCommunicationDevice: null
10:41:40.193 … 41.705 … 43.216 … (every 1.5 s for the whole call)
10:41:50.764 AS.BtHelper: requestScoState: failed to connect in state 1
AS.AudioDeviceBroker: failure to start BT SCO for uid: 10424
Call audio and the app's own media both stayed on the loudspeaker for the duration,
and both returned to the headset on hang-up, when the pin was cleared.
An outstanding pin now gets PinSettleTimeout to take effect before being issued
again. Recovering from a pin the platform drops silently — what the poll exists for
since PAR-020 — is unaffected: once the pin has been seen honored, any later
divergence re-pins immediately, and real route changes arrive through the change
listener rather than the poll.
This only became reachable with the session gating: while the route was pinned at
construction, it had long since been applied before any call, so the poll never
re-issued it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Withdrawing the outstanding legacy SCO request from here does not work. Measured on
device: stopBluetoothSco ran 158 ms before the pin and the pin was refused all the
same ("requestScoState: failed to connect in state 1"), because the request belongs
to a different client in the process and cannot be cancelled by this one.
So this is a situation the SDK can report but not repair. A pin the platform takes
without acting on is now retried with a doubling backoff up to 30 s instead of every
6 s, and the first failure logs a warning naming the likely cause and the
consequence, so it is diagnosable without LK_VERBOSE. The README documents it as a
known limitation with the workaround that does work — connect the headset after the
app has started.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A trace containing both outcomes inside one call settled it. Joining with the
headset connected since before launch left the pin unapplied; the headset then
dropped, the platform ran resetBluetoothSco, and on reconnect the same pin was
applied 708 ms later:
13:47:32.690 setCommunicationDevice(bt_sco_hs) -> preferredCommunicationDevice: null
13:47:40.227 updateCommunicationRoute … eventSource: resetBluetoothSco
13:47:49.828 setCommunicationDevice(bt_sco_hs) -> null
13:47:50.536 … preferredCommunicationDevice: bt_sco_hs
eventSource: BtHelper.onScoAudioStateChanged, state: 12
So the limitation is a platform SCO state that can be left pending, not Unity's
grab specifically — the grab provokes it but the previous wording made it the
whole cause, and the same failure occurs on Unity 6, which does not grab. The
note now says what the state does, what clears it (a headset reconnect, a
Bluetooth toggle, a reboot), and that routing is prompt once it is clear.
This also refutes the A2DP-streaming theory the card carried: the failing join
happened with the app's own audio stopped.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Probably something to note for android as a later TODO, since you're working with communication mode: Android MODE_IN_COMMUNICATION drops if no audio playback or recording is done for 6 seconds. The most common repro case would be if the client joins an empty room with no mic turned on, the communication will drop unexpectedly. The workaround Android uses is to just play zeroed audio during these silent moments. |
| queue:[NSOperationQueue mainQueue] | ||
| usingBlock:^(NSNotification* note) { | ||
| if (!s_liveKitConfigured) { | ||
| return; |
Contributor
There was a problem hiding this comment.
should we remove the observer in this case so the rest of the code will be run after s_liveKitConfigured is true ?
or they don't matter any more ?
|
|
||
| public PlatformAudioController(string trackName, AudioProcessingOptions audioOptions) | ||
| { | ||
| _trackName = trackName; |
Contributor
There was a problem hiding this comment.
why do we need to take a custom trackName from the app ?
| // playout for remote tracks to a PlatformAudio that already exists at connect time; | ||
| // initializing it after Connect leaves remote (agent) audio silent. | ||
| _audio = new PlatformAudioController(); | ||
| _audio = new PlatformAudioController(MicTrackName, AudioProcessingOptions.Default); |
Contributor
There was a problem hiding this comment.
Do you think we will allow changing the audio processing on the fly ?
And I wonder if this AudioProcessingOptions should go to startCapture() instead for the long term
| using LiveKit.Proto; | ||
| using UnityEngine; | ||
|
|
||
| // Drives the duplex platform audio (WebRTC ADM): captures the default microphone with |
Contributor
There was a problem hiding this comment.
is this PlatformAudioController.cs the same as the one at Agents/Runtime/Agent ?
If yes, any chance that we can share the code ?
davidliu
reviewed
Sep 2, 2026
| <uses-permission android:name="android.permission.INTERNET" /> | ||
| <uses-permission android:name="android.permission.RECORD_AUDIO" /> | ||
| <uses-permission android:name="android.permission.CAMERA" /> | ||
| <uses-permission android:name="android.permission.MODIFY_AUDIO_SETTINGS" /> |
There was a problem hiding this comment.
Is this something we can bake into the library rather than require consumers to add themselves? Not sure on how Unity packages their libraries.
Alternatively, adding a permission check to AndroidRouteController for MODIFY_AUDIO_SETTINGS might be a good idea to have regardless:
Android.Permission.HasUserAuthorizedPermission("android.permission.MODIFY_AUDIO_SETTINGS")
| public void SetPlayoutDevice(uint index) | ||
| { | ||
| ThrowIfDisposed(); | ||
| var (_, playout) = GetDevices(); |
There was a problem hiding this comment.
I think this should be GetDevicesViaFFI? GetDevices used to be grabbing from FFI previously, but now it's grabbing the platform devices, which mean if the GUIDs aren't exactly the same, this'll be unsafe.
Alternatively, keep GetDevices under the old retrieve from FFI behavior to ensure that consumers don't break, and introduce the new behavior under a new method.
MaxHeimbrock
changed the title
A local Disconnect() or Dispose() now raises ConnectionStateChanged, Disconnected and DisconnectedWithReason with DisconnectReason.ClientInitiated, synchronously and before Cleanup releases the handles, as the other LiveKit SDKs and the Rust core do. Previously Cleanup unsubscribed the room from FFI events before the core's own Disconnected event could arrive, so apps needed two teardown paths. The report happens once per room across the remote event, panic and local paths, so a handler calling Disconnect() re-entrantly is a no-op. OnConnect records ConnConnected, which the core records before this wrapper subscribes to room events, so IsConnected is true on a connected room; repeats of the current state arriving from the core are dropped. Un-ignores the two RoomTests that covered these gaps and adds two more. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
All three disconnect paths (local Disconnect/Dispose, the core's Disconnected
event, panic) now exit through one Teardown(reason, closeFfiRoom): it marks the
room disposed, unsubscribes from FFI events, raises the events with the handles
still live, and releases the handles in a finally, so a throwing handler cannot
leave the Rust-side room alive. The local path sends the FFI disconnect request
after the handlers, because the Rust-side close replaces remote-track handles on
its worker thread immediately.
The core's ConnectionStateChanged(Disconnected) is recorded from the
Disconnected{reason} event that always follows it, so handlers of either event
see the same DisconnectReason on every path and a handler that disconnects in
between cannot replace the server's reason with ClientInitiated. OnConnect
resets the disposed flag so a Room can be connected again, and skips Connected
when a ConnectionStateChanged handler already disconnected. ConnectInstruction
completes in a finally so a throwing handler cannot hang the awaiting code.
Comments now describe the mechanism as measured (the core's Connected copy
always drains one pass later; the ready timeout is 15s). Tests pin the handles
being live inside the handler, one report per event, the repeat-drop, a
throwing handler, and a synthetic two-event server disconnect with re-entrant
handlers. The pending-connect Disconnect() no-op stays and is documented.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
With Room.Disconnect() reporting ClientInitiated through Disconnected, the sample no longer needs its own teardown behind the hang-up button: OnEndCall just disconnects, and OnDisconnected owns the teardown for the button, a server-side disconnect and OnDestroy alike. That also fixes the sample leaving tracks and the "connected" UI behind on a server-side disconnect, which it only logged before. OnDestroy loses its dead second Disconnect() call. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…phase-a-validation Room.Disconnected now fires for a local Disconnect() as well, so the Meet sample's two teardown paths (OnEndCall + TeardownCall, OnDisconnected) collapse into OnDisconnected, which also releases the call audio session. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…c switch PlatformAudio.SetSessionAudioEnabled is removed. The platform's call audio session (iOS VPIO unit, Android MODE_IN_COMMUNICATION + route pin) is now taken while at least one Room is connected and released when the last one disconnects. Room keeps an internal connected-room count in SetConnectionState — a room counts while its state is anything but ConnDisconnected, so a reconnect still counts as a call — and raises an internal event before the public ConnectionStateChanged, so the session is settled by the time app handlers run, on a local Disconnect() and a server-side disconnect alike. An instance created mid-call takes the session at once; one created outside a call drops to idle. The samples lose every SetSessionAudioEnabled call site and the "disable right after construction" footgun; the README describes the automatic behavior. Two E2E tests cover the session following connect and disconnect and an instance created during a call. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…f tracking it in the client code
A plain Unity 2022 scene with no LiveKit code — one AudioSource playing on Android, Bluetooth headset connected mid-playback — stops the source just the same. Unity reinitializes its audio engine whenever an output device is added or removed, and that stops every AudioSource; the SDK, the ADM and the route pin play no part in it (PAR-023 F17, confirming F9's no-call reproducer). The remember/restore code in the Meet PlatformAudioController was therefore a generic Unity workaround, and the largest and most heuristic part of the controller: a remembered-sources dictionary, a forget-stopped mode, a first-switch gate for adopting loops, and active-and-enabled checks, none of which demonstrates a LiveKit API. It is removed; OnDevicesChanged keeps its device-list logging, which is the routing observability the sample is meant to show. The two facts a LiveKit user still needs stay in the README: Unity stops the sources and notifies afterwards, so restart them from app state, and never call AudioSettings.Reset during a call on Android (it evicts the SDK's route pin). The README now points at the Agents copy, which keeps the recovery for the time being. Compile-checked with csc (editor and UNITY_ANDROID passes, 0 errors). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…n redundant start/stop The flag stays as a private, platform-independent input to the iOS session-state machine, but it only mirrors what we last asked the ADM to do and can diverge from the ADM's real state (e.g. after an iOS interruption stops the capture). Exposing it invited callers to gate StartRecording/StopRecording on it, which could refuse a needed restart. The native ADM already ignores a start while recording and a stop while idle, so both sample controllers now just call through. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… SetRecordingDevice On Android, SetPlayoutDevice selects the call route, not only the output: the platform pairs the microphone with the communication device (verified against AOSP's default policy engine for the VOICE_COMMUNICATION source, which also re-routes a running capture in place). Say so on SetPlayoutDevice, PlayoutPreference and in the README routing section, and describe the same duplex rule for iOS. SetRecordingDevice on Android and iOS used to hand the request to the native ADM, which acknowledged and ignored it. It now logs a warning and returns before the FFI call, matching SetPlayoutDevice's no-op signalling on iOS. Drop the samples' startup SetRecordingDevice(0) (and the README snippet's). WebRTC already selects the OS default at factory init — the Default Communication Device role on Windows, index 0 on macOS/Linux — so the call was a no-op on macOS/Linux and on Windows replaced the role-based default with the first enumerated endpoint. It was the leftover half of the "pick the defaults" pair whose SetPlayoutDevice(0) half was removed earlier. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
MaxHeimbrock
commented
Sep 9, 2026
| m_SortingLayerID: 0 | ||
| m_SortingOrder: 0 | ||
| m_TargetDisplay: 0 | ||
| --- !u!1 &1060469597 |
Contributor
Author
There was a problem hiding this comment.
I should probably check what changed in the scene. I don't think this should be commited.
MaxHeimbrock
force-pushed
the
max/par-022-phase-a-validation
branch
from
September 9, 2026 14:29
e1a1db4 to
5b06ca9
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Background
We had several issues with platform audio on mobile (iOS & Android), especially with:
Changes
What did you do?
Roomconnections automatically — no client-side call-boundary APIRoom.Disconnectedreported for local disconnects too) by merge; it shows up in this diff until Report client-initiated disconnects through Room.Disconnected #382 lands onmainPublic API changes
Everything below is in namespace
LiveKit, inRuntime/Scripts/Audio/PlatformAudio.csunless noted. The newIRouteController/DesktopRouteController/UnsupportedRouteController/AndroidRouteController/IosRouteControllertypes are allinternal.Added
AudioDeviceKindpublic enum—Unknown=0, Earpiece=1, Speaker=2, WiredHeadset=3, Bluetooth=4, Usb=5, HearingAid=6(numbering mirrors the planned FFI enum of the same name, do not renumber)AudioDevice.Kindpublic AudioDeviceKind Kind— classified for playout devices on iOS and Android 12+;UnknownelsewhereAudioDevice.IsSelectedpublic bool IsSelected— whether this is the active output route; alwaysfalsewhere no backend reports itPlatformAudio.PlayoutPreferencepublic IReadOnlyList<AudioDeviceKind> { get; set; }— ranked automatic routing policy, defaultBluetooth > WiredHeadset > Speaker > Earpiece; setter throwsArgumentNullException, orArgumentExceptiononUnknown/ duplicates; aSetPlayoutDeviceoverride shadows it until clearedPlatformAudio.ClearPlayoutDeviceSelectionpublic void ClearPlayoutDeviceSelection()— clears the stickySetPlayoutDeviceoverride soPlayoutPreferenceapplies again; no-op where no override can exist (desktop, iOS, Android < API 31)PlatformAudio.DevicesChangedpublic event Action<IReadOnlyList<AudioDevice>, IReadOnlyList<AudioDevice>>—(playout, recording), raised on the Unity main threadChanged
No signature changed; these are behavior/contract changes on existing members.
ObjectDisposedExceptionafterDispose;Disposeis idempotent; subscribing to / unsubscribing fromDevicesChangedstays safe post-dispose.GetDevices()now goes through the platform routing backend instead of straight to the FFI. iOS returns the audio session's current output route withKind/IsSelected; Android 12+ returns the available communication devices. Previously a single placeholder entry per list on all mobile.SetPlayoutDevice(uint)/SetRecordingDevice(uint)resolve indices against that new list.SetPlayoutDevice(string)/(uint)now go through the routing backend as a sticky override ofPlayoutPreference(the earlierSelectOutputdraft was folded into this existing member instead of adding a second verb). Desktop: unchanged — FFI device selection, unknown GUID throwsInvalidOperationExceptionas before. Android 12+: was a no-op, now pins the device as the communication device — dropped when the device disappears, deferred while noRoomis connected — and an unknown GUID throwsInvalidOperationException. iOS and Android < API 31: still no effect, now logged as a warning. Both samples dropped theirSetPlayoutDevice(0)-at-startup call: on Android 12+ it would pin whichever device the OS lists first and shadow the ranking for the whole session.StartRecording()gained an iOS microphone-permission gate, so it now yields across frames on iOS and throwsInvalidOperationExceptionwhen the user denies. It also re-asserts the routing policy after capture starts, and switches the iOS session to its recording state.StopRecording()returns the iOS session to its idle, music-friendly state.StartRecording()/StopRecording()calls are left to the platform ADM, which ignores a start while already recording and a stop while idle. The SDK deliberately adds no guard of its own: the request-side flag it keeps to drive the iOS session-state machine can diverge from the ADM's real state (e.g. after the platform stops the capture on an iOS interruption), so gating on it could refuse a needed restart. A publicIsRecordingquery was tried in an earlier revision and dropped for the same reason; both samples'PlatformAudioControllerjust call through instead of keeping their own_isRecordingbookkeeping.Roomconnections (next bullet).Roomconnections (new behavior, no new API):PlatformAudioholds the platform's call audio session — on iOS WebRTC's VPIO unit, on Android 12+MODE_IN_COMMUNICATIONplus the output route pin — while at least oneRoomis connected and releases it when the last one disconnects. Taken and released inside the room's connection-state transition, beforeRoom.Connected/Room.Disconnectedreach app handlers, for a localDisconnect()and a server-side disconnect alike; a reconnect in progress still counts as a call. An instance created while a room is already connected takes the session immediately; one created outside a call sits idle (music-friendly iOS state, platform routing on Android). Plumbing isinternal(Room.ConnectedRoomCount/Room.ConnectedRoomCountChanged, maintained inRoom.SetConnectionState). There is deliberately no manual override: a room that carries no audio simply should not have aPlatformAudioalive.Room.Disconnect()/Dispose()(from Report client-initiated disconnects through Room.Disconnected #382, merged in): a local disconnect is now reported like a server-side one —ConnectionStateChanged,Disconnected,DisconnectedWithReasonwithDisconnectReason.ClientInitiated— synchronously with the room's handles still live, so one teardown handler covers both;IsConnectedis true fromConnectedon. Details and tests in Report client-initiated disconnects through Room.Disconnected #382.Dispose(): refcounted restore of the pre-LiveKit iOS audio session once the last instance goes away.StartRecording's old claim that recording auto-starts on construction was never true.Removed
Nothing relative to
main/ 2.0.0. No public type, member, or overload was removed, and no signature, parameter, or return type changed. (PlatformAudio.SetSessionAudioEnabledandPlatformAudio.IsRecording, both added in earlier revisions of this PR, were dropped again before merge — the former in favor of the automatic behavior above, the latter because it could misreport the ADM's real state; neither ever shipped.)AudioDeviceis a struct that grew two fields — source-compatible; only relevant when linking a precompiled assembly, which this source-distributed UPM package does not do.New integration requirement
Android routing needs
<uses-permission android:name="android.permission.MODIFY_AUDIO_SETTINGS" />in the app'sAndroidManifest.xml(both sample manifests updated).Proposed semver
Minor — 2.1.0 (from
package.json2.0.0). The surface is purely additive, so existing code compiles and runs unchanged. The behavior changes are real but confined toPlatformAudio— worth prominent release notes (theMODIFY_AUDIO_SETTINGSrequirement, the call audio session now followingRoomconnections — nothing to gate in app code, and aPlatformAudiokept alive next to an audio-less room now takes the call session for it — the iOSStartRecordingyield,SetPlayoutDevicenow routing on Android 12+ — drop select-device-0-at-startup calls — and, via #382,Room.Disconnectedfiring for local disconnects, so teardown handlers must tolerate running on hang-up) rather than a major bump.