Skip to content

[Android] XR Composition Layer: MV-HEVC reports TwoTextures but only one view reaches the compositor on a second Android OpenXR runtime #2496

Description

@adamlisagor

Unity version

6000.3.11f1

Unity editor platform

macOS

AVPro Video edition

Core

AVPro Video version

3.3.6-core, build 2602121701

Device hardware

Unreleased Android OpenXR headset (NDA). Samsung Galaxy XR is the working control only.

Which Android OS version are you using?

14

Unity Graphics API

Vulkan

Video API

media3/ExoPlayer

Texture format

Y Cb Cr 420_OES

Audio output

System Direct

Any other Media Player component configuration required to reproduce the issue.

XR Composition Layer + YCbCr420_OES + ExoPlayer/media3. Auto Open/Play off; open after XrSession. Equirect2 180° via MediaPlayerLayerExtension. MV-HEVC HLS 4096x4096 per eye. Hardware decode. Fallback hint LeftRight (runtime reports TwoTextures). No second player, no ImageArrayIndex 1.

Which output component(s) are you using?

No response

Any other component configuration required to reproduce the issue.

None of the listed Core outputs (Apply to Mesh/Material, uGUI, IMGUI, Playlist, Resolve to RenderTexture). Reproduce with com.renderheads.avpro.video.xr.compositionlayers: MediaPlayerLayerExtension on a Unity Composition Layer whose data is MediaPlayerEquirectMeshLayerData (equirect2, 180° horizontal, ±90° vertical, radius 0). MediaPlayer: ExoPlayer/media3, Video Output Mode = XR Composition Layer, Texture Format = YCbCr420_OES, Audio = System Direct, Auto Open/Play off. Open only after a live XrSession. Content: stereo VR180 MV-HEVC HLS, 4096x4096 per eye. Hardware decode. A linked AVPro Video Stereo layer is present for packed-SBS control only; it is not required for the MV-HEVC case. Same MediaPlayer setup is stereoscopic on Samsung Galaxy XR.

The issue

The failure is on an unreleased Android OpenXR headset we cannot name (NDA). Please treat that as the device under test. Samsung Galaxy XR is a working control only. It is not the failing hardware.

On the NDA headset, MV-HEVC stereo VR180 plays through AVPro XR Composition Layer (Equirect Mesh) as upright video with audio, but only one eye view. Both eyes show the same picture. AVPro reports hasVideo=true, 4096x4096, stereoPacking=TwoTextures / MultiviewLeftPrimary. Time advances. This is not a black layer or a decode failure.

Same MediaPlayer, same master, same Equirect path is true stereoscopic on Samsung Galaxy XR. We submit EyeVisibility=Both and ImageArrayIndex=0. Galaxy’s compositor expands that Android Surface into two eyes. The NDA runtime does not. Packed SBS of the same title (8192x4096 LeftRight) is true 3D on the NDA headset on this same XR Equirect path, so the layer, crop, and submission are fine. The missing piece is the dependent MV-HEVC view.

What we measured on the NDA headset:

The XR Android Surface swapchain is array size 1. ImageArrayIndex=1 produces xrEndFrame XR_ERROR_VALIDATION_FAILURE. We stopped doing that.
A second MediaPlayer opened in Texture mode on the same URL (XR player left running), BGRA and YCbCr420_OES: packing=TwoTextures, GetTextureCount()=1, GetTexture(0)=4096x4096, GetTexture(1)=null. GetTextureCount is plane count, not view count.
ForceDispose of the live XR Composition Layer MediaPlayer to rebuild it as Texture/BGRA produces xrEndFrame XR_ERROR_RUNTIME_FAILURE. Unity shuts down the XR session and the app quits. We stopped doing that.
We are not asking you to debug a compositor we cannot name. When MediaCodec reports TwoTextures on Android, how does AVPro expose the dependent view if the runtime does not expand a size-1 XR_KHR_android_surface_swapchain?

A supported answer is any of: a second Unity texture (t1 populated), a second Android Surface or legal swapchain array slice, or an explicit packed-SBS decode of an MV-HEVC source. If the answer is that the runtime must expand the surface and AVPro will not emit a second view on Android, please say that. We can take that to the runtime vendor.

Not the earlier Galaxy black-layer / no-swapchain issue. Decode and the layer are up. Not a packed-SBS bug. Not a request to name or support the unreleased headset in your docs. If you need the vendor name under NDA, we can arrange that separately.

Media information

HLS: https://dev-cdn-02.spatialgen.com/theater/URBAN_THEORY_BT2020_MVHEVC_V1/index.m3u8
Stereo VR180, MV-HEVC multiview (base + dependent view, not packed SBS). Per-eye 4096×4096. Rec.2020 PQ HDR10, 10-bit, 45 fps. No DRM. SpatialGen encode.

AVPro after open: hasVideo=true, 4096x4096, stereoPacking=TwoTextures / MultiviewLeftPrimary. Audio plays. Time advances.

Working control (same title, same Equirect path)
Packed SBS HEVC, 8192×4096 LeftRight. True stereoscopic on the NDA headset. Proves the layer and crop are fine; only the second MV-HEVC view is missing.

Decoder on device reports MV-HEVC (video/x-mvhevc / Qualcomm c2.qti.mvhevc.decoder on Galaxy; same clip on the NDA headset). We can share a short sample offline if you cannot reach the HLS.

Logcat output

# Healthy decode, one view. Current path (XR Composition Layer + OES).
[VoidLayer][decode 2/10] hasVideo=true 4096x4096 @ 45.0fps | playing=true t=1.50s | hasAudio=true | stereoPacking=LeftRight texCount=0
[VoidLayer][decode 3/10] hasVideo=true 4096x4096 @ 45.0fps | playing=true t=2.25s | hasAudio=true | stereoPacking=TwoTextures texCount=0

# Second MediaPlayer in Texture mode, same URL, XR player left running.
# BGRA and YCbCr420_OES both looked like this.
packing=TwoTextures texCount=1 t0=4096x4096 t1=null

# What happens if we submit ImageArrayIndex=1 (swapchain array size 1).
# ~1900 hits per play. We stopped doing this.
xrEndFrame: XR_ERROR_VALIDATION_FAILURE

# What happens if we ForceDispose the live XR MediaPlayer and rebuild as Texture/BGRA.
# We stopped doing this.
xrEndFrame: XR_ERROR_RUNTIME_FAILURE
Unrecoverable runtime error has occurred, shutting down XR experience

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions