Skip to content

Misleading -ENOMEM errors caused by nvidia-drm on multi-GPU system #1302

Description

@anishgoyal1108

NVIDIA Open GPU Kernel Modules Version

610.57.04

Please confirm this issue does not happen with the proprietary driver (of the same version). This issue tracker is only for bugs specific to the open kernel driver.

  • [x ] I confirm that this does not happen with the proprietary driver package.

Operating System and Version

Arch Linux

Kernel Release

7.1.0-2-cachyos

Please confirm you are running a stable release kernel (e.g. not a -rc). We do not accept bug reports for unreleased kernels.

  • I am running on a stable kernel release.

Hardware: GPU

NVIDIA GeForce RTX 5090

Describe the bug

My desktop is primarily composited by a secondary amdgpu driver, but I offload much my graphics rendering to my 5090. The vivmem-backend buffer is forced through nvidia-drm PRIME, but nv_drm_gem_prime_get_sg_table() returns ERR_PTR(-ENOMEM), which causes my AMD card to think it's out of memory and crashes kwin_wayland.

To Reproduce

  • Dual-GPU system: Wayland desktop composited by a non-NVIDIA GPU, NVIDIA GPU present as a render device
  • An application on the NVIDIA GPU whose shared surfaces are allocated in video memory rather than through the PRIME-aware sysmem path.

An example: The Windows Steam client's steamwebhelper crashes on every launch. It runs GPU-accelerated Chromium/CEF (D3D11 via Wine + DXVK picking the NVIDIA adapter). It might seem weird that I'm trying to run Windows-native Steam under Wine, but it's just a current project I'm working on. Ordinary Vulkan swapchains take the PRIME sysmem path and thus don't trigger any crashes

Bug Incidence

Always

nvidia-bug-report.log.gz

nvidia-bug-report.log.gz

More Info

Proposed fix: give objects that can never produce an sg_table a funcs table without .get_sg_table. This is chosen automatically for GEM types lacking a prime_get_sg_table op, and per-instance for vidmem-backed NVKMS memory.

I also wrote a patch to help fix this (sorry for all the comments; I did ask AI to help me).

nvidia-drm-vidmem-no-sgt.patch

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions