Skip to content

nvidia: fix GPU removal with active clients - #1292

Open
alexey-lukashov wants to merge 1 commit into
NVIDIA:mainfrom
alexey-lukashov:fix/610-unbind-active-client-deadlock
Open

nvidia: fix GPU removal with active clients#1292
alexey-lukashov wants to merge 1 commit into
NVIDIA:mainfrom
alexey-lukashov:fix/610-unbind-active-client-deadlock

Conversation

@alexey-lukashov

Copy link
Copy Markdown

Fixes #1119

Problem

PCI removal removes a GPU from nv_linux_devices before waiting for its
usage count to reach zero. Existing kernel clients release their references
through nvidia_dev_put() or nvidia_dev_put_uuid(), which locate the GPU
through that same list.

Once removal has unlinked the GPU, those release paths can no longer find it.
The usage count therefore does not reach zero and PCI unbind can remain
blocked indefinitely.

Fix

Move GPUs being removed to a private list. New acquisitions continue to
search only the active-device list, while release paths search both active
and removing devices. Remove the GPU from the private list after its
references have drained.

This preserves the early unlinking introduced by the removal lifecycle
refactor, prevents new clients from acquiring the GPU during removal, and
allows existing clients to release their references.

Testing

Tested with the 610.57.04 open kernel modules on Linux 6.8.0-137-generic and
an NVIDIA GeForce RTX 5080 Laptop GPU.

  • Held NVIDIA device references with an active GLX client.
  • Started PCI unbind and observed the non-zero usage-count path.
  • Released the client and confirmed that GPU removal continued and completed.
  • Rebound the GPU to the NVIDIA driver.
  • Confirmed that nvidia-smi and a new GLX client could use the GPU again.
  • Observed no Xid, BUG, Oops, or call trace during the test.

PCI removal removes a GPU from nv_linux_devices before waiting for its
usage count to reach zero. Existing kernel clients release references
through nvidia_dev_put() or nvidia_dev_put_uuid(), which locate the GPU
through that same list. Once removal has unlinked the GPU, those release
paths can no longer find it and the removal wait can block indefinitely.

Move GPUs being removed to a private list. Keep acquisition lookups
limited to active GPUs, while release paths search both active and
removing GPUs. Drop the GPU from the private list after its references
have drained.

This preserves the early unlinking introduced by the removal lifecycle
refactor, prevents new clients from acquiring the GPU during removal,
and allows existing clients to release their references.
@CLAassistant

CLAassistant commented Aug 14, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

NVIDIA driver unbinding more unforgiving (595/Resolute) compared to (590/Questing)

2 participants