Skip to content

feat(compat): cuda-runtime — reach the host NVIDIA driver from an mcpp binary - #346

Merged
Sunrisepeak merged 1 commit into
mainfrom
feat/compat-cuda-runtime
Sep 4, 2026
Merged

feat(compat): cuda-runtime — reach the host NVIDIA driver from an mcpp binary#346
Sunrisepeak merged 1 commit into
mainfrom
feat/compat-cuda-runtime

Conversation

@Sunrisepeak

@Sunrisepeak Sunrisepeak commented Sep 4, 2026

Copy link
Copy Markdown
Member

compat.cuda-runtime — the mcpp-side half of reaching an NVIDIA driver.

Why an mcpp binary cannot see the host driver

An mcpp-built program runs under mcpp's own glibc:

interp: .../xpkgs/xim-x-glibc/2.44/lib64/ld-linux-x86-64.so.2
rpath : .../xim-x-glibc/2.44/lib64:.../xim-x-gcc/16.1.0/lib64:...

so a bare-soname dlopen from inside it does not search the host's library
path at all. A program that links the CUDA runtime statically therefore carries
every redistributable component and still cannot start — the runtime cannot
dlopen libcuda.so.1 and reports it as an insufficient driver version, which
is a confusing way to say "not found".

Observed before this package, on a machine whose driver (550.144.03, CUDA 12.4)
is comfortably newer than the toolkit (12.0):

cudaMalloc: CUDA driver version is insufficient for CUDA runtime version

The shape

The same one compat.glx-runtime and compat.vulkan-runtime already use: a
package-owned directory of symlinks placed on the artifact's runtime search
path through runtime.library_dirs. Nothing is vendored. The driver's
userspace library is in ABI lockstep with the kernel module and NVIDIA's
licence forbids redistributing it, so it can only ever be a host capability.

Two details are load-bearing.

Every pattern is versioned, and the unversioned libcuda.so is deliberately
absent.
mcpp puts runtime.library_dirs on the link line as well as the
runtime path, so a bare libcuda.so harvested here would be picked up by
-lcuda and bind a build to one machine's driver. Versioned sonames are
invisible to the linker and are exactly what dlopen asks for.

Only libcuda.so.1 is linked, and that is measured rather than minimal.
An earlier revision of this PR also harvested libnvidia-ptxjitcompiler, on the
stated theory that PTX JIT would otherwise fail. That claim was wrong. Measured
on driver 550.144.03: a binary built for compute_80 alone, run with only the
one symlink reachable, JITs and returns the correct result on an sm_89 device.
The driver loads its own siblings through its own paths, which the private
loader does not interfere with.

Relationship to xim's libcuda-host-link

That package answers the same question one layer down: it is the single source
of truth for where the host's libcuda is, so xim consumers do not each
reimplement an ldconfig probe. This package answers the mcpp-side question —
how a built artifact reaches it — which needs a directory on the runtime
search path, and that is a thing only an mcpp package can declare.

The probe is not repeated here. An earlier revision of this PR re-probed the
host with its own candidate-directory list, which is exactly the drift that rule
exists to prevent: xim's hostlib module documents four copies of that probe,
three of which were wrong, each making the same reasonable-looking assumption
about a directory layout that FHS, Debian multiarch and Arch answer differently.
This revision declares an install-time edge to the sentinel and links through it.

The edge is xpm.<platform>.deps rather than the package's own [xlings],
because mcpp materialises [xlings] deps for the root project only and this has
to resolve when the package itself installs. Verified: the farm ends up holding
exactly one entry, pointing at the sentinel.

libcuda.so.1 -> .../xpkgs/xim-x-libcuda-host-link/0.0.1/lib/libcuda.so.1

Test

tests/examples/cuda-runtime asserts the two properties that hold on a machine
with a driver and on one without, since every runner here is the latter. It
does not require a device.

Linking at all is the real assertion: the failure this package exists to
prevent is a link-time one, and a build that links proves the versioned-pattern
rule held.

Verified on a machine with a driver:

$ mcpp test
   Compiling compat.cuda-runtime v2026.09.05
   Compiling farm (test)
     Running bin/farm
host driver present, cuInit=0x7bd71fe7cbc0
farm ... ok (0.02s)

and the no-driver path returns 0 with no host driver: ..., which is the
correct answer rather than a skip.

Context

Part of accelerator support in mcpp (mcpp-community/mcpp#559). That change adds
the accelerator dimension of an artifact's identity and the device
translation-unit classification; this package is what lets the resulting binary
actually reach a GPU. Neither depends on the other landing first — this one
uses only keys the current index floor already supports.

@Sunrisepeak
Sunrisepeak force-pushed the feat/compat-cuda-runtime branch from 012f194 to 17a7e26 Compare September 4, 2026 18:47
…p binary

An mcpp-built program runs under mcpp's own glibc, so a bare-soname dlopen from
inside it does not search the host's library path at all. A program that links
the CUDA runtime statically therefore carries every redistributable component
and still cannot start: the runtime cannot dlopen libcuda.so.1 and reports it
as an insufficient driver version, which is a confusing way to say not found.

This is the same problem compat.glx-runtime and compat.vulkan-runtime solve,
and it takes the same shape: a package-owned directory placed on the artifact's
runtime search path through runtime.library_dirs. Nothing is vendored. The
driver's userspace library is in ABI lockstep with the kernel module and
NVIDIA's licence forbids redistributing it, so it can only ever be a host
capability.

The host probe is NOT repeated here. xim's libcuda-host-link already owns the
question of where the host's libcuda is, and its own recipe states why that has
to live in one place: xim's hostlib module documents four copies of that probe,
three of which were wrong, each making the same reasonable-looking assumption
about a directory layout that FHS, Debian multiarch and Arch answer
differently. This package declares an install-time edge to the sentinel and
links through it. The edge is xpm.<platform>.deps rather than the package's own
[xlings], because mcpp materialises [xlings] deps for the root project only and
this has to resolve when the package itself installs.

Only libcuda.so.1 is linked, and that is measured rather than minimal. A draft
also harvested libnvidia-ptxjitcompiler on the theory that PTX JIT would
otherwise fail. Measured on driver 550.144.03: a binary built for compute_80
alone, run with only that one symlink reachable, JITs and produces the correct
result on an sm_89 device. The driver loads its own siblings through its own
paths, which the private loader does not interfere with.

The versioned soname is deliberate and the unversioned libcuda.so is
deliberately absent: mcpp puts runtime.library_dirs on the link line as well as
the runtime path, so a bare libcuda.so here would be picked up by -lcuda and
bind a build to one machine's driver.

The test example asserts what holds on a machine with a driver and on one
without, since every runner here is the latter. It does not require a device:
linking at all is the real assertion, because the failure this package prevents
is a link-time one.

The adapter is Linux-only: the problem it solves is that an mcpp binary on
Linux runs under mcpp's own loader and therefore cannot see the host's driver.
The test member conditions its dependency on the platform rather than being
excluded from the workspace, so the source still compiles on macOS and Windows
and cannot rot unnoticed on the two platforms that do not exercise it.
@Sunrisepeak
Sunrisepeak force-pushed the feat/compat-cuda-runtime branch from 17a7e26 to 1fe2ab8 Compare September 4, 2026 18:53
@Sunrisepeak
Sunrisepeak merged commit a6f625e into main Sep 4, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant