fix(docker): keep libgomp1 in the Ubuntu runtime image (cuDSS threading layer, #1670) - #1888
fix(docker): keep libgomp1 in the Ubuntu runtime image (cuDSS threading layer, #1670)#1888vgong-nv wants to merge 1 commit into
Conversation
cuDSS's threading layer (libcudss_mtlayer_gomp) dlopens libgomp.so.1. The 26.08 Ubuntu images do not have it: ci/docker/Dockerfile installs gcc to build psutil and purges it with --autoremove in the same layer, and libgomp1 was only ever present as a dependency of gcc. Any LP/MIP whose root relaxation reaches barrier then dies in cudssSetThreadingLayer (status 3) and the REST client sees "<id> was aborted"; small problems pass because simplex/PDLP finish the root LP first, which is why NVIDIA#1670 looked random. Install libgomp1 explicitly alongside gcc so apt marks it manual and the purge keeps it. Verified on ubuntu:24.04: with `gcc` alone the purge removes libgomp1, with `gcc libgomp1` it stays. Dockerfile.ubi is unchanged — UBI 10's base already ships libgomp and dnf remove gcc leaves it. Closes NVIDIA#1670 Signed-off-by: Vincent Gong <vgong@nvidia.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (1)
Included review availability: Your plan provides up to 12 included reviews per hour; 11 remain after this review. 📝 WalkthroughWalkthroughThe Docker install stage now installs ChangescuDSS runtime dependency
Priority: ➖ Normal Estimated code review effort: 2 (Simple) | ~10 minutes Change: Bug fix · Severity of issue fixed: Medium Merge Risk: ⚪ Minimal · up to The runtime image retains the library required for cuDSS threading after compiler cleanup, with no actionable merge risk identified. 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 inconclusive)
✅ Passed checks (4 passed)
Full details: Linked Issues checkExplanation The change in Resolution Provide reviewable results for the two specified Docker environments, including the repeated 100-execution test and its segmentation-fault count. The current evidence is insufficient to decide whether the issue's validation requirements are complete. ✨ Finishing Touches 💡 1🛠️ Fix failing CI checks 💡
🧪 Generate unit tests (beta)
Comment |
|
Thanks for the writeup and validation, Vincent — added the Different mechanism than #1219/#1768/#1774 (wheel-bundled libgomp), same underlying gap: nothing treats "cuDSS needs a real system libgomp" as a first-class dependency. #1883 fixes the #1219 crash mechanism directly (RTLD_NODELETE on cuDSS's threading-layer plugin); this PR closes the corresponding Docker-side gap. Complementary, not overlapping. |
Closes #1670.
What happens
On the 26.08
-cu12/-cu13Ubuntu images, any LP/MIP whose root relaxation reaches the barrier method dies inside cuDSS:The REST client only sees
{'error': '<id> was aborted', 'error_result': True}. Small problems pass because dual simplex / PDLP finish the root LP before barrier initialises cuDSS; large ones fail every time. Same model, same machine, 26.04 image: fine.Why
cuDSS's threading layer (
libcudss_mtlayer_gomp.so) dlopenslibgomp.so.1, and the 26.08 image does not have it:ci/docker/Dockerfileinstallsgccto build psutil and purges it in the same layer (apt-get purge -y --autoremove gcc, since #1602). libgomp1 was only ever present as a dependency of gcc, so the autoremove takes it too. 26.04 had it by that side effect.Fix
Install
libgomp1explicitly in the sameapt-get installas gcc. apt then marks it as manually installed and the purge keeps it. One word in the Dockerfile plus a comment.Verified the apt semantics on
ubuntu:24.04(theLINUX_VERCI builds with):apt-get purge -y --autoremove gccgcc(today)gcc libgomp1(this PR)ii libgomp1:arm64 14.2.0-4ubuntu2~24.04Dockerfile.ubiis unchanged: UBI 10's base image already shipslibgompanddnf remove gccleaves it in place (checked the same way).Testing
apt-get install libgomp1) has been running on a DGX Spark (GB10, arm64, driver 580) for a week of MIP solves — 67k-binary facility-location models, 45–300 s, REST and gRPC — with no cuDSS failure; the same models abort within a second on the unmodified image.build_imagesworkflow; the change is confined to the apt line.Not in this PR
A failed
cudssSetThreadingLayercurrently ends the solver process. Falling back to single-threaded cuDSS would turn a missing library into a slower solve instead of an aborted job; that is a libcuopt change and out of scope here.