fix(pack): the build machine does not travel, and packages ship stripped (#460) - #464
Merged
Conversation
…ped (#460) A `kind = "shared"` package kept the DT_RUNPATH the link gave it — a list of absolute paths into the BUILD MACHINE's store — so on any other machine the consumer died with `libstdc++.so.6: cannot open shared object file`. The issue's suggested fix does not work, and that is the whole design. Measured on a real package with the build machine's store made unreachable: stale absolute DT_RUNPATH consumer's DT_RPATH inherited? no rc=127 no tag at all YES ok DT_RUNPATH = $ORIGIN no rc=127 DT_RUNPATH = "" (what --set-rpath '' writes) no rc=127 An object carrying ANY DT_RUNPATH makes the loader skip the whole inherited DT_RPATH chain for that object's dependencies. So the criterion is "there is no tag", and removing it is the right answer rather than a compromise: the consumer's own DT_RPATH is the same closure — payload, package dir, SubOS farm — resolved on the machine that will actually run it. New `mcpp.pack.relocate` edits PT_DYNAMIC in process (delete the slot, shift the tail, pad with DT_NULL; same file length) instead of shelling out to patchelf. Library packs are cross-target by construction and have no host gate, so `sandbox_patchelf` resolves to nothing on a macOS or Windows host — and the application packer's shape for that is `if (!patchelf.empty())`, i.e. silently do nothing. ELF32 and big-endian are covered by unit tests; no CI job produces one and `--target` can. Mach-O LC_RPATH is read and reported, not yet rewritten. Also refuses `mcpp pack <program>` for a Mach-O artifact. That path resolves the dependency closure with `LD_TRACE_LOADED_OBJECTS=1 '<binary>'`, which is glibc's variable — dyld ignores it and RUNS THE PROGRAM, then parses its stdout as a dependency table and reports `Packed`. Keyed on the format, not the host, like the `_WIN32` refusal beside it. Never noticed because the e2e harness grants the `pack` capability only where elf+patchelf exist, i.e. Linux. And the third silent one: the SONAME alias' copy fallback read `leg.artifact` rather than the staged file. Byte-identical while nothing modified the staging copy; with relocate and strip in place it would ship an unprocessed library under the exact name the loader asks for, on the machines where create_symlink fails. Packaging now builds release and strips what it ships. Only the profile FALLBACK changes (dev -> release); `--profile` and `[build] default-profile` still win, so pack never produces flags `mcpp build` would not. Stripping follows dh_strip's division, and the archive row is measured: `--strip-all` on a `.a` removes the archive symbol index and the consumer's link fails with `archive has no index; run ranlib to add one`, while `--strip-debug` links and runs. Shared libraries get `--strip-unneeded` (keeps .dynsym), executables `--strip-all`, and bundled third-party .so files nothing at all. New `--profile` / `--no-strip` / `--debug-symbols DIR` and `[pack] strip` / `[pack] debug_symbols`; `--debug-symbols` separates rather than discards and adds a .gnu_debuglink. e2e 264 puts the defect BACK with patchelf and requires the consumer to FAIL before restoring it: 251 consumed the package on the machine that built it, so it was green throughout this bug's life. The guard reads the DYNAMIC ENTRIES, never the file's bytes — the dead string stays in .dynstr (patchelf leaves the identical residue; .dynstr is tail-merged and deleting it cannot be shown safe), and a byte-pattern check would also have flipped to green for an unrelated reason the day stripping landed.
… the compiler
Keying `inBandDebugInfo` on `tc.compiler != MSVC` is wrong in both
directions, and each direction is a configuration mcpp ships:
clang -> x86_64-windows-msvc produces .pdb debug info, and would have been
asked to strip in-band DWARF that is not there.
Apple clang on macOS ships no llvm-strip, so the rule would REFUSE
every `mcpp pack` on a Mac — for a format whose
linked image carries a debug MAP (N_OSO stanzas
naming the .o files) and leaves the DWARF
outside it. `objcopy --only-keep-debug` there
has nothing to copy, and .dSYM is dsymutil's job.
`debug_info_is_in_band(canonicalTriple)` answers it segment-wise from the
canonical triple, which both packers have already resolved. ELF and PE/MinGW
are in-band; Mach-O and the MSVC ABI are not.
check_docs_style.sh enforces bilingual heading parity, and the zh section had been anchored on the wrong neighbour: it landed after 配置项 as an h3 where the English one is an h4 immediately following the Windows cross-packing note. Same place, same level — the two files are read side by side.
…-bundle relocate needs no patchelf Two defects found reviewing my own diff. A fat package's legs SHARE an artifact name — `libmathkit-shared.so` for both the gnu and the musl leg is the documented normal case, not a corner one — so a flat `--debug-symbols` directory had the second leg overwrite the first, and the first artifact's .gnu_debuglink then resolved to the other target's symbols. Silently. The debug files now mirror `lib/<triple>/`, and 265 asserts the layout rather than just the file's existence. And the empty-bundle case sat inside `if (!patchelf.empty())`, which is the exact shape the in-process editor exists to remove: `mcpp pack --mode system` on a host whose sandbox has no patchelf left the build machine's store in the artifact and said nothing at all.
…e a bare build does The PE cross-pack test drops a stand-in msvcrt.dll into the build tree and then expects `mcpp pack` to find it in the closure. With packaging defaulting to release, the two commands resolve to different profiles and therefore different target/<triple>/<fingerprint>/ directories — the file lands in the one pack does not use, and the assertion reads as 'the closure reader failed' when nothing about the closure is wrong. The fixture now states `[build] default-profile`, which settles it for both and doubles as a check that the manifest still outranks pack's fallback. The user-visible half of the same fact is now in docs/02 and the changelog: a file placed beside a built artifact by hand is only visible to pack when both commands resolve to the same profile. The declarative channels are unaffected.
Measured on a macOS ARM64 runner, in a step that had not yet executed a line of mcpp: fatal: unable to access 'https://github.com/openxlings/xlings/': Could not resolve host: github.com Thirty seconds of resolver timeout, then a failed PR check that says nothing about the change under test. Every workflow that reaches an external repository did it with a bare `git clone` — six call sites, one failure mode — while this repository had already learned the same lesson one protocol over: fetch_release.sh carries a long note on `curl --retry` not covering transport-layer errors and `--retry-all-errors` being the flag that does. git has no such flag. .github/tools/git_clone_retry.sh retries EVERY failure, bounded. It does not try to tell a DNS blip from a missing repository: git reports both as exit 128 with only the message to distinguish them, and parsing that message would be a weaker copy of git's own taxonomy that breaks the first time git rephrases one. A genuinely missing repo costs the attempts and then fails with git's own last message intact (measured: 4s, exit 128); a resolver blip costs one backoff.⚠️ The helper's first draft exited 0 on a clone that never succeeded — `$?` after an `if` is the status of the IF STATEMENT, which is 0 when the body did not run. That turns a hard failure into a green step with a missing checkout. Caught by asserting the exit code in its own test rather than by reading the output, which looked correct. ci-aarch64-fresh-install spells the retry inline instead: it checks the repository out LAST ON PURPOSE (a .xlings.json in the workspace re-points where `xlings install` writes, so an early checkout silently changes what the fresh-install steps are testing), so the shared helper does not exist on disk at those two clones. Same policy, and the comment says why it is not the same file. Verified: real clone exits 0 with content; bad repo exits 128 bounded; unreachable host retries then fails with git's message; a partial destination is cleared before the retry; the inline form survives `set -e`.
00_fixture_path_hygiene refuses an inline `$(host_path …)` in a manifest heredoc, and it is right to: the manifest is FILE CONTENT, and on Git Bash a shell-spelled /tmp/... path is read by a native mcpp.exe as 'root of the current drive'. Naming the converted value is what makes the conversion visible where it is used rather than buried in an interpolation.
… names the artifact Both files are about ROUTING — a workspace root hands through to its member's program; `mcpp pack <name>` reaches the target it was given — and both proved it by looking inside a produced bundle. On macOS that bundle can no longer be produced, so the macOS e2e job went red on two tests that are not about bundling at all. Worth noting what their previous green meant: they passed on macOS while `mcpp pack` was resolving the dependency closure by RUNNING the user's program. The bundle they inspected was the one that produced. The Mach-O refusal now names the artifact it reached. That is not decoration: a refusal that does not say WHICH program it got to is indistinguishable from one that resolved the wrong target — the exact defect route_pack_target exists to prevent — and it is the only evidence available on a platform where no bundle can be inspected. 250 additionally keeps the unknown-name case, which must still fail EARLIER and differently, or 'refuses everything' would pass. Linux paths unchanged and re-verified; the new string is present in the rebuilt binary.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #460.
What was wrong
mcpp packof akind = "shared"target copied the linked.sointo the package and did nothing else, so it kept theDT_RUNPATHthe link gave it — a list of absolute paths into the build machine's~/.mcpp/store. On any other machine the consumer dies withlibstdc++.so.6: cannot open shared object file.Why the issue's own suggested fix does not work
The report asks for "no
RUNPATH(or$ORIGIN)". Measured on a realmcpp packproduct consumed by a real mcpp-built program, with the build machine's store made unreachable:.soDT_RPATHinherited?DT_RUNPATH(the old behaviour)DT_RUNPATH = $ORIGINDT_RUNPATH = ""(what--set-rpath ''writes)DT_RPATH(--force-rpath)An object carrying any
DT_RUNPATHmakes the loader skip the entire inheritedDT_RPATHchain when resolving that object's own dependencies. So the criterion is "there is no tag", not "the tag is relative" — and removing it is the right answer rather than a compromise: the consumer's ownDT_RPATHis the same closure (payload, package dir, SubOS farm) resolved on the machine that will actually run it.Three defects, one code path
mcpp pack 对 kind = "shared" 的库没有剥离构建机的 RUNPATH，产物不可移植 #460 itself. New
mcpp.pack.relocateremoves the entry by editingPT_DYNAMICin process — delete the slot, shift the tail, pad withDT_NULL; same file length, no offset fixups. Not patchelf: library packs are cross-target by construction and have no host gate, sosandbox_patchelfresolves to nothing on a macOS or Windows host, and the application packer's shape for that case isif (!patchelf.empty())— silently do nothing. ELF32 and big-endian are covered by unit tests (no CI job produces one;--targetcan). Mach-OLC_RPATHis read and reported, not yet rewritten.mcpp pack <program>on a Mach-O artifact RAN THE USER'S PROGRAM. The closure step asks the dynamic linker viaLD_TRACE_LOADED_OBJECTS=1 '<binary>'; that variable is glibc's, dyld ignores it, so the command just executes the program — and its stdout is then parsed as a dependency table, yielding a bundle reported asPacked. Now refused, keyed on the format (not the host), matching the_WIN32refusal beside it. Never noticed because the e2e harness grants thepackcapability only whereelf+patchelfexist, i.e. Linux.The SONAME alias' copy fallback read
leg.artifact, not the staged file. Byte-identical while nothing modified the staging copy; with relocate and strip in place it would ship an unprocessed library under the exact name the loader asks for — on the machines wherecreate_symlinkfails.Packaging now builds release and strips what it ships
Only the profile fallback changes (
dev→release).--profileand[build] default-profilestill win, somcpp packnever produces flagsmcpp buildwould not.Stripping follows dh_strip's division, and the archive row is measured:
--strip-all--strip-unneeded.dynsym— that IS the export list--strip-debug --enable-deterministic-archives--strip-allremoves the archive symbol index → the consumer's link fails witharchive has no index; run ranlib to add oneBundled third-party
.sofiles are not stripped — mcpp did not build them. New--profile/--no-strip/--debug-symbols DIR, and[pack] strip/[pack] debug_symbols.--debug-symbolsseparates rather than discards and adds a.gnu_debuglink.[pack] stripis deliberately not[profile.<n>].strip: the profile key appends-sto the link, which never touches a static archive and cannot separate anything.Guards
Both, already-clean, archive, missing file) and the strip table.tests/e2e/_elf_tag.sh: 215 and 264 now share one ELF reader..dynstr(patchelf --remove-rpathleaves the identical residue at the identical size;.dynstris tail-merged so deleting it cannot be shown safe). Agrep-style criterion would report a correctly relocated artifact as dirty — and would also have flipped to green for an unrelated reason the day stripping landed.Found while reviewing this diff (later commits on the branch)
Three more silent-wrong-answers, each of which would have shipped:
Whether stripping applies is a property of the TARGET, not the compiler. Keying it on
tc.compiler != MSVCis wrong in both directions: clang →x86_64-windows-msvcproduces.pdbdebug info and would have been asked to strip in-band DWARF that is not there; and Apple's clang ships nollvm-strip, so the rule would have refused everymcpp packon macOS — for a format whose linked image carries a debug map (N_OSO stanzas naming the.ofiles) and leaves the DWARF outside it.debug_info_is_in_band(canonicalTriple)answers it segment-wise.A fat package's
--debug-symbolsfiles collided. Its legs share an artifact name (libmathkit-shared.sofor both a gnu and a musl leg is the documented normal case), so a flat debug directory had the second leg overwrite the first, and the first artifact's.gnu_debuglinkthen resolved to the other target's symbols. They now mirrorlib/<triple>/, and 265 asserts the layout rather than just the file's existence.The empty-bundle relocate was gated on patchelf — the exact shape the in-process editor exists to remove.
mcpp pack --mode systemon a host whose sandbox has no patchelf left the build machine's store in the artifact and said nothing.Locally verified on every target family this machine can reach
--mode staticexamples/05-lib-distribution--profile dev17968 /[build] default-profile = "dev"17968Verified locally
--profile dev17968,[build] default-profile = "dev"17968.Design records:
.agents/docs/2026-08-20-issue460-shared-library-runpath.md,.agents/docs/2026-08-20-pack-and-consumer-model-review.md.