-ffile-prefix-map=<staging>=. and --build-id injection in compiler-wrapper

[SebastianPaucar]

This PR makes compiler-wrapper's debug-reproducibility flags compiler-agnostic, per recent @becker33 review that hardcoding -ffile-prefix-map and --build-id would break unsupported compilers and platforms.

Changes:

• cc.sh no longer hardcodes flag names or does platform detection. It appends SPACK_PREFIX_MAP_ARGS/ SPACK_BUILD_PREFIX_MAP_ARGS (file-prefix remapping) and SPACK_BUILD_ID_ARGS (--build-id) only when set and supported (build-id wrapped in $linker_arg).
• Each compiler package now sets these vars itself, based on its own support (see set SPACK_PREFIX_MAP_ARGS for -ffile-prefix-map=<stage>=. and SPACK_BUILD_ID_ARGS for --build-id injection at compiler-wrapper level (companion to spack/compiler-wrapper#19) spack-packages#5353).
• Unsupported compilers just get nothing injected.
• test/run.sh: renamed vars and added tests for the unsupported/unset case. All checks pass.

Companion PR: spack/spack-packages#5353

@haampie @becker33 could you take another look when you have a chance?

cc @wdconinc @tgamblin

[haampie]

This is a compiler flag, it should not be passed to the linker (see mode).

[SebastianPaucar]

Just updated. Thanks for pointing it out! Added mode guard on it, so the compiler flag is injected during compilation, not linking.

[wdconinc]

@SebastianPaucar Can you fix the shellcheck checks that are failing here?

[wdconinc]

I think this is also ready for review, right?

[SebastianPaucar]

@SebastianPaucar Can you fix the shellcheck checks that are failing here?

Done, just fixed the shellchecks failures

[wdconinc]

@haampie Please review this.

[wdconinc]

@haampie @tgamblin Anyone available to review this?

[becker33]

This argument is not supported by all compilers that Spack supports. This needs to be refactored in such a way that Spack is still usable with older compilers and compilers that do not support this option.

Reference table generated by Gemini:

Compiler Suite | Supports -ffile-prefix-map? | Notes / Engine
________________________________________________
GCC (8+) | Yes | Native implementation.
Clang / LLVM (10+) | Yes | Native implementation.
Intel oneAPI (icx) | Yes | Inherited from Clang frontend.
Intel Classic (icc) | No | Proprietary engine.
IBM Open XL | Yes | Modern Clang-based frontend.
IBM Classic XL | No | Proprietary engine.
Fujitsu (fcc) | Yes / No | Supported only in Clang Mode (-Nclang).
NVHPC (nvc/nvfortran) | No | Uses LLVM backend but rejects this frontend flag.
NAG Fortran (nagfor) | No | Completely standalone, proprietary architecture.
MSVC | No | Uses native Microsoft engine.

There is no need to support MSVC, since that doesn't use the same compiler wrapper. But this needs to support:

1. intel-oneapi-compilers-classic
2. nvhpc
3. gcc@:7
4. clang@:9
5. xl@:15 and xl_r@:15
6. fujitsu in non-clang mode
7. nag

Most likely, the best option is to include the option name in the variable we emit from Spack. Then individual compilers can either override it with their own option for the same capability, or set it to empty if they lack the support entirely.

[SebastianPaucar]

Thanks @becker33 for reviewing this. Just updated. Checks pass successfully.

Yes, -ffile-prefix-map is GCC/Clang specific and would break against other compilers that don't support it.

I redesigned this so cc.sh no longer needs to know the compiler-specific flag name for the prefix remapping. It is wired in the new changes I made in spack/spack-packages#5353. Now each compiler package declares its own support (right now available for gcc, llvm, and intel-oneapi-compilers), but other compilers like nvhpc, nag, xl, etc. could also be plugged. The complete flag string is passed from compiler_wrapper/package.py via the SPACK_PREFIX_MAP_ARGS and SPACK_BUILD_PREFIX_MAP_ARGS shell variables, and cc.sh just appends them. cc.sh is agnostic to the specific flag throughout this process.

cc @haampie @wdconinc @tgamblin

[becker33]

This needs to be abstracted into a variable (which can be set empty or unset on macos)

[SebastianPaucar]

@becker33 to clarify, should --build-id injection be controlled by platform detection only, or let it be user-configurable (via config, or even a variant)? The purpose of adding --build-id is to let binaries built with this compiler wrapper be identified by an ID that GDB can automatically look up for debugging.

Indeed, the combination of -ffile-prefix-map + --build-id is meant to make debug info GDB-findable automatically, with no machine-specific build paths embedded in DWARF. I was thinking of this PR as a proposal for a new compiler-wrapper release (e.g. compiler-wrapper v1.1.1) to get reproducible (and redistributable) debug info and debuggable installations.

[becker33]

I was just thinking use SPACK_BUILD_ID_ARGS and then the compilers can set it appropriately based on the platform, I don't think we want to make it user configurable.

So gcc could have:

setup_dependent_build_environment(self, env, dependent_spec):
    if dependent_spec.satisfies("platform=linux"):
        env.set("SPACK_BUILD_ID_ARGS", "--build-id")

And then the wrapper could use:

        case "$mode" in
            ld)
                append flags_list "$SPACK_BUILD_ID_ARGS" ;;
            ccld)
                append flags_list "$linker_arg$SPACK_BUILD_ID_ARGS" ;;

with a guard above to only do this if SPACK_BUILD_ID_ARGS is set and non-empty.

[SebastianPaucar]

@becker33 Thanks for this direction. Updated. SPACK_BUILD_ID_ARGS is now set per-compiler-package per-platform (can see those new commits for --build-id in spack/spack-packages#5353), so cc.sh just appends it via $linker_arg when non-empty. Verified with spack build-env that SPACK_BUILD_ID_ARGS reaches the real build environment correctly. Added a test to cover the unset case. All tests pass.

[SebastianPaucar]

@becker33 PR description updated, suggested changes made, and I've requested review again. This PR is ready for another look. As mentioned before, this is the foundation for debuggable installations and debug info redistribution in Spack, worked on with @wdconinc and previously discussed with @haampie. Happy to dig into any observations you can point out.