Gate PUBLIC compiler-specific flags by consumer compiler ID

[nyoma-diamond]

Fixes #3125.

Problem

dlib's CMake build applies compiler-specific flags (e.g. /bigobj on MSVC, -mavx/-msse4 on GCC-like compilers) to the dlib target with PUBLIC scope, so they get baked into the exported dlib.cmake and forced onto every downstream consumer regardless of what compiler the consumer uses. This breaks whenever the build compiler and consumer compiler differ, since a flag one compiler understands is often rejected outright by another (the reported case: MSVC-built dlib + clang++ consumer, where clang++ errors on /bigobj).

This is especially easy to hit with vcpkg: a triplet controls which compiler builds a dependency independently of what compiler the consuming project uses, so it's entirely normal to end up with an MSVC-built dlib (per triplet) linked into a project built with clang++ (my case) without doing anything unusual.

#3126 tried fixing this by making the flags PRIVATE, but that stops them reaching any consumer, including same-compiler ones that need them (e.g. MSVC clients that rely on /bigobj being set automatically), which is why it stalled.

This fix

Keep the flags PUBLIC, but gate each one behind a generator expression checked against the consumer's compiler ID instead of the compiler dlib itself was built with:

• dlib/cmake_utils/set_compiler_specific_options.cmake: splits active_compile_opts into active_compile_opts_gcc_public (GNU/Clang/Intel flags) and active_compile_opts_msvc_public (MSVC-only flags).
• dlib/CMakeLists.txt: applies each bucket via target_compile_options(dlib PUBLIC $<$<COMPILE_LANG_AND_ID:CXX,...>:...>), so a bucket only reaches a consumer whose own CXX_COMPILER_ID matches. A mismatched consumer gets neither.
• test_for_sse4/test_for_avx CMakeLists updated for the renamed variables.

A same-compiler consumer sees no behavior change. A different-compiler consumer no longer receives flags it can't parse. COMPILE_LANG_AND_ID needs CMake ≥3.15; dlib already requires ≥3.17.

Testing

• Exported dlib.cmake now shows $<$<COMPILE_LANG_AND_ID:CXX,MSVC>:/bigobj> instead of an unconditional /bigobj.
• End-to-end: dlib built with MSVC, consumer compiled with clang++ (x86_64-pc-windows-msvc) via a downstream CMake+vcpkg project — fails on /bigobj before this change, compiles and links cleanly after.

Known limitation

clang-cl building dlib + clang-cl consuming it won't get /bigobj forwarded either, since CXX_COMPILER_ID is Clang there, not MSVC. $<CXX_COMPILER_FRONTEND_VARIANT:MSVC> would cover it but needs CMake ≥3.30. Can add behind a version check if wanted.

[nyoma-diamond]

The fail on Python / Ubuntu appears to be a separate problem. Builds fine but one runtime test produces a failing result. Seems likely to be a stochastic error since it passed fine on the Windows check or maybe caused by some difference under the hood between Windows vs. Linux/Ubuntu regarding floating point operations? In any case I don't think it's a result of this change

As a side note: With this fix in place, it may be worth adding new CI checks that verify future changes don't regress this issue

[davisking]

Yeah that error wasn't your PR. I'll review this stuff in a bit. Seems good at first glance though.

[davisking]

I pushed a commit that keeps compiler-specific options out of the installed target and reconstructs them in dlibConfig.cmake for the consuming compiler.

This is necessary because dlib may be built with one compiler frontend and consumed with another—for example, built with MSVC and consumed with GNU-style Clang or clang-cl.
Exporting the original flags directly can leak incompatible options such as /bigobj, while dropping them entirely can change SIMD configuration and ABI-sensitive type
alignment.

The new arrangement should keep the existing behavior for build-tree users, selects /bigobj, SIMD, exception, warning, and template-depth options according to the consumer’s
compiler, and adds Windows package-consumer tests for MSVC, clang++, and clang-cl. I also moved the packaging test directories outside the source tree to avoid recursive
installation.

[davisking]

Alright, CI is happy. This still working for you?

[nyoma-diamond]

Yup this looks good. All functional from my end

[davisking]

Sweet. Thanks for the PR :D