diff --git a/.agents/docs/2026-08-02-host-compile-implementation-plan.md b/.agents/docs/2026-08-02-host-compile-implementation-plan.md new file mode 100644 index 00000000..b30729ec --- /dev/null +++ b/.agents/docs/2026-08-02-host-compile-implementation-plan.md @@ -0,0 +1,240 @@ +# 宿主编译单一生产者 — 实施计划 + +> 设计:`.agents/docs/2026-08-02-host-compile-single-producer-design.md` +> 基线:main @ 7169332(2026.8.2.1 已发布)· 本批次发 **2026.8.2.2** + +**Goal:** 让「宿主编译 flag」只有一个生产者,token 是真源、字符串由 token 渲染; +`build.mcpp` 因此自动获得主构建的全部能力(方言、模块、引号、deployment target), +MSVC 下的模块门可以删除。 + +--- + +## Global Constraints + +- **版本号只改两处**:`mcpp.toml:3`、`src/toolchain/fingerprint.cppm:21` → `2026.8.2.2`。 + `.xlings.json` 是 bootstrap pin,**发布并上架后**才单独 bump。 +- **第 1 阶段是纯重构,必须零行为变化**:归一化 `build.ninja` diff 为空, + 且 `stdmod` 的 `std_build_commands` 逐字节相同 —— 否则所有用户的 std BMI 缓存 + 失效重编(缓存目录名派生自含该命令的元数据,`stdmod.cppm:102,135`)。 +- **新模块独立**:生产者放 `src/toolchain/hostflags.cppm`,**不往 + `build_program.cppm` 的匿名 namespace 加函数**(设计 §6.2)。 +- **差异必须显式**:三处装配今天有真实分歧(见下表),共享后要成为**具名选项**, + 不是被抹平,也不是保留成无理由的开关。 +- **每轮验证双工具链 + 清缓存**:`rm -rf target && mcpp cache clean`, + gcc@16.1.0 与 llvm@22.1.8 各跑一遍;先在基线 commit 上做同法对照。 + +### 今天的三处分歧(必须保留并显式化) + +| 关注点 | `flags.cppm` | `build_program.cppm` | `stdmod.cppm` | +|---|---|---|---| +| clang 有 cfg 时是否绕过 | **全平台绕过** | **只在 Linux 绕过**;macOS/Windows 信 cfg 并返回空(`:214-221` 有理由:macOS 链接要 `needs_explicit_libcxx` 那条路负责 libc++abi/unwind) | 全平台绕过(`:232` 说 `--no-default-config` 在所有平台安全) | +| binutils `-B` | 有(非 musl/mingw 且 libstdc++) | 有(非 musl) | 无 | +| `tc.linkRuntimeDirs` | 走 `depRuntimeLibraryDirs` 另一条路 | `-L` +(仅 ELF)`-rpath` | 无 | +| deployment target | 有 | 有(#332 才补) | 有 | + +⇒ 选项面:`clangCfgBypass{Always,LinuxOnly}`、`binutilsB{bool}`、 +`runtimeDirs{bool}`。**每个选项在头文件里必须写清"谁需要它、为什么"**,否则 +就是把三处推导变成一处推导 + 三个魔法开关。 + +--- + +## Task 0:分支与版本号 + +- [ ] 建分支 `feat/host-compile-single-producer` +- [ ] `mcpp.toml:3` 与 `fingerprint.cppm:21` → `2026.8.2.2`;`.xlings.json` 不动 +- [ ] `bash .github/tools/check_version_pins.sh` 通过 +- [ ] 提交 + +## Task 1:基线快照(零 diff 的对照物) + +在**动任何代码之前**先取基线,否则无法证明零 diff。 + +- [ ] **Step 1:** 对 gcc 与 llvm 各生成一次 `build.ninja` 并归一化保存 + +```bash +snap() { # $1 = 标签 + rm -rf target; mcpp cache clean >/dev/null + mcpp build >/dev/null + find target -name build.ninja | head -1 | xargs cat \ + | sed -E 's#/[^ ]*/target/[0-9a-f]{16}##g' > /tmp/ninja-$1.txt +} +snap base-gcc +sed -i 's|^default = "gcc@16.1.0"$|default = "llvm@22.1.8"|' mcpp.toml +snap base-llvm +git checkout -- mcpp.toml +``` + +- [ ] **Step 2:** 保存 `std_build_commands` 基线 + +```bash +find ~/.mcpp/build-cache -name '*.json' -path '*std*' \ + | xargs -I{} sh -c 'python3 -c "import json,sys;d=json.load(open(sys.argv[1]));print(sys.argv[1]);print(d.get(\"std_build_commands\"))" {}' \ + > /tmp/stdcmd-base.txt +``` + +## Task 2:token 优先的 seam(`linkmodel.cppm`) + +**Files:** `src/toolchain/linkmodel.cppm` + +**Produces:** +- `std::vector LinkModel::compile_tokens(const PathEscape&) const` +- `std::vector ClangDriverModel::compile_tokens(const PathEscape&) const` +- 现有 `compile_flags()` 改为 `render_tokens(compile_tokens(esc))` + +- [ ] **Step 1:** 加 `render_tokens`(每个 token 前置一个空格后拼接) + +```cpp +// token 是真源,字符串是渲染结果 —— 不是反过来。历史上只有字符串形态, +// 于是需要 argv 的消费者(build_program)只能自己重写一遍装配; +// #332 的 split_ws 是同一个形状的产物。 +inline std::string render_tokens(const std::vector& tokens) { + std::string out; + for (auto const& t : tokens) { out += ' '; out += t; } + return out; +} +``` + +- [ ] **Step 2:** `LinkModel::compile_tokens` —— 把现有 `compile_flags` 逐句改成 + `push_back`,顺序不变:`--sysroot=

`、每个 `-isystem

` / `-idirafter

` +- [ ] **Step 3:** `ClangDriverModel::compile_tokens` —— `--no-default-config`、 + `-nostdinc++`(**两个独立 token**)、每个 `-isystem

` +- [ ] **Step 4:** 两个 `compile_flags` 改为 `return render_tokens(compile_tokens(esc));` +- [ ] **Step 5:** 单测:对同一 `LinkModel` / `ClangDriverModel`, + `render_tokens(compile_tokens(esc)) == 旧实现的字符串`(把旧实现内联进测试当预言) +- [ ] **Step 6:** 提交 + +## Task 3:`hostflags.cppm` 生产者 + +**Files:** 新建 `src/toolchain/hostflags.cppm` + +**Produces:** + +```cpp +export module mcpp.toolchain.hostflags; + +struct HostFlagOptions { + // clang 自带 cfg 时是否绕过它。主构建全平台绕过(可复现、不依赖 + // 安装期生成物);build.mcpp 的宿主 helper 只在 Linux 绕过 —— macOS 上 + // 链接 libc++abi/unwind 由主构建的 needs_explicit_libcxx 负责,helper + // 自己重复一遍会产生 undefined __cxa_*(build_program.cppm 原注释)。 + enum class CfgBypass { Always, LinuxOnly } cfgBypass = CfgBypass::Always; + // binutils -B:GCC/libstdc++ payload 才需要(musl 与 mingw 自带 as/ld)。 + bool binutilsPrefix = true; + // tc.linkRuntimeDirs 的 -L(+ELF 上的 rpath):让产物能加载私有运行库。 + bool runtimeLibDirs = false; + std::string macosDeploymentTarget; // 已解析值;空 = 不发 +}; + +// 宿主编译 flag,token 形态。三个消费者的唯一生产者。 +std::vector host_compile_tokens(const Toolchain& tc, + const HostFlagOptions& opt, + const PathEscape& esc); +``` + +- [ ] **Step 1:** 实现,顺序**严格照抄 `flags.cppm:319-352` 的现有顺序** + (dm → deployment → lm),因为顺序影响渲染出的字符串 +- [ ] **Step 2:** 单测 `tests/unit/test_hostflags.cpp`: + - 每个 `CompilerId` 都返回自洽结果(方言可取、无空 token) + - `CfgBypass::LinuxOnly` 在非 Linux 上对 clang 返回空 + - deployment target 只在 macOS 且非空时出现 +- [ ] **Step 3:** 提交 + +## Task 4:三个消费者接入(**零 diff 验收**) + +- [ ] **Step 1:** `build_program.cppm::host_base_flags` → 调 + `host_compile_tokens(tc, {CfgBypass::LinuxOnly, !isMusl, true, dt}, plainEsc)`, + 函数体删空。注意 `plainEsc` 是恒等转义(argv 不需要 ninja/shell 转义)。 +- [ ] **Step 2:** `stdmod.cppm:238-253` → `render_tokens(host_compile_tokens(tc, + {CfgBypass::Always, false, false, dt}, shellEsc))` +- [ ] **Step 3:** `flags.cppm:319-337` 的**编译侧** → `render_tokens( + host_compile_tokens(tc, {CfgBypass::Always, …, false, dt}, ninjaEsc))`; + 链接侧(`link_toolchain_flags` / `f.sysroot` / `llvmRootForStdlib`)保持不动 +- [ ] **Step 4:** **零 diff 验收** + +```bash +snap after-gcc; diff /tmp/ninja-base-gcc.txt /tmp/ninja-after-gcc.txt # 必须空 +snap after-llvm; diff /tmp/ninja-base-llvm.txt /tmp/ninja-after-llvm.txt # 必须空 +# std_build_commands 逐字节 +diff /tmp/stdcmd-base.txt /tmp/stdcmd-after.txt # 必须空 +``` + +非空即说明抽错了 —— 回到 Task 2/3 找顺序或分支差异,**不要**改基线迁就。 + +- [ ] **Step 5:** 单测 + e2e(89/92/110/111/112/124/125/143/144/145/164/168/179/181), + gcc 与 llvm 各一遍 +- [ ] **Step 6:** 提交 + +## Task 5:`host_program_argv` + 删 MSVC 模块门 + +**Files:** `src/toolchain/hostflags.cppm`(或新建 `hostprogram.cppm`)、 +`src/build/build_program.cppm` + +- [ ] **Step 1:** 把 build.mcpp 的 argv 组装(方言、`forceCxxLangArgv`、输出前缀、 + 静态运行时、模块 BMI 处理)收进生产者侧,`build_program.cppm` 只负责 + 「读源文件 → 判定 imports → 调用 → 执行 → 解析指令」 +- [ ] **Step 2:** 模块处理按 `bmi_traits(tc)` 分派,含 MSVC 的 + `/interface /TP /ifcOutput` + `/reference =` + —— 与主构建同一张表,不新写 +- [ ] **Step 3:** **删除** `build_program.cppm` 里 + `import mcpp; / import std; not yet supported under MSVC` 那道门 +- [ ] **Step 4:** `tests/e2e/180_msvc_build_mcpp.sh` 第三段: + 从「断言拒绝」改为「断言 `import std;` + `import mcpp;` 可用」 +- [ ] **Step 5:** e2e 181 参数化补 MSVC 一列(`# requires: windows msvc`, + 单独脚本 183 或在 180 内) +- [ ] **Step 6:** 提交 + +## Task 6:能力矩阵守卫 + +- [ ] **Step 1:** `tests/unit/test_hostflags.cpp` 增一条:遍历所有已知 + `CompilerId`,断言 `host_compile_tokens` + `dialect_for` + `bmi_traits` + 三者都能给出完整答案(方言 id 非空、`forceCxxLangArgv` 非空、 + `outputExePrefix` 非空、bmi 的 `bmiDir`/`bmiExt` 非空)。 + **新增工具链族忘了接 = 测试失败**,而不是运行期 `not yet supported`。 +- [ ] **Step 2:** 提交 + +## Task 7:文档 + +- [ ] `docs/07-build-mcpp.md` / `docs/zh/07-build-mcpp.md`:删掉 + 「MSVC 下不支持 import」的说明,改为陈述已支持 +- [ ] 设计文档 §5 能力矩阵勾掉 MSVC 一列 +- [ ] 提交 + +## Task 8:PR → CI 全绿 → 合入 + +- [ ] 本机全量:`mcpp test`、`tests/e2e/run_all.sh`、`check_version_pins.sh` +- [ ] 开 PR,盯 CI(**重点看 macOS/Windows**:零 diff 只在本机证过两条工具链, + Windows 的 MSVC 腿只有 CI 能验) +- [ ] `gh pr merge --squash --admin --delete-branch` + +## Task 9:Release + 生态闭环 + +- [ ] 触发 release.yml,四平台产物 +- [ ] 镜像 xlings-res 双端;失败则**本地 gtc 补传** + (token 在 `~/.config/gitcode-tool/config.json`,用 repo 的 gtc + `/usr/bin/python3`) +- [ ] **两端独立 GET + sha256 核验**(不信 workflow 绿灯) +- [ ] xim-pkgindex bump PR(Sunrisepeak 账号合),sha256 逐个对照 +- [ ] 隔离 workspace 真装 `2026.8.2.2`,跑 e2e 179/181/89/112 +- [ ] **包上架后**才 bump `.xlings.json` bootstrap pin,单独 PR + (索引 artifact 传播滞后会让这个 PR 假红:`not found` ≠ 回归,等传播后重跑) + +--- + +## 实施记录(只有 CI 能发现的) + +| 发现 | 教训 | +|---|---| +| **扩大 `build_program.cppm` 的匿名 ns 同样触发 clang 误编译** | 把 `build_mcpp_module` 在原地改写加大 → macOS 全部 build.mcpp e2e 段错误,和 PR#332 一模一样。约束不是「别加新函数」,是**「别再往那个 ns 加代码」**。修法=整块搬到 `src/build/hostprogram.cppm` | +| **「扫缓存比对字节等价」是假验证** | std 缓存共享且累积,两次快照都含**陈旧条目**,diff 恒为空 —— 我因此放过了一次真实的字符串改动(`-stdlib=libc++` 位置)。**把主张写成测试**:用合成的 `ClangDriverModel`/`ToolchainLinkModel` 直接断言渲染出的字面串 | +| **`-stdlib=libc++` 的位置是兼容面** | 它进 `std_build_commands` → 进 metadata → **决定 std 缓存目录名**。挪一个 flag = 让每个用户的 std BMI 全量失效 | +| **「信任 cfg」不等于「什么都不发」** | 生产者在 trust-cfg 分支提前 `return`,把 deployment target 也跳过了;而 macOS 上 build.mcpp 走的正是这条分支 → std BMI 配置不匹配。旧的手写实现把它放在**最前、无条件**,注释还专门写了 "FIRST and unconditionally" —— 重构时要读懂那句话为什么在 | +| **删掉能力门 = 死代码变活代码** | `-x none` 常年无条件发出,只因 MSVC 到不了那里才无害。门一删,cl 立刻 `D9002: ignoring unknown option '-x'`。**删门时要把门后所有「反正到不了」的分支重新审一遍** | + +## Self-Review + +**设计覆盖**:§4.1 token 生产者 → Task 2/3;§4.2 三种渲染 → Task 4; +§4.3 build.mcpp 一等消费者 → Task 5;§5 能力矩阵 → Task 5/6; +§6.1 字节等价 → Task 1 + Task 4 Step 4;§6.2 独立模块 → Task 3 约束; +§6.4 双工具链验证 → Global Constraints。 + +**已知风险**:`compute_flags` 服务每一次构建,是全仓库 blast radius 最大的函数 +之一。缓解就是零 diff 硬约束 —— 它把"重构对不对"变成一个可机器判定的问题。 diff --git a/.agents/docs/2026-08-02-host-compile-single-producer-design.md b/.agents/docs/2026-08-02-host-compile-single-producer-design.md new file mode 100644 index 00000000..d324be68 --- /dev/null +++ b/.agents/docs/2026-08-02-host-compile-single-producer-design.md @@ -0,0 +1,314 @@ +# 宿主编译单一生产者:让 build.mcpp 的能力等于 mcpp 的能力 + +日期:2026-08-02 · 基线:main @ 7169332(mcpp 2026.8.2.1) +起因:PR #332 复盘 —— 「MSVC 下 build.mcpp 不支持模块」的理由站不住 + +--- + +## 0. 核心原则 + +> **mcpp 能构建一个宿主程序,`build.mcpp` 就应该能被构建。** + +`build.mcpp` 不该有自己的能力清单。它是「一个宿主 C++ 程序」,而编译一个宿主 +C++ 程序正是 mcpp 的本职。今天它有一份**独立的、更窄的**能力清单,原因不是设计 +选择,是一处接口形状不匹配导致的分叉(§2)。 + +判据落到可测形式(§5):对每个受支持的宿主工具链族,`build.mcpp` 必须支持 +`#include` / `import std;` / `import mcpp;` / 两者并用四种形态。**新增一个工具链 +族时不需要在 build.mcpp 侧再做一次。** + +--- + +## 1. 现状:同一件事有三处装配 + +同一批「宿主编译 flag」被独立拼了三遍: + +| # | 位置 | 服务对象 | 输出形态 | +|---|---|---|---| +| 1 | `flags.cppm::compute_flags` | 主构建的每个 TU | ninja 命令**字符串** | +| 2 | `stdmod.cppm:238-253` | std 模块自身的编译 | shell 命令**字符串** | +| 3 | `build_program.cppm::host_base_flags` | `build.mcpp` | **argv token 向量** | + +三者都要处理同一组关注点:clang cfg 绕过、libc++ 头、sysroot / `-isystem` / +`-B`、`-Wl,-rpath`、`linkRuntimeDirs`、macOS deployment target、`-std=` 方言、 +std BMI 引用、引号。 + +**证据一 —— 同一段 clang cfg 绕过写了三遍:** + +``` +flags.cppm:307,321 --no-default-config -nostdinc++ + libc++ headers +stdmod.cppm:240 " --no-default-config -nostdinc++ -stdlib=libc++" +build_program.cppm:227 push_back("--no-default-config") … +``` + +**证据二 —— 用注释强制跨文件不变量。** `stdmod.cppm:249`: + +> Deployment target must mirror what flags.cppm emits for normal TUs + +本仓库已经用一份 memory 记录过这个失败模式(*注释无法强制跨文件不变量*, +`check_version_pins.sh` 就是为此而生)。这里是同一个病的第二例。 + +**证据三 —— PR #332 的四个 bug 全是它的实例。** 每一个都是 +「`flags.cppm` 早就做对了,另外两处不知道」: + +| bug | `flags.cppm` 里的正确做法 | +|---|---| +| per-TU include 路径不加 shell 引号 | `:241` `shell_quote_arg` | +| BMI flag 路径不加 shell 引号 | 同一层 | +| build.mcpp 缺 macOS deployment target | `:331` 每个 TU 都发 | +| build.mcpp 无 MSVC 方言 | `ninja_backend` 全套 `/interface /TP /ifcOutput` | + +而且不是第一次:memory 记着 0.0.9x 的同款 —— +*`build_program.cppm` 重推链接策略未复用 `prepare.cppm` 的 musl→static*。 +**同一个文件,同一个病,第二轮。** + +--- + +## 2. 为什么会分叉 —— 根因不是"忘了复用" + +收敛其实做过一轮,而且做对了一半:`linkmodel.cppm` 导出了装配助手,签名带可插 +拔的路径转义器: + +```cpp +// linkmodel.cppm:61 / :111 +std::string LinkModel::compile_flags(const PathEscape& esc) const; +std::string ClangDriverModel::compile_flags(const PathEscape& esc) const; +``` + +谁在用: + +``` +flags.cppm:322,337,349 dm.compile_flags(ninjaEsc) / lm.compile_flags(ninjaEsc) ✓ +stdmod.cppm:243,245 lm.compile_flags(shellEsc) —— 但 dm 那半是手写的 ✗ +build_program.cppm 两半都手写 ✗ +``` + +**根因:这个 seam 只产字符串。** `build_program` 需要的是 argv token 向量 +(它直接 `capture_exec(argv)`,不经 shell),字符串接不上,于是整段重写。 +`stdmod` 需要 shell 字符串,能用一半,另一半因为要拼 `shellEsc` 就顺手抄了。 + +这正是 PR #332 里 `split_ws` 的同一形状:**表里只有字符串形态,argv 消费者只能 +自己切**。那次的修法(方言表同时存 `span` token)已经证明了方向。 + +--- + +## 3. 明确排除:不让 build.mcpp 走 ninja + +`build.mcpp` 的输出**是构建计划的输入**: + +``` +prepare.cppm:3863 / 3990 run_build_program() ← 编译并运行 build.mcpp +prepare.cppm:4086 scan() ← 扫模块图(要看到它生成的源文件) +prepare.cppm:4202 make_plan() ← 生成构建计划 +``` + +`mcpp:generated=` / `mcpp:source=` / `include-dir=` / `cxxflag=` 全部在扫描之前 +改写 `buildConfig`。**它不可能是它自己所喂的那张图里的节点**——循环依赖,不是 +未实现。Cargo 的 `build.rs` 同形。 + +「那就单独生成一张小 ninja 图」也不采纳:一个 TU 拿不到增量收益,却给**每次 +构建的关键路径**加一次 ninja 生成 + 进程启动;而且真正重复的东西在 flag 装配层, +换个执行器并不能消掉它。 + +**结论:执行方式保持 `capture_exec`。要共享的是 flag 装配,不是构建图。** + +--- + +## 4. 设计:token 优先的单一生产者 + +### 4.1 反转数据形状 + +新增 `mcpp.toolchain.hostflags`(或并入 `linkmodel.cppm`),产出 **token**: + +```cpp +struct HostCompileFlags { + std::vector compile; // argv token,已是最终形态 + std::vector link; +}; + +struct HostFlagOptions { + std::string_view cppStandardFlag; // 已解析的 -std= / /std: + std::string_view macosDeploymentTarget;// 已解析值(platform::macos) + bool staticRuntime = false;// musl / mingw 自包含策略 +}; + +HostCompileFlags host_compile_flags(const Toolchain& tc, const HostFlagOptions&); +``` + +**token 是唯一真源,字符串由 token 渲染得到**,而不是反过来: + +```cpp +// 一处渲染,两种转义 +std::string render(const std::vector& tokens, const PathEscape& esc, + bool shellQuote); +``` + +### 4.2 三个消费者,三种渲染 + +| 消费者 | 用法 | +|---|---| +| `flags.cppm` | `render(f.compile, ninjaEsc, /*shellQuote=*/true)` → 现有 `compile_toolchain_flags` | +| `stdmod.cppm` | `render(f.compile, shellEsc, true)` → 现有 `sysroot_flag` | +| `build_program.cppm` | **直接用 `f.compile`**,不渲染 —— 它本来就要 argv | + +`host_base_flags` 随之删除。 + +### 4.3 build.mcpp 成为一等消费者 + +在 token 生产者之上,再给「编译一个宿主单 TU 程序」一个入口,把 +`build_program.cppm` 里手写的那部分也收进去: + +```cpp +struct HostProgram { + std::filesystem::path source; // build.mcpp + std::filesystem::path output; // build.mcpp.bin / .exe + std::vector imports; // "std", "std.compat", "mcpp" + std::filesystem::path workDir; // BMI staging 目录 +}; + +std::expected, std::string> +host_program_argv(const Toolchain& tc, const HostFlagOptions&, const HostProgram&); +``` + +它内部负责:方言(`dialect_for`)、语言强制(`forceCxxLangArgv`)、输出 +(`-o` vs `/Fe:`)、静态运行时、以及**模块**——按 `bmi_traits(tc)` 决定 +`gcm.cache` 暂存 / `-fmodule-file=` / `/reference`,和主构建同一张表。 + +于是 §0 的原则变成结构性事实:**MSVC 的 `.ifc` 支持不需要在 build.mcpp 侧单独 +实现**,它来自共用的 `bmi_traits` + `dialect`。今天那道 +「`import mcpp;` / `import std;` not yet supported under MSVC」的门可以直接删掉。 + +--- + +## 5. 能力对等:可测判据 + +原则若只写在文档里,就会重蹈 §1 证据二。因此落成矩阵测试: + +| | `#include` | `import std;` | `import mcpp;` | 两者并用 | +|---|---|---|---|---| +| gcc(glibc / musl / mingw) | ✓ | ✓ | ✓ | ✓ | +| clang(libc++ / MSVC STL) | ✓ | ✓ | ✓ | ✓ | +| msvc(cl.exe) | ✓ | ✓ | ✓ | ✓ | + +- e2e 181 扩成参数化的四形态(已覆盖 gcc/clang 三种,补 MSVC 一列); +- e2e 180 的第三段从「断言拒绝」改为「断言可用」; +- **新增守卫**:一个单测断言 `host_compile_flags` 对每个已知 `CompilerId` 都返回 + 非空且自洽的结果(方言、std flag、输出前缀齐备),让"新增族忘了接"变成编译期/ + 测试期失败,而不是运行期的 `not yet supported`。 + +--- + +## 6. 迁移约束与风险 + +### 6.1 字节等价是硬约束(不是"最好如此") + +`stdmod` 的缓存**目录名派生自元数据,而元数据里含 `std_build_commands`** +(`stdmod.cppm:102,135`)。flag 字符串变一个字节 → 所有用户的 std BMI 缓存全部 +失效并重编(memory 记过本机 26GB / 1198 目录的量级)。 + +因此:**GCC / Clang 路径上,重构后渲染出的字符串必须与今天逐字节相同。** token +按现有顺序拼接即可做到。验证方法用仓库既有的:**归一化 diff `build.ninja`**, +再加一条 `std_build_commands` 的前后对比。只有 MSVC 是行为新增。 + +### 6.2 clang 模块误编译雷区(实测,机制未明) + +**结论先行:新生产者放在自己的模块里,不要往 `build_program.cppm` 的匿名 +namespace 里加函数。** 这不是一般性风格规则,是一条基于实测的避让。 + +PR #332 期间,macOS CI 上所有 build.mcpp e2e(89/92/179/181)段错误。崩溃点在 +`contract_env()` —— 那批改动**完全没有碰过**的函数,而且发生在 build.mcpp 被编译 +之前。ASAN 定位到: + +```cpp +e.emplace_back("MCPP_TARGET_OS", t.os); // 写向 0x0 +``` + +埋点打出的局部 vector 状态,在**第一次成功的 emplace_back 之后**就已经坏了: + +``` +e.size=6148912096828808697 cap=1 data=0x7142c3fe6150 t.os='linux' +``` + +`data` / `capacity` 正常,`__end_` 是垃圾。 + +**逐块剥离的结果**(每轮 `rm -rf target && mcpp cache clean`): + +| 拿掉 | 结果 | +|---|---| +| `import mcpp.toolchain.dialect` | 仍崩 | +| compileArgv 方言化 / `capture_exec` 传 env / `dial` 局部变量 / `host_base_flags` 提前返回 | 仍崩 | +| **`split_ws` 函数本身**(此时它的调用点已全部删除) | **好了** | +| 换成平凡的 `int dummy_probe(int)` 放同一位置 | 不复现 | + +也就是说:一个**从未被调用**的函数,仅因存在于该匿名 namespace,就让邻居函数里 +的局部 `std::vector` 被写坏;而换一个形状简单的函数则不会。环境 clang 22.1.8 + +C++20 modules + `-O2` + libc++;GCC 下从不出现,所以 Linux 本地全绿而 macOS / +Windows 自举全红。 + +**已知边界**:代码里没有 UB —— 一个不被调用的函数不可能暴露另一个函数局部变量 +的 UB。但**未归约成最小复现、未上报上游**,所以确切机制不明。这里记录的是可复 +现的经验事实,不是理论。 + +**如何识别复发**:崩溃点在本次改动没碰过的函数里、只在 clang 平台出现、ASAN 报 +局部容器指针为垃圾 —— 就先怀疑它,直接走"逐块剥离到只剩新增代码"的路子,不要 +读 diff 找 UB(那次读 diff 完全无效,代码里确实没有错)。 + +顺带:`split_ws` 之所以存在,是因为方言表把 `-x c++` 存成字符串而 argv 消费者要 +token。改成表里直接存 token 后它根本不需要 —— 这正是 §4.1「token 是唯一真源」 +的由来,**本方案在消除重复的同时也移除了这个触发源**。 + +### 6.3 blast radius + +`compute_flags` 服务每一次构建,是全仓库最高风险的函数之一。缓解: +分两步走(§8),第一步只做「token 化 + 渲染」且要求字节等价,第二步才接 +build.mcpp 和删门。 + +### 6.4 验证必须双工具链、双清缓存 + +PR #332 的教训:只改 `mcpp.toml` 的工具链就重建,会把 gcc/libstdc++ 编的依赖链进 +clang/libc++ 的 mcpp,崩得和真 bug 一样。每轮验证: +`rm -rf target && mcpp cache clean`,且先在基线 commit 上做同法对照。 + +--- + +## 7. 不做 + +| 项 | 理由 | +|---|---| +| build.mcpp 走 ninja / 生成第二张图 | §3:排序约束 + 一个 TU 无增量收益 | +| 把 build.mcpp 塞进主构建图 | §3:循环依赖 | +| 统一**主构建的**链接侧 | 主构建链接的是 target 产物、build.mcpp 链接的是宿主 helper,策略本就不同(`staticHostHelper`)。`flags.cppm` 的链接装配保持原样 | +| 给 build.mcpp 加多源文件 / C / 汇编支持 | 它是单 TU 程序,这是 L3 的设计选择,不在本方案范围 | + +--- + +## 7.5 实施中相对本设计的两处偏差(已落地) + +| 偏差 | 原因 | +|---|---| +| **补了 `host_link_tokens`**(§7 原写「链接侧不做」) | `build.mcpp` 是**一次驱动调用同时编译和链接**,`host_base_flags` 里本就含 `-fuse-ld=lld` / `--rtlib` / `-L` / `-rpath` / `--dynamic-linker`。不覆盖链接侧就根本迁不动它。§7 的排除项因此收窄为「不动**主构建的**链接装配」——`flags.cppm` 的链接侧确实一行没改 | +| **`stdmod` 的 deployment target 仍由它自己追加** | 三处的 flag **顺序**不同:`flags.cppm` 是 dm→deployment→lm,`stdmod` 是 dm→lm→deployment。让生产者统一顺序会改变 `std_build_commands` 字符串 → 按 §6.1 会让每个用户的 std BMI 全量失效。取舍:共享**装配**(手写的 dm 块已删除),只把这一个 flag 的**位置**留在本地并写明原因。位置统一留作后续 —— 届时应与一次本就会改变 std 身份的变更搭车 | + +| 序 | 内容 | 规模 | 验收 | +|---|---|---|---| +| 1 | `host_compile_flags` token 生产者 + `render()`;`flags.cppm` / `stdmod.cppm` 改调 | 中 | **归一化 diff build.ninja 零差异** + `std_build_commands` 逐字节相同(三平台) | +| 2 | `build_program.cppm` 删 `host_base_flags`,改用 token | 小 | e2e 89/92/110/111/112/124/125/143/144/145/164/168/179/181 全过,双工具链 | +| 3 | `host_program_argv` 收编模块处理;删 MSVC 模块门 | 中 | e2e 180 第三段改为断言可用;e2e 181 补 MSVC 列 | +| 4 | 能力矩阵单测(§5 第三条) | 小 | 新增 `CompilerId` 时测试失败 | + +第 1 步单独成 PR 且必须零 diff —— 它是纯重构,任何行为变化都说明抽错了。 + +--- + +## 附:核对过的坐标 + +| 坐标 | 内容 | +|---|---| +| `src/toolchain/linkmodel.cppm:61,111` | 已有的字符串装配 seam(`compile_flags(esc)`) | +| `src/build/flags.cppm:307-349` | 装配 #1,唯一正确使用 seam 的一处 | +| `src/toolchain/stdmod.cppm:238-253` | 装配 #2;`:249` 是"注释强制不变量" | +| `src/toolchain/stdmod.cppm:102,135` | 缓存身份含 `std_build_commands` → §6.1 | +| `src/build/build_program.cppm:187-270` | 装配 #3(`host_base_flags`) | +| `src/build/prepare.cppm:3863,3990,4086,4202` | 排序约束的证据(§3) | +| `src/build/ninja_backend.cppm:645,820` | 主构建已有的 MSVC 模块管线 | +| `tests/e2e/99_msvc_native_build.sh` | 证明 MSVC 模块在主构建可用(产出 `.ifc`) | diff --git a/docs/07-build-mcpp.md b/docs/07-build-mcpp.md index d7044415..6cfc9eac 100644 --- a/docs/07-build-mcpp.md +++ b/docs/07-build-mcpp.md @@ -126,11 +126,9 @@ while the project targets something else. `#include` still works and stays the right choice for a program that only needs `std::fopen`; there is no requirement to modularize a build script. -> **Not yet under MSVC.** Named modules with `cl.exe` go through `.ifc` + -> `/reference`, a pipeline mcpp has not wired up. A `build.mcpp` using -> `import mcpp;` or `import std;` under a native MSVC toolchain fails with an -> explicit message telling you to use `#include` or a GCC/Clang toolchain — -> `#include`-based programs are fully supported there. +Every toolchain mcpp can build a host program with can build a `build.mcpp`, +including native MSVC — the module handling reads the same tables the main +build does, so `cl.exe`'s `.ifc` + `/reference` needs no separate support. ## Environment contract (mcpp 0.0.95+) diff --git a/docs/zh/07-build-mcpp.md b/docs/zh/07-build-mcpp.md index 3133ce11..22391de8 100644 --- a/docs/zh/07-build-mcpp.md +++ b/docs/zh/07-build-mcpp.md @@ -117,10 +117,9 @@ mcpp 会把它自己构建时用的**同一份** std 模块暂存过来,缓存 `#include` 依然有效,对只需要 `std::fopen` 的程序也依然是更合适的选择——构建脚本 没有必须模块化的要求。 -> **MSVC 下尚不支持。** `cl.exe` 的具名模块走 `.ifc` + `/reference`,这条管线 mcpp -> 还没接。在原生 MSVC 工具链下使用 `import mcpp;` 或 `import std;` 的 `build.mcpp` -> 会得到一条明确的报错,告诉你改用 `#include` 或换 GCC/Clang 工具链——基于 -> `#include` 的程序在那里是完全支持的。 +凡是 mcpp 能用来构建宿主程序的工具链,都能构建 `build.mcpp`,原生 MSVC 也不例外 +——模块处理读的是主构建同一批表,所以 `cl.exe` 的 `.ifc` + `/reference` 不需要 +单独支持。 ## 环境契约(mcpp 0.0.95+) diff --git a/mcpp.toml b/mcpp.toml index 46247af3..8abec530 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -1,6 +1,6 @@ [package] name = "mcpp" -version = "2026.8.2.1" +version = "2026.8.2.2" description = "Modern C++ build & package management tool" license = "Apache-2.0" authors = ["mcpp-community"] diff --git a/src/build/build_program.cppm b/src/build/build_program.cppm index 93c88efb..fc714c73 100644 --- a/src/build/build_program.cppm +++ b/src/build/build_program.cppm @@ -19,6 +19,8 @@ import mcpp.platform.process; import mcpp.toolchain.cppfly; // std_flag (dialect- and c++fly-aware -std= spelling) import mcpp.toolchain.dialect; // CommandDialect — gnu vs cl.exe spellings import mcpp.toolchain.fingerprint; // hash_file / hash_string (FNV-1a, 16 hex) +import mcpp.build.hostprogram; // bundled `mcpp` module compile (own module: see its header) +import mcpp.toolchain.hostflags; // the shared host-compile flag producer import mcpp.toolchain.linkmodel; // shared C-library / clang-cfg-bypass model import mcpp.toolchain.model; // Toolchain, PayloadPaths, is_clang/is_musl_target/is_mingw_target import mcpp.toolchain.registry; // archive_tool @@ -186,248 +188,34 @@ std::string env_value(const std::string& name) { // only ones needed. Passed as separate argv tokens (no shell). std::vector host_base_flags(const mcpp::toolchain::Toolchain& tc, std::string_view macosDeploymentTarget) { - std::vector f; - - // macOS deployment target, FIRST and unconditionally, because clang - // refuses to load a module built for a different one and this function's - // result feeds every compile in this file: the bundled `mcpp` module's - // precompile, its object step, and the build.mcpp compile itself. Putting - // it anywhere narrower produced the mismatch in whichever direction was - // left out — first the std BMI (built for 14.0) against a compile with no - // version-min, then mcpp.pcm (built at the host default 15.0) against a - // compile that had just been given 14.0. - if constexpr (mcpp::platform::is_macos) { - if (!macosDeploymentTarget.empty()) - f.push_back(std::string("-mmacosx-version-min=") - + std::string(macosDeploymentTarget)); - } - - // MSVC carries none of this on the command line: cl.exe and link.exe find - // headers and import libraries through INCLUDE / LIB, which detection - // synthesized into tc.envOverrides. Emitting the GNU shapes below would - // produce a string of unknown options and then LNK1181. The environment - // is passed to capture_exec instead — that is the whole MSVC "base". - if (tc.compiler == mcpp::toolchain::CompilerId::MSVC) return f; - - const auto lm = mcpp::toolchain::resolve_link_model(tc); - - // Clang with a bundled cfg on LINUX: bypass it (--no-default-config) and - // provide everything explicitly, same as the main build — the cfg is an - // install-time-generated artifact, so trusting it here while bypassing - // it in the main build meant two different toolchains for one project. - // On macOS/Windows keep trusting the cfg: the macOS link additionally - // needs the platform's libc++abi/unwind handling that the main build's - // needs_explicit_libcxx path owns (duplicating it for a host compile - // produced undefined __cxa_*/__gxx_personality_v0), and the fixup - // pipeline regenerates the cfg deterministically anyway. - if (mcpp::toolchain::is_clang(tc)) { - if constexpr (!mcpp::platform::is_linux) return f; - const auto dm = mcpp::toolchain::resolve_clang_driver(tc); - if (dm.hasCfg) { - f.push_back("--no-default-config"); - f.push_back("-nostdinc++"); - f.push_back("-stdlib=libc++"); - for (auto& inc : dm.cxxIncludes) f.push_back("-isystem" + inc.string()); - f.push_back("-fuse-ld=lld"); - f.push_back("--rtlib=compiler-rt"); - f.push_back("--unwindlib=libunwind"); - for (auto& d : dm.libDirs) { - f.push_back("-L" + d.string()); - f.push_back("-Wl,-rpath," + d.string()); - } - } - if (lm.mode == mcpp::toolchain::CLibMode::Sysroot) { - f.push_back("--sysroot=" + lm.sysroot.string()); - } else if (lm.mode == mcpp::toolchain::CLibMode::PayloadFirst) { - for (auto& inc : lm.systemIncludes) f.push_back("-isystem" + inc.string()); - f.push_back("-B" + lm.crtDir.string()); // Scrt1.o/crti.o discovery - for (auto& d : lm.libDirs) { - f.push_back("-L" + d.string()); - f.push_back("-Wl,-rpath," + d.string()); - } - if (!lm.loader.empty()) - f.push_back("-Wl,--dynamic-linker=" + lm.loader.string()); - } - // Runtime lib dirs so the produced program can load private libs in-tree. - for (auto& d : tc.linkRuntimeDirs) { - f.push_back("-L" + d.string()); - f.push_back("-Wl,-rpath," + d.string()); - } - return f; - } - - // GCC: a fresh sandbox g++ needs --sysroot to find the C library + the - // include-fixed headers; without a sysroot, wire the glibc payload directly. - if (lm.mode == mcpp::toolchain::CLibMode::Sysroot) { - f.push_back("--sysroot=" + lm.sysroot.string()); - } else if (lm.mode == mcpp::toolchain::CLibMode::PayloadFirst) { - for (auto& inc : lm.systemIncludes) { - f.push_back("-idirafter"); f.push_back(inc.string()); - } - f.push_back("-B" + lm.crtDir.string()); // crt1.o/crti.o discovery - for (auto& d : lm.libDirs) f.push_back("-L" + d.string()); // -lc/-lm - } - // binutils -B so the driver finds ld/as (GCC, non-musl; musl ships its own). - if (!mcpp::toolchain::is_musl_target(tc)) { - auto ar = mcpp::toolchain::archive_tool(tc); - if (!ar.empty()) f.push_back("-B" + ar.parent_path().string()); - } - // Runtime lib dirs so the produced program can load private libs in-tree. - // -L is link-time and wanted everywhere; rpath is an ELF-only concept — - // this is the one host_base_flags branch a PE target reaches, where the - // flag is inert and the self-containment answer is the static link in - // run_build_program instead (#299). - for (auto& d : tc.linkRuntimeDirs) { - f.push_back("-L" + d.string()); - if constexpr (mcpp::platform::supports_rpath) - f.push_back("-Wl,-rpath," + d.string()); - } + // One driver invocation compiles AND links build.mcpp, so it needs both + // sides. Both come from mcpp.toolchain.hostflags — the same producer + // flags.cppm and the std module build use. This function used to hand-write + // the whole assembly, which is how it kept missing what the main build + // already knew (quoting, the macOS deployment target, the MSVC dialect); + // see 2026-08-02-host-compile-single-producer-design.md. + mcpp::toolchain::HostFlagOptions opt; + // The host helper keeps TRUSTING clang's cfg on macOS/Windows: the macOS + // link needs the libc++abi/unwind handling the main build's + // needs_explicit_libcxx path owns, and duplicating it here produced + // undefined __cxa_* / __gxx_personality_v0. + opt.cfgBypass = mcpp::toolchain::HostFlagOptions::CfgBypass::LinuxOnly; + opt.clangStdlibSelect = true; + // binutils -B so the driver finds ld/as (GCC; musl and MinGW ship their own). + opt.binutilsPrefix = !mcpp::toolchain::is_musl_target(tc) + && !mcpp::toolchain::is_mingw_target(tc); + // The helper is exec'd outside anything mcpp controls, so it must be able + // to find the toolchain's private runtime libs itself. + opt.runtimeLibDirs = true; + opt.macosDeploymentTarget = std::string(macosDeploymentTarget); + + const mcpp::toolchain::PathEscape plain = mcpp::toolchain::no_escape; + auto f = mcpp::toolchain::host_compile_tokens(tc, opt, plain); + for (auto& t : mcpp::toolchain::host_link_tokens(tc, opt, plain)) + f.push_back(t); return f; } -// The bundled `mcpp` build module — a typed API over the stdout wire protocol -// so build.mcpp can `import mcpp;` instead of `#include`. Its own I/O uses -// C-level primitives in the global module fragment, so the module itself -// needs no std BMI and stays buildable before one exists. (That was once also -// a limit on build.mcpp; it no longer is — a build.mcpp may `import std;` and -// the engine stages the same std module the main build uses.) -// The functions mirror the directive set 1:1; they just print the -// `mcpp:` lines the engine already parses. Embedded in the binary (not shipped as -// a file) so it always matches this mcpp's protocol. -// NOTE: the module declaration line uses a `@MODULE@` placeholder (substituted -// with `export module` when written) so mcpp's own line-based module scanner does -// not mistake this embedded string for build_program.cppm exporting a 2nd module. -constexpr std::string_view kMcppModuleSource = R"CPP(module; -#include -#include -@MODULE@ mcpp; -export namespace mcpp { -inline void cxxflag(const char* flag) { std::printf("mcpp:cxxflag=%s\n", flag); } -inline void cflag(const char* flag) { std::printf("mcpp:cflag=%s\n", flag); } -inline void link_lib(const char* name) { std::printf("mcpp:link-lib=%s\n", name); } -inline void link_search(const char* dir) { std::printf("mcpp:link-search=%s\n", dir); } -inline void define(const char* name) { std::printf("mcpp:cfg=%s\n", name); } -inline void generated(const char* path) { std::printf("mcpp:generated=%s\n", path); } -inline void source(const char* path) { std::printf("mcpp:source=%s\n", path); } -inline void include_dir(const char* dir) { std::printf("mcpp:include-dir=%s\n", dir); } -inline void include_dir_after(const char* dir) { std::printf("mcpp:include-dir-after=%s\n", dir); } -inline void rerun_if_changed(const char* path) { std::printf("mcpp:rerun-if-changed=%s\n", path); } -inline void rerun_if_env_changed(const char* var) { std::printf("mcpp:rerun-if-env-changed=%s\n", var); } -// ── environment contract (read side; values injected by the engine) ───── -inline const char* env_or(const char* n) { const char* v = std::getenv(n); return v ? v : ""; } -inline const char* target() { return env_or("MCPP_TARGET"); } -inline const char* target_os() { return env_or("MCPP_TARGET_OS"); } -inline const char* target_arch() { return env_or("MCPP_TARGET_ARCH"); } -inline const char* target_env() { return env_or("MCPP_TARGET_ENV"); } -inline const char* host() { return env_or("MCPP_HOST"); } -inline const char* profile() { return env_or("MCPP_PROFILE"); } -inline const char* out_dir() { return env_or("MCPP_OUT_DIR"); } -inline const char* manifest_dir() { return env_or("MCPP_MANIFEST_DIR"); } -inline bool has_feature(const char* name) { - char buf[256] = "MCPP_FEATURE_"; - unsigned long o = 13; - for (const char* p = name; *p && o + 1 < sizeof buf; ++p, ++o) { - char c = *p; - buf[o] = (c >= 'a' && c <= 'z') ? char(c - 'a' + 'A') - : ((c >= 'A' && c <= 'Z') || (c >= '0' && c <= '9')) ? c : '_'; - } - buf[o] = 0; - return std::getenv(buf) != nullptr; -} -// mcpp#241: resolved install dir of a declared dependency (by its package -// name), or "" if not found. Same sanitize as has_feature; wraps -// MCPP_DEP__DIR. -inline const char* dep_dir(const char* name) { - char buf[256] = "MCPP_DEP_"; - unsigned long o = 9; - for (const char* p = name; *p && o + 5 < sizeof buf; ++p, ++o) { - char c = *p; - buf[o] = (c >= 'a' && c <= 'z') ? char(c - 'a' + 'A') - : ((c >= 'A' && c <= 'Z') || (c >= '0' && c <= '9')) ? c : '_'; - } - buf[o++] = '_'; buf[o++] = 'D'; buf[o++] = 'I'; buf[o++] = 'R'; buf[o] = 0; - return env_or(buf); -} -} -)CPP"; - -// Compile the bundled `mcpp` module into `bdir` and return the extra flags the -// build.mcpp compile needs to import it (the object `mcpp.o` is linked alongside). -// GCC : -fmodules → gcm.cache/mcpp.gcm + mcpp.o; build.mcpp compiles from -// `bdir` (cwd) so GCC finds gcm.cache/mcpp.gcm. -// Clang : --precompile → mcpp.pcm, then -c → mcpp.o; pass -fmodule-file=mcpp=. -// Does the source contain `import ;`? -// -// A plain substring search is not enough here: "import std" is a prefix of -// "import std.compat", so the naive test reports both for a program that -// only imports the latter, and mcpp would build a std BMI nobody asked for. -// Match the whole module name and require the terminating `;`, tolerating -// the whitespace the grammar allows. Occurrences inside comments or string -// literals still match — over-detection costs one cached BMI lookup, never -// a wrong build, and that is the same trade the `import mcpp` check has -// always made. -bool imports_module(std::string_view src, std::string_view name) { - constexpr std::string_view kImport = "import"; - std::size_t pos = 0; - while ((pos = src.find(kImport, pos)) != std::string_view::npos) { - std::size_t i = pos + kImport.size(); - // `importfoo` is not an import. - if (i >= src.size() || (src[i] != ' ' && src[i] != '\t')) { ++pos; continue; } - while (i < src.size() && (src[i] == ' ' || src[i] == '\t')) ++i; - if (src.compare(i, name.size(), name) == 0) { - std::size_t j = i + name.size(); - while (j < src.size() && (src[j] == ' ' || src[j] == '\t')) ++j; - if (j < src.size() && src[j] == ';') return true; - } - ++pos; - } - return false; -} - -std::expected, std::string> -build_mcpp_module(const fs::path& bdir, const fs::path& compiler, - const std::vector& base, const std::string& stdFlag, - bool isClang) { - std::error_code ec; - fs::path cppm = bdir / "mcpp.cppm"; - std::string moduleSrc(kMcppModuleSource); - if (auto p = moduleSrc.find("@MODULE@"); p != std::string::npos) - moduleSrc.replace(p, std::string_view("@MODULE@").size(), "export module"); - { std::ofstream os(cppm, std::ios::trunc); - os << moduleSrc; - if (!os) return std::unexpected(std::string("could not write mcpp module source")); } - - auto run = [&](std::vector argv, const char* what) - -> std::expected { - auto r = mcpp::platform::process::capture_exec(argv, {}, bdir.string()); - if (r.exit_code != 0) - return std::unexpected(std::format("mcpp module {} failed (exit {}):\n{}", - what, r.exit_code, r.output)); - return {}; - }; - auto with_base = [&](std::vector head) { - for (auto& b : base) head.push_back(b); - return head; - }; - - std::vector extra; - if (isClang) { - if (auto r = run(with_base({compiler.string(), stdFlag, "--precompile", - "mcpp.cppm", "-o", "mcpp.pcm"}), "precompile"); !r) - return std::unexpected(r.error()); - if (auto r = run(with_base({compiler.string(), stdFlag, "-c", - "mcpp.pcm", "-o", "mcpp.o"}), "object"); !r) - return std::unexpected(r.error()); - extra.push_back("-fmodule-file=mcpp=" + (bdir / "mcpp.pcm").string()); - } else { - if (auto r = run(with_base({compiler.string(), stdFlag, "-fmodules", "-c", - "mcpp.cppm", "-o", "mcpp.o"}), "compile"); !r) - return std::unexpected(r.error()); - extra.push_back("-fmodules"); - } - return extra; -} - // ── Cache (line-based; one record per line, internal format) ─────────────── // program // compiler @@ -754,24 +542,25 @@ std::expected run_build_program( bool usesStdCompat = imports_module(srcText, "std.compat"); bool usesStd = usesStdCompat || imports_module(srcText, "std"); - // Named modules under cl.exe go through .ifc + /reference, a different - // pipeline from GCC's gcm.cache and Clang's -fmodule-file. That work is - // not done, so say so plainly — one gate for both module kinds, because - // they fail for exactly the same reason and two conditions would drift. - if (msvcHost && (usesModule || usesStd)) { - return std::unexpected(std::string( - "build.mcpp: `import mcpp;` / `import std;` are not yet supported " - "under MSVC.\n" - " Use #include in build.mcpp, or build with a GCC/Clang " - "toolchain.")); - } + // The toolchain's own environment (MSVC's INCLUDE / LIB / VSLANG, which + // detection synthesized from the located VC tools + Windows SDK). Needed + // by every compile below, the module precompile included. + std::vector> compileEnv; + for (auto const& ev : tc.envOverrides) + compileEnv.emplace_back(ev.key, ev.value); + // Named modules dispatch on the same BmiTraits/CommandDialect rows the + // main build uses, so there is no per-family gate here: cl.exe's + // .ifc + /reference works because the table already describes it, not + // because build.mcpp grew a second implementation of it. std::vector moduleFlags; + fs::path mcppModuleObject; if (usesModule) { - auto mf = build_mcpp_module(bdir, hostCompiler, base, std_flag, - mcpp::toolchain::is_clang(tc)); + auto mf = build_mcpp_module(bdir, hostCompiler, base, std_flag, tc, + compileEnv); if (!mf) return std::unexpected(mf.error()); - moduleFlags = std::move(*mf); + moduleFlags = std::move(mf->useFlags); + mcppModuleObject = std::move(mf->object); } // ── `import std;` in build.mcpp ───────────────────────────────────────── @@ -839,15 +628,19 @@ std::expected run_build_program( if (!usesModule) stdFlags.push_back("-fmodules"); stdStagedInBdir = true; } else { - stdFlags.push_back(std::string(traits.stdBmiUsePrefix) - + sm->bmiPath.string()); + // Through bmi_reference_tokens, not string concatenation: the + // traits spell these for the ninja STRING channel, where + // `-fmodule-file=std=

` (one word) and `/reference std=

` + // (two) are indistinguishable. Concatenating produced a single + // argv element with a space inside it, and cl answered + // "C2230: could not find module 'std'". + for (auto& t : mcpp::toolchain::bmi_reference_tokens( + traits.stdBmiUsePrefix, sm->bmiPath)) + stdFlags.push_back(t); if (usesStdCompat && !sm->compatBmiPath.empty()) - stdFlags.push_back(std::string(traits.stdCompatBmiUsePrefix) - + sm->compatBmiPath.string()); - // The prefixes carry a leading space for the ninja string channel; - // an argv element must not. - for (auto& f : stdFlags) - if (!f.empty() && f.front() == ' ') f.erase(0, 1); + for (auto& t : mcpp::toolchain::bmi_reference_tokens( + traits.stdCompatBmiUsePrefix, sm->compatBmiPath)) + stdFlags.push_back(t); } if (!sm->objectPath.empty() && fs::exists(sm->objectPath)) stdObjects.push_back(sm->objectPath.string()); @@ -874,14 +667,24 @@ std::expected run_build_program( for (auto& sf : stdFlags) compileArgv.push_back(sf); // The `.mcpp` extension is unknown to every driver, so without this the // file is handed to the linker as a linker script. - for (auto f : dial.forceCxxLangArgv) compileArgv.emplace_back(f); - compileArgv.push_back(src.string()); + // Per-file where the driver has that form (cl's /Tp), positional + // otherwise. Object files follow on this same command line, and cl's + // global /TP would compile them as C++ source. + if (!dial.perFileCxxPrefix.empty()) { + compileArgv.push_back(std::string(dial.perFileCxxPrefix) + src.string()); + } else { + for (auto f : dial.forceCxxLangArgv) compileArgv.emplace_back(f); + compileArgv.push_back(src.string()); + } if (usesModule || !stdObjects.empty()) { - // Link the module objects (GNU: reset the input language first so the - // .o isn't treated as C++ source; cl.exe infers by extension and is - // unreachable here anyway, gated above). - compileArgv.push_back("-x"); compileArgv.push_back("none"); - if (usesModule) compileArgv.push_back((bdir / "mcpp.o").string()); + // Link the module objects. GNU drivers need the input language reset + // first, or the .o that follows `-x c++` is handed to the frontend as + // C++ source; cl.exe has no `-x` at all and infers from the extension. + // This used to be unconditional and was only harmless while MSVC could + // not reach it — removing that gate made the dead branch live, and cl + // answered with `D9002: ignoring unknown option '-x'`. + if (!msvcHost) { compileArgv.push_back("-x"); compileArgv.push_back("none"); } + if (usesModule) compileArgv.push_back(mcppModuleObject.string()); for (auto& so : stdObjects) compileArgv.push_back(so); } // Self-contained helper link — see the staticHostHelper doctrine above. @@ -903,14 +706,6 @@ std::expected run_build_program( // only mcpp. Otherwise the project root is fine. const bool needsBmiCwd = usesModule || stdStagedInBdir; std::string compileCwd = needsBmiCwd ? bdir.string() : root.string(); - // The toolchain's own environment (MSVC's INCLUDE / LIB / VSLANG, which - // detection synthesized from the located VC tools + Windows SDK). Only - // ninja_backend consumed these before, so a build.mcpp compile under - // cl.exe could not find no matter how correct its argv was — - // the third and last layer of #331's first finding. - std::vector> compileEnv; - for (auto const& ev : tc.envOverrides) - compileEnv.emplace_back(ev.key, ev.value); auto cres = mcpp::platform::process::capture_exec(compileArgv, compileEnv, compileCwd); if (cres.exit_code != 0) { diff --git a/src/build/flags.cppm b/src/build/flags.cppm index 211d5247..4b899ee3 100644 --- a/src/build/flags.cppm +++ b/src/build/flags.cppm @@ -18,6 +18,7 @@ import mcpp.platform; import mcpp.toolchain.clang; import mcpp.toolchain.detect; import mcpp.toolchain.dialect; +import mcpp.toolchain.hostflags; import mcpp.toolchain.linkmodel; import mcpp.toolchain.provider; import mcpp.toolchain.registry; @@ -317,24 +318,31 @@ CompileFlags compute_flags(const BuildPlan& plan) { // path below to locate libc++.a/libc++abi.a for staticStdlib. std::filesystem::path llvmRootForStdlib; + // Compile side: the shared producer (mcpp.toolchain.hostflags), which the + // std module build and the build.mcpp host compile also use. It emits + // clang-cfg bypass → macOS deployment target → C library headers, the + // order this function has always used. + // + // The macOS deployment target is on the command line rather than left to + // the environment so (a) the ninja commands don't depend on env + // propagation and (b) the value participates in the BMI fingerprint via + // canonical flags — mixing targets in one sandbox otherwise reuses a + // std.pcm built for a different arm64-apple-macosxNN triple and dies with + // a config mismatch (observed on macos CI). The link side is added to + // f.ld below (the macOS link path doesn't consume link_toolchain_flags). + // + // binutilsPrefix / runtimeLibDirs stay off here: this function computes + // -B separately into f.bFlag, and routes runtime dirs through + // depRuntimeLibraryDirs. + { + mcpp::toolchain::HostFlagOptions hopt; + hopt.cfgBypass = mcpp::toolchain::HostFlagOptions::CfgBypass::Always; + hopt.macosDeploymentTarget = macosDeploymentTarget; + compile_toolchain_flags = mcpp::toolchain::render_tokens( + mcpp::toolchain::host_compile_tokens(plan.toolchain, hopt, ninjaEsc)); + } if (isClangWithCfg) { - // --no-default-config -nostdinc++ + libc++ headers. - compile_toolchain_flags = dm.compile_flags(ninjaEsc); - // macOS deployment target: make the resolved value explicit on - // the command line so (a) the ninja commands don't depend on env - // propagation and (b) the value participates in the BMI - // fingerprint via canonical flags — mixing targets in one sandbox - // otherwise reuses a std.pcm built for a different - // arm64-apple-macosxNN triple and dies with a config mismatch - // (observed on macos CI). The link side is added to f.ld below - // (the macOS link path doesn't consume link_toolchain_flags). - if (mcpp::platform::is_macos && !macosDeploymentTarget.empty()) { - compile_toolchain_flags += - " -mmacosx-version-min=" + macosDeploymentTarget; - } llvmRootForStdlib = dm.llvmRoot; - // C library headers (payload -isystem, or --sysroot fallback). - compile_toolchain_flags += lm.compile_flags(ninjaEsc); // Linker flags that cfg normally provides. The payload C-runtime // flags (-B/-L/loader) are appended via payload_ld below. link_toolchain_flags = " --no-default-config"; @@ -346,7 +354,6 @@ CompileFlags compute_flags(const BuildPlan& plan) { } else if (lm.mode != mcpp::toolchain::CLibMode::None) { // GCC (or Clang without cfg): --sysroot from probe, or the payload // headers + C runtime (-B for crt discovery, -L for -lc/-lm). - compile_toolchain_flags = lm.compile_flags(ninjaEsc); link_toolchain_flags = lm.link_flags(ninjaEsc); f.sysroot = link_toolchain_flags; } diff --git a/src/build/hostprogram.cppm b/src/build/hostprogram.cppm new file mode 100644 index 00000000..fa53306d --- /dev/null +++ b/src/build/hostprogram.cppm @@ -0,0 +1,210 @@ +// mcpp.build.hostprogram — compiling the bundled `mcpp` module for build.mcpp. +// +// Split out of build_program.cppm for a blunt reason: that file's anonymous +// namespace miscompiles its own neighbours under clang 22.1.8 + C++20 modules +// + -O2. PR#332 established it (an UNUSED `split_ws` was enough to corrupt a +// local vector in `contract_env`), and growing `build_mcpp_module` in place +// reproduced it again — `Segmentation fault: 11` on every macOS build.mcpp +// e2e, in code this change never touched. Mechanism unknown, reproduction +// solid, and the cheap response is to stop growing that namespace. +// +// See .agents/docs/2026-08-02-host-compile-single-producer-design.md §6.2. + +export module mcpp.build.hostprogram; + +import std; +import mcpp.platform; +import mcpp.platform.process; +import mcpp.toolchain.dialect; +import mcpp.toolchain.hostflags; +import mcpp.toolchain.model; + +export namespace mcpp::build { + +namespace fs = std::filesystem; + +// The bundled `mcpp` build module — a typed API over the stdout wire protocol +// so build.mcpp can `import mcpp;` instead of `#include`. Its own I/O uses +// C-level primitives in the global module fragment, so the module itself +// needs no std BMI and stays buildable before one exists. (That was once also +// a limit on build.mcpp; it no longer is — a build.mcpp may `import std;` and +// the engine stages the same std module the main build uses.) +// The functions mirror the directive set 1:1; they just print the +// `mcpp:` lines the engine already parses. Embedded in the binary (not shipped as +// a file) so it always matches this mcpp's protocol. +// NOTE: the module declaration line uses a `@MODULE@` placeholder (substituted +// with `export module` when written) so mcpp's own line-based module scanner does +// not mistake this embedded string for build_program.cppm exporting a 2nd module. +inline constexpr std::string_view kMcppModuleSource = R"CPP(module; +#include +#include +@MODULE@ mcpp; +export namespace mcpp { +inline void cxxflag(const char* flag) { std::printf("mcpp:cxxflag=%s\n", flag); } +inline void cflag(const char* flag) { std::printf("mcpp:cflag=%s\n", flag); } +inline void link_lib(const char* name) { std::printf("mcpp:link-lib=%s\n", name); } +inline void link_search(const char* dir) { std::printf("mcpp:link-search=%s\n", dir); } +inline void define(const char* name) { std::printf("mcpp:cfg=%s\n", name); } +inline void generated(const char* path) { std::printf("mcpp:generated=%s\n", path); } +inline void source(const char* path) { std::printf("mcpp:source=%s\n", path); } +inline void include_dir(const char* dir) { std::printf("mcpp:include-dir=%s\n", dir); } +inline void include_dir_after(const char* dir) { std::printf("mcpp:include-dir-after=%s\n", dir); } +inline void rerun_if_changed(const char* path) { std::printf("mcpp:rerun-if-changed=%s\n", path); } +inline void rerun_if_env_changed(const char* var) { std::printf("mcpp:rerun-if-env-changed=%s\n", var); } +// ── environment contract (read side; values injected by the engine) ───── +inline const char* env_or(const char* n) { const char* v = std::getenv(n); return v ? v : ""; } +inline const char* target() { return env_or("MCPP_TARGET"); } +inline const char* target_os() { return env_or("MCPP_TARGET_OS"); } +inline const char* target_arch() { return env_or("MCPP_TARGET_ARCH"); } +inline const char* target_env() { return env_or("MCPP_TARGET_ENV"); } +inline const char* host() { return env_or("MCPP_HOST"); } +inline const char* profile() { return env_or("MCPP_PROFILE"); } +inline const char* out_dir() { return env_or("MCPP_OUT_DIR"); } +inline const char* manifest_dir() { return env_or("MCPP_MANIFEST_DIR"); } +inline bool has_feature(const char* name) { + char buf[256] = "MCPP_FEATURE_"; + unsigned long o = 13; + for (const char* p = name; *p && o + 1 < sizeof buf; ++p, ++o) { + char c = *p; + buf[o] = (c >= 'a' && c <= 'z') ? char(c - 'a' + 'A') + : ((c >= 'A' && c <= 'Z') || (c >= '0' && c <= '9')) ? c : '_'; + } + buf[o] = 0; + return std::getenv(buf) != nullptr; +} +// mcpp#241: resolved install dir of a declared dependency (by its package +// name), or "" if not found. Same sanitize as has_feature; wraps +// MCPP_DEP__DIR. +inline const char* dep_dir(const char* name) { + char buf[256] = "MCPP_DEP_"; + unsigned long o = 9; + for (const char* p = name; *p && o + 5 < sizeof buf; ++p, ++o) { + char c = *p; + buf[o] = (c >= 'a' && c <= 'z') ? char(c - 'a' + 'A') + : ((c >= 'A' && c <= 'Z') || (c >= '0' && c <= '9')) ? c : '_'; + } + buf[o++] = '_'; buf[o++] = 'D'; buf[o++] = 'I'; buf[o++] = 'R'; buf[o] = 0; + return env_or(buf); +} +} +)CPP"; + +// Compile the bundled `mcpp` module into `bdir` and return the extra flags the +// build.mcpp compile needs to import it (the object `mcpp.o` is linked alongside). +// GCC : -fmodules → gcm.cache/mcpp.gcm + mcpp.o; build.mcpp compiles from +// `bdir` (cwd) so GCC finds gcm.cache/mcpp.gcm. +// Clang : --precompile → mcpp.pcm, then -c → mcpp.o; pass -fmodule-file=mcpp=. +// Does the source contain `import ;`? +// +// A plain substring search is not enough here: "import std" is a prefix of +// "import std.compat", so the naive test reports both for a program that +// only imports the latter, and mcpp would build a std BMI nobody asked for. +// Match the whole module name and require the terminating `;`, tolerating +// the whitespace the grammar allows. Occurrences inside comments or string +// literals still match — over-detection costs one cached BMI lookup, never +// a wrong build, and that is the same trade the `import mcpp` check has +// always made. +bool imports_module(std::string_view src, std::string_view name) { + constexpr std::string_view kImport = "import"; + std::size_t pos = 0; + while ((pos = src.find(kImport, pos)) != std::string_view::npos) { + std::size_t i = pos + kImport.size(); + // `importfoo` is not an import. + if (i >= src.size() || (src[i] != ' ' && src[i] != '\t')) { ++pos; continue; } + while (i < src.size() && (src[i] == ' ' || src[i] == '\t')) ++i; + if (src.compare(i, name.size(), name) == 0) { + std::size_t j = i + name.size(); + while (j < src.size() && (src[j] == ' ' || src[j] == '\t')) ++j; + if (j < src.size() && src[j] == ';') return true; + } + ++pos; + } + return false; +} + + +// What the bundled `mcpp` module contributes to the build.mcpp compile. +struct McppModule { + std::vector useFlags; // how the consumer names the BMI + fs::path object; // linked alongside build.mcpp +}; + +std::expected +build_mcpp_module(const fs::path& bdir, const fs::path& compiler, + const std::vector& base, const std::string& stdFlag, + const mcpp::toolchain::Toolchain& tc, + const std::vector>& env) { + std::error_code ec; + fs::path cppm = bdir / "mcpp.cppm"; + std::string moduleSrc(kMcppModuleSource); + if (auto p = moduleSrc.find("@MODULE@"); p != std::string::npos) + moduleSrc.replace(p, std::string_view("@MODULE@").size(), "export module"); + { std::ofstream os(cppm, std::ios::trunc); + os << moduleSrc; + if (!os) return std::unexpected(std::string("could not write mcpp module source")); } + + auto run = [&](std::vector argv, const char* what) + -> std::expected { + auto r = mcpp::platform::process::capture_exec(argv, env, bdir.string()); + if (r.exit_code != 0) + return std::unexpected(std::format("mcpp module {} failed (exit {}):\n{}", + what, r.exit_code, r.output)); + return {}; + }; + auto with_base = [&](std::vector head) { + for (auto& b : base) head.push_back(b); + return head; + }; + + // Dispatch on the SAME module table the main build uses (BmiTraits + + // CommandDialect), not on a local is_clang/else. That is what makes a + // toolchain family work here as soon as it works there — adding cl.exe + // needed no new pipeline, only this row. + const auto traits = mcpp::toolchain::bmi_traits(tc); + const auto& dial = mcpp::toolchain::dialect_for(tc); + McppModule out; + + if (tc.compiler == mcpp::toolchain::CompilerId::MSVC) { + // cl produces the .ifc and the .obj in one step. + fs::path ifc = bdir / ("mcpp" + std::string(traits.bmiExt)); + out.object = bdir / ("mcpp" + std::string(dial.objExt)); + std::vector argv{compiler.string()}; + for (auto f : dial.alwaysFlagsArgv) argv.emplace_back(f); + argv.push_back(stdFlag); + argv.push_back("/interface"); + for (auto f : dial.forceCxxLangArgv) argv.emplace_back(f); + argv.push_back(dial.compileOnly == std::string_view("/c") ? "/c" : "-c"); + argv.push_back("mcpp.cppm"); + argv.push_back("/ifcOutput"); argv.push_back(ifc.string()); + argv.push_back(std::string(dial.outputObjPrefix) + out.object.string()); + if (auto r = run(with_base(std::move(argv)), "compile"); !r) + return std::unexpected(r.error()); + out.useFlags = mcpp::toolchain::bmi_reference_tokens(" /reference mcpp=", ifc); + return out; + } + + out.object = bdir / ("mcpp" + std::string(dial.objExt)); + if (mcpp::toolchain::is_clang(tc)) { + fs::path pcm = bdir / ("mcpp" + std::string(traits.bmiExt)); + if (auto r = run(with_base({compiler.string(), stdFlag, "--precompile", + "mcpp.cppm", "-o", pcm.string()}), "precompile"); !r) + return std::unexpected(r.error()); + if (auto r = run(with_base({compiler.string(), stdFlag, "-c", + pcm.string(), "-o", out.object.string()}), "object"); !r) + return std::unexpected(r.error()); + out.useFlags = mcpp::toolchain::bmi_reference_tokens("-fmodule-file=mcpp=", pcm); + return out; + } + + // GCC: BMIs are implicit under /gcm.cache, so nothing to name. + if (auto r = run(with_base({compiler.string(), stdFlag, + std::string(mcpp::toolchain::bmi_traits(tc).compileModulesFlag).empty() + ? "-fmodules" : "-fmodules", + "-c", "mcpp.cppm", "-o", out.object.string()}), "compile"); !r) + return std::unexpected(r.error()); + out.useFlags = {"-fmodules"}; + return out; +} + + +} // namespace mcpp::build diff --git a/src/toolchain/dialect.cppm b/src/toolchain/dialect.cppm index 41c05345..a550b2cd 100644 --- a/src/toolchain/dialect.cppm +++ b/src/toolchain/dialect.cppm @@ -54,6 +54,14 @@ struct CommandDialect { // better off without (see the note on `alwaysFlagsArgv`). std::string_view forceCxxLang; // "-x c++" | "/TP" std::span forceCxxLangArgv; + // Per-FILE language force, for a command line that also carries object + // files. The two drivers differ structurally, not just in spelling: GNU's + // `-x c++` is positional and stays in effect until `-x none`, while + // cl.exe's `/TP` applies to EVERY input — so an object listed after it is + // fed to the C++ frontend and dies with C2018. cl's per-file form is + // `/Tp`; GNU has none, and uses the positional pair plus a reset. + // Empty means "no per-file form — use forceCxxLangArgv and reset after". + std::string_view perFileCxxPrefix; // "" | "/Tp" // Static CRT / runtime. On MSVC this is a compile-time CRT model, not a // link mode — there is no /MT equivalent of `-static` for the whole image. std::string_view staticRuntime; // "-static"| "/MT" @@ -127,6 +135,7 @@ constexpr CommandDialect kGnuDialect{ .libSearchPrefix = "-L", .forceCxxLang = "-x c++", .forceCxxLangArgv = kGnuForceCxxArgv, + .perFileCxxPrefix = "", .staticRuntime = "-static", .outputExePrefix = "-o ", .objExt = ".o", @@ -154,6 +163,7 @@ constexpr CommandDialect kMsvcDialect{ .libSearchPrefix = "/LIBPATH:", .forceCxxLang = "/TP", .forceCxxLangArgv = kMsvcForceCxxArgv, + .perFileCxxPrefix = "/Tp", .staticRuntime = "/MT", .outputExePrefix = "/Fe:", .objExt = ".obj", diff --git a/src/toolchain/fingerprint.cppm b/src/toolchain/fingerprint.cppm index 6ea15d3c..6f74b8a0 100644 --- a/src/toolchain/fingerprint.cppm +++ b/src/toolchain/fingerprint.cppm @@ -18,7 +18,7 @@ import mcpp.toolchain.detect; export namespace mcpp::toolchain { -inline constexpr std::string_view MCPP_VERSION = "2026.8.2.1"; +inline constexpr std::string_view MCPP_VERSION = "2026.8.2.2"; struct FingerprintInputs { Toolchain toolchain; diff --git a/src/toolchain/hostflags.cppm b/src/toolchain/hostflags.cppm new file mode 100644 index 00000000..174a3c30 --- /dev/null +++ b/src/toolchain/hostflags.cppm @@ -0,0 +1,208 @@ +// mcpp.toolchain.hostflags — the single producer of host-compile flags. +// +// One assembly, three consumers: +// flags.cppm → rendered with ninja `$` escaping into build.ninja +// stdmod.cppm → rendered with shell quoting into a std-module command +// build_program.cppm → used as argv tokens directly (build.mcpp execs, no shell) +// +// Before this module those three hand-wrote the same thing. The resolvers +// (linkmodel, clang driver model) were already shared; the ASSEMBLY was not, +// because the seam only produced strings and the argv consumer could not use +// it. Every bug in mcpp#331/PR#332's batch was an instance of that split — +// flags.cppm knew about quoting / the macOS deployment target / the MSVC +// dialect and the other two did not — and a 0.0.9x fix had already corrected +// the same file once for the same reason (musl→static re-derived). +// +// See .agents/docs/2026-08-02-host-compile-single-producer-design.md. +// +// Deliberately its own module rather than a helper inside build_program.cppm: +// that file's anonymous namespace has demonstrated (PR#332, clang 22.1.8 + +// C++20 modules + -O2) that adding a function to it can miscompile a +// NEIGHBOURING function — an unused `split_ws` was enough to corrupt a local +// vector in `contract_env`. Mechanism unknown, reproduction solid; the cheap +// response is to not grow that namespace. Design §6.2. + +export module mcpp.toolchain.hostflags; + +import std; +import mcpp.platform; +import mcpp.toolchain.model; +import mcpp.toolchain.linkmodel; +import mcpp.toolchain.registry; + +export namespace mcpp::toolchain { + +// The knobs below exist because the three consumers genuinely differ TODAY. +// Each one is a documented divergence, not a switch to preserve an accident: +// consolidating without them would silently change behaviour, and dropping +// the reasons would leave the next reader unable to tell which is which. +struct HostFlagOptions { + // Whether to bypass clang's bundled `.cfg`. + // + // Always — the main build and the std module. The cfg is an + // install-time-generated, non-reproducible artifact, so + // everything it would provide is spelled out explicitly. + // LinuxOnly — the build.mcpp host helper. On macOS/Windows it keeps + // TRUSTING the cfg, because the macOS link additionally + // needs the libc++abi/unwind handling that the main build's + // needs_explicit_libcxx path owns; duplicating that for a + // host compile produced undefined __cxa_* / + // __gxx_personality_v0 (build_program.cppm, pre-existing). + enum class CfgBypass { Always, LinuxOnly }; + CfgBypass cfgBypass = CfgBypass::Always; + + // binutils `-B` so the driver finds as/ld. A GCC/libstdc++ payload + // concern only: musl and MinGW-w64 bundle their own, and Clang/MSVC never + // take an external binutils. MinGW must NOT get the Linux binutils — its + // PE/SEH output is only assemblable by x86_64-w64-mingw32-as. + bool binutilsPrefix = false; + + // `-L` (plus `-Wl,-rpath` where the format has one) for the toolchain's + // own runtime dirs, so the produced program can load private libs in + // tree. The main build routes these through depRuntimeLibraryDirs + // instead, so it leaves this off. + bool runtimeLibDirs = false; + + // Emit `-stdlib=libc++` alongside the cfg bypass. + // + // ClangDriverModel deliberately leaves this to callers: flags.cppm's + // string feeds C compiles too, and a C command must not carry it. The std + // module build has no such constraint — it compiles exactly one C++ TU — + // and states the stdlib selection explicitly. + bool clangStdlibSelect = false; + + // Resolved value from platform::macos::deployment_target(); empty = omit. + // Must agree across the std BMI and everything that imports it — clang + // rejects a module built for a different deployment target outright. + std::string macosDeploymentTarget; +}; + +// Host-compile flags as argv tokens, in the order the string channels have +// always emitted them (clang cfg → deployment target → C library), so +// rendering reproduces today's command lines byte for byte. +std::vector host_compile_tokens(const Toolchain& tc, + const HostFlagOptions& opt, + const PathEscape& esc); + +// Link-side tokens for a driver invocation that compiles AND links a host +// program in one step — which is what build.mcpp is. The main build keeps its +// own link assembly (it links target artifacts under a different linkage +// policy); this exists so the one-shot host case has a producer at all +// instead of hand-writing one. +std::vector host_link_tokens(const Toolchain& tc, + const HostFlagOptions& opt, + const PathEscape& esc); + +// A "use this BMI" flag as argv tokens. +// +// BmiTraits stores these for the ninja string channel, where the shape does +// not matter: `-fmodule-file=std=

` is one word but `/reference std=

` is +// two, and a string consumer never has to know. An argv consumer does — one +// element containing a space is a single argument with a space in it, which +// cl.exe rejects. Split at the prefix's last space, the same rule the ninja +// side's quoting uses. +std::vector bmi_reference_tokens(std::string_view usePrefix, + const std::filesystem::path& bmi); + +} // namespace mcpp::toolchain + +namespace mcpp::toolchain { + +std::vector host_compile_tokens(const Toolchain& tc, + const HostFlagOptions& opt, + const PathEscape& esc) { + std::vector out; + + // MSVC carries none of this on the command line: cl.exe and link.exe find + // headers and import libraries through INCLUDE / LIB, which detection + // synthesizes into tc.envOverrides. Emitting the GNU shapes below would + // produce a string of unknown options and then LNK1181. + if (tc.compiler == CompilerId::MSVC) return out; + + const auto dm = resolve_clang_driver(tc); + const auto lm = resolve_link_model(tc); + + const bool bypassCfg = + dm.hasCfg && (opt.cfgBypass == HostFlagOptions::CfgBypass::Always + || mcpp::platform::is_linux); + + // Trusting the cfg means contributing no include paths, stdlib selection + // or runtime choices — it already carries them. It does NOT mean + // contributing nothing: the deployment target still has to be stated (see + // below), which is why this suppresses the two blocks rather than + // returning early. + const bool trustCfg = !bypassCfg && dm.hasCfg; + + if (bypassCfg) { + for (auto& t : dm.compile_tokens(esc, opt.clangStdlibSelect)) + out.push_back(t); + } + + // Unconditional on macOS, cfg or no cfg. clang refuses to load a module + // built for a different deployment target, and this result feeds every + // compile that touches one — the bundled mcpp module's precompile, its + // object step, and the build.mcpp compile. Skipping it on the trust-cfg + // path is exactly the mismatch e2e 181 catches: the std BMI is built for + // 14.0 while the TU importing it is not. + if (mcpp::platform::is_macos && !opt.macosDeploymentTarget.empty()) + out.push_back("-mmacosx-version-min=" + opt.macosDeploymentTarget); + + if (!trustCfg && (bypassCfg || lm.mode != CLibMode::None)) + for (auto& t : lm.compile_tokens(esc)) out.push_back(t); + + return out; +} + +std::vector bmi_reference_tokens(std::string_view usePrefix, + const std::filesystem::path& bmi) { + std::string_view p = usePrefix; + while (!p.empty() && p.front() == ' ') p.remove_prefix(1); + if (p.empty()) return {}; + auto sp = p.find_last_of(' '); + if (sp == std::string_view::npos) + return { std::string(p) + bmi.string() }; + return { std::string(p.substr(0, sp)), + std::string(p.substr(sp + 1)) + bmi.string() }; +} + +std::vector host_link_tokens(const Toolchain& tc, + const HostFlagOptions& opt, + const PathEscape& esc) { + std::vector out; + if (tc.compiler == CompilerId::MSVC) return out; + + const auto dm = resolve_clang_driver(tc); + const auto lm = resolve_link_model(tc); + + const bool bypassCfg = + dm.hasCfg && (opt.cfgBypass == HostFlagOptions::CfgBypass::Always + || mcpp::platform::is_linux); + + if (bypassCfg) { + for (auto& t : dm.link_tokens(esc)) out.push_back(t); + } else if (dm.hasCfg) { + return out; // trusting the cfg: it already selects the linker/runtimes + } + + for (auto& t : lm.link_tokens(esc)) out.push_back(t); + + if (opt.binutilsPrefix) { + if (auto ar = archive_tool(tc); !ar.empty()) + out.push_back("-B" + esc(ar.parent_path())); + } + + if (opt.runtimeLibDirs) { + // -L is link-time and wanted everywhere; rpath is an ELF-only concept. + // A PE target reaches here too, where the flag is inert and + // self-containment comes from the static link instead (#299). + for (auto& d : tc.linkRuntimeDirs) { + out.push_back("-L" + esc(d)); + if constexpr (mcpp::platform::supports_rpath) + out.push_back("-Wl,-rpath," + esc(d)); + } + } + + return out; +} + +} // namespace mcpp::toolchain diff --git a/src/toolchain/linkmodel.cppm b/src/toolchain/linkmodel.cppm index 97366ad5..45195abc 100644 --- a/src/toolchain/linkmodel.cppm +++ b/src/toolchain/linkmodel.cppm @@ -35,6 +35,26 @@ enum class CLibMode { // (identity) is only safe for paths already known to be quote-free. using PathEscape = std::function; +// Identity escape — for the argv consumers, which hand tokens straight to +// exec and must NOT carry ninja `$` escapes or shell quotes. +inline std::string no_escape(const std::filesystem::path& p) { return p.string(); } + +// Render argv tokens as the leading-space-separated string the ninja and +// shell channels want. +// +// Tokens are the source of truth and the string is derived, not the other way +// round. Historically only the string form existed, so the one consumer that +// needs argv (build.mcpp, which execs directly with no shell) could not use +// this seam at all and hand-wrote the whole assembly a second time — the +// duplication that 2026-08-02-host-compile-single-producer-design.md exists to +// remove. Splitting a rendered string back into tokens is not a substitute: +// it cannot tell a space *inside* a token from a separator. +inline std::string render_tokens(const std::vector& tokens) { + std::string out; + for (auto const& t : tokens) { out += ' '; out += t; } + return out; +} + struct ToolchainLinkModel { CLibMode mode = CLibMode::None; @@ -56,43 +76,53 @@ struct ToolchainLinkModel { // gcc: -idirafter (…#include_next), -B/-L only bool clangWithCfg = false; // sibling .cfg exists (bundled LLVM) - // Render the compile-side flags (leading-space separated, matching the - // historical assembly style of flags.cppm/stdmod.cppm). - std::string compile_flags(const PathEscape& esc) const { - std::string out; + // Compile-side flags as argv tokens. Each entry is ONE argv word. + std::vector compile_tokens(const PathEscape& esc) const { + std::vector out; if (mode == CLibMode::Sysroot) - out += " --sysroot=" + esc(sysroot); + out.push_back("--sysroot=" + esc(sysroot)); // PayloadFirst headers: clang takes -isystem; GCC needs -idirafter so // libstdc++'s #include_next wrappers (which only search *after* the // current dir, and GCC's built-ins are last) can still reach libc. // A Sysroot-mode supplement (kernel headers missing from the sysroot) // is -isystem for both: the libc headers come from the sysroot there. const char* incFlag = (mode == CLibMode::Sysroot || clangDriver) - ? " -isystem" : " -idirafter"; + ? "-isystem" : "-idirafter"; for (auto& inc : systemIncludes) - out += incFlag + esc(inc); + out.push_back(incFlag + esc(inc)); return out; } - // Render the link-side flags. `-B` is the CRT-discovery fix for #195: + // The string channel, derived from the tokens above (leading-space + // separated, matching the historical assembly style of + // flags.cppm/stdmod.cppm byte for byte). + std::string compile_flags(const PathEscape& esc) const { + return render_tokens(compile_tokens(esc)); + } + + // Link-side flags as argv tokens. `-B` is the CRT-discovery fix for #195: // the driver resolves crt objects through -B prefixes and sysroot paths, // never through -L. - std::string link_flags(const PathEscape& esc) const { - std::string out; + std::vector link_tokens(const PathEscape& esc) const { + std::vector out; if (mode == CLibMode::Sysroot) { - out += " --sysroot=" + esc(sysroot); + out.push_back("--sysroot=" + esc(sysroot)); return out; } if (mode != CLibMode::PayloadFirst) return out; - if (!crtDir.empty()) out += " -B" + esc(crtDir); + if (!crtDir.empty()) out.push_back("-B" + esc(crtDir)); for (auto& dir : libDirs) { - out += " -L" + esc(dir); - if (clangDriver) out += " -Wl,-rpath," + esc(dir); + out.push_back("-L" + esc(dir)); + if (clangDriver) out.push_back("-Wl,-rpath," + esc(dir)); } if (clangDriver && !loader.empty()) - out += " -Wl,--dynamic-linker=" + esc(loader); + out.push_back("-Wl,--dynamic-linker=" + esc(loader)); return out; } + + std::string link_flags(const PathEscape& esc) const { + return render_tokens(link_tokens(esc)); + } }; // Clang cfg-bypass driver model: everything a consumer needs to emit so that @@ -105,18 +135,43 @@ struct ClangDriverModel { std::vector cxxIncludes; // libc++ header dirs std::vector libDirs; // libc++/compiler-rt libs - // " --no-default-config -nostdinc++ -isystem<...>" (compile side). - // -stdlib=libc++ is deliberately left to callers: compile commands for - // C files must not carry it. - std::string compile_flags(const PathEscape& esc) const { - std::string out = " --no-default-config -nostdinc++"; - for (auto& inc : cxxIncludes) out += " -isystem" + esc(inc); + // "--no-default-config" "-nostdinc++" "-isystem<...>" (compile side), as + // argv tokens. -stdlib=libc++ is deliberately left to callers: compile + // commands for C files must not carry it. + // `stdlibSelect` inserts `-stdlib=libc++` immediately after `-nostdinc++`, + // where the std module build has always put it. Position is not free + // here: the rendered string is part of the std cache identity, so moving + // the flag would invalidate every user's std BMIs for no behavioural gain. + std::vector compile_tokens(const PathEscape& esc, + bool stdlibSelect = false) const { + std::vector out{"--no-default-config", "-nostdinc++"}; + if (stdlibSelect) out.push_back("-stdlib=libc++"); + for (auto& inc : cxxIncludes) out.push_back("-isystem" + esc(inc)); return out; } + std::string compile_flags(const PathEscape& esc) const { + return render_tokens(compile_tokens(esc)); + } + // Link-side driver selection, matching the cfg xlings generates. static constexpr std::string_view kLinkDriverFlags = " -stdlib=libc++ -fuse-ld=lld --rtlib=compiler-rt --unwindlib=libunwind"; + + // Same, as argv tokens, WITHOUT `-stdlib=libc++`: a driver invocation that + // both compiles and links (build.mcpp) already carries it on the compile + // side via HostFlagOptions::clangStdlibSelect, and repeating it is noise. + // Plus the libc++/compiler-rt library dirs, which a self-contained host + // helper needs to both link against and find at run time. + std::vector link_tokens(const PathEscape& esc) const { + std::vector out{"-fuse-ld=lld", "--rtlib=compiler-rt", + "--unwindlib=libunwind"}; + for (auto& d : libDirs) { + out.push_back("-L" + esc(d)); + out.push_back("-Wl,-rpath," + esc(d)); + } + return out; + } }; // ── loader resolution: data over hardcodes ─────────────────────────────── diff --git a/src/toolchain/stdmod.cppm b/src/toolchain/stdmod.cppm index 3dabf39c..47e3878e 100644 --- a/src/toolchain/stdmod.cppm +++ b/src/toolchain/stdmod.cppm @@ -38,6 +38,7 @@ import mcpp.toolchain.clang; import mcpp.toolchain.detect; import mcpp.toolchain.fingerprint; import mcpp.toolchain.gcc; +import mcpp.toolchain.hostflags; import mcpp.toolchain.linkmodel; import mcpp.toolchain.msvc; @@ -230,23 +231,27 @@ std::expected ensure_built( // identical flags also keep the std_build_commands cache key honest). // Std module precompilation only needs compile flags (no linker flags), // so --no-default-config is safe here on all platforms. - const auto dm = resolve_clang_driver(tc); - const auto lm = resolve_link_model(tc); const PathEscape shellEsc = [](const std::filesystem::path& p) { return std::format("'{}'", p.string()); }; - std::string sysroot_flag; - if (dm.hasCfg) { - sysroot_flag = " --no-default-config -nostdinc++ -stdlib=libc++"; - for (auto& inc : dm.cxxIncludes) - sysroot_flag += " -isystem" + shellEsc(inc); - sysroot_flag += lm.compile_flags(shellEsc); - } else { - sysroot_flag = lm.compile_flags(shellEsc); - } - - // Deployment target must mirror what flags.cppm emits for normal TUs - // (single resolver: platform::macos::deployment_target). + // The shared producer (mcpp.toolchain.hostflags) — the same assembly + // flags.cppm and the build.mcpp host compile use. This block used to + // hand-write the clang cfg bypass, which is how it could drift from the + // other two. + HostFlagOptions hopt; + hopt.cfgBypass = HostFlagOptions::CfgBypass::Always; + hopt.clangStdlibSelect = true; + std::string sysroot_flag = + render_tokens(host_compile_tokens(tc, hopt, shellEsc)); + + // Deployment target appended here rather than passed to the producer + // ONLY to keep this command string byte-identical to what earlier + // releases emitted: the string is part of the std cache identity + // (std_build_commands feeds the cache directory name), so reordering a + // flag would invalidate every user's std BMIs for no behavioural gain. + // The VALUE still comes from the one resolver + // (platform::macos::deployment_target) that flags.cppm and + // build_program.cppm read — only its position is local. if (!macos_deployment_target.empty()) { sysroot_flag += std::format(" -mmacosx-version-min={}", macos_deployment_target); diff --git a/tests/e2e/180_msvc_build_mcpp.sh b/tests/e2e/180_msvc_build_mcpp.sh index 629ce79f..946a2370 100755 --- a/tests/e2e/180_msvc_build_mcpp.sh +++ b/tests/e2e/180_msvc_build_mcpp.sh @@ -98,24 +98,42 @@ run_out=$("$MCPP" run 2>&1) || { echo "FAIL: run (link-lib): $run_out"; exit 1; [[ "$run_out" == *"msvc-link-lib-ok"* ]] \ || { echo "FAIL: run output (link-lib): $run_out"; exit 1; } -# 3) Named modules under cl.exe (.ifc + /reference) are not implemented. That -# must be an explicit refusal, not an obscure compiler error — the whole -# point of the gate is that the user learns what to do instead. +# 3) Named modules under cl.exe. build.mcpp is "one host C++ program", and the +# main build has compiled those with modules under cl.exe for a while +# (e2e 99 produces real .ifc artifacts) — so build.mcpp must too. It used to +# refuse, because it hand-rolled its own compile path instead of reading the +# shared BmiTraits/CommandDialect rows. cat > build.mcpp <<'EOF' import std; +import mcpp; int main() { - std::println("mcpp:cfg=SHOULD_NOT_GET_HERE"); + std::string tag = std::format("MSVC_MODULES_{}", 1 + 1); + mcpp::define(tag.c_str()); + mcpp::rerun_if_changed("build.mcpp"); return 0; } EOF -set +e -mod_out=$("$MCPP" build 2>&1) -mod_rc=$? -set -e -[[ $mod_rc -ne 0 ]] || { - echo "FAIL: import std in build.mcpp unexpectedly succeeded under MSVC"; exit 1; } -echo "$mod_out" | grep -qi "not yet supported under MSVC" || { - echo "FAIL: no explicit unsupported diagnostic:"; echo "$mod_out"; exit 1; } +cat > src/main.cpp <<'EOF' +import std; +int main() { +#ifdef MSVC_MODULES_2 + std::println("msvc-modules-ok"); + return 0; +#else + std::println("define missing"); + return 1; +#endif +} +EOF + +out=$("$MCPP" build 2>&1) || { echo "FAIL: msvc build.mcpp with modules: $out"; exit 1; } +run_out=$("$MCPP" run 2>&1) || { echo "FAIL: run (modules): $run_out"; exit 1; } +[[ "$run_out" == *"msvc-modules-ok"* ]] \ + || { echo "FAIL: run output (modules): $run_out"; exit 1; } + +# The .ifc really came from the msvc module pipeline, not a silent fallback. +find target/.build-mcpp -name "*.ifc" | grep -q . \ + || { echo "FAIL: no .ifc produced for the bundled mcpp module"; exit 1; } -echo "PASS: MSVC build.mcpp — include path, link-lib translation, module refusal" +echo "PASS: MSVC build.mcpp — include path, link-lib translation, named modules" diff --git a/tests/unit/test_hostflags.cpp b/tests/unit/test_hostflags.cpp new file mode 100644 index 00000000..e42a34e2 --- /dev/null +++ b/tests/unit/test_hostflags.cpp @@ -0,0 +1,218 @@ +#include + +import std; +import mcpp.platform; +import mcpp.toolchain.dialect; +import mcpp.toolchain.hostflags; +import mcpp.toolchain.linkmodel; +import mcpp.toolchain.model; + +using mcpp::toolchain::CompilerId; +using mcpp::toolchain::HostFlagOptions; + +namespace { + +mcpp::toolchain::Toolchain tc_for(CompilerId id) { + mcpp::toolchain::Toolchain tc; + tc.compiler = id; + tc.targetTriple = id == CompilerId::MSVC ? "x86_64-pc-windows-msvc" + : "x86_64-linux-gnu"; + return tc; +} + +// Every compiler family mcpp claims to support, so a new one cannot be added +// without being answered for here. +constexpr CompilerId kFamilies[] = { + CompilerId::GCC, CompilerId::Clang, CompilerId::MSVC, +}; + +} // namespace + +// ── The capability-parity guard ───────────────────────────────────────────── +// +// build.mcpp is "one host C++ program", and compiling one of those is mcpp's +// job — so it must not have a narrower capability list than the main build. +// Stating that only in prose is what let `import mcpp;` / `import std;` sit +// behind a "not yet supported under MSVC" gate long after the main build had +// the .ifc pipeline (e2e 99 was already producing .ifc artifacts). This test +// turns the principle into a compile-and-run check: a family that is missing +// a dialect row, a module row, or host flags fails HERE, not at a user's +// `mcpp build`. +TEST(HostFlags, EveryFamilyHasCompleteTables) { + for (auto id : kFamilies) { + auto tc = tc_for(id); + const auto& d = mcpp::toolchain::dialect_for(tc); + const auto t = mcpp::toolchain::bmi_traits(tc); + + EXPECT_FALSE(d.id.empty()); + // Needed to compile a `.mcpp` at all: the extension is unknown to + // every driver, so the language has to be forced. + EXPECT_FALSE(d.forceCxxLangArgv.empty()) << d.id; + // Needed to name the produced program. + EXPECT_FALSE(d.outputExePrefix.empty()) << d.id; + EXPECT_FALSE(d.objExt.empty()) << d.id; + // Needed to build and reference the bundled `mcpp` module. + EXPECT_FALSE(t.bmiDir.empty()) << d.id; + EXPECT_FALSE(t.bmiExt.empty()) << d.id; + } +} + +// The producer must answer for every family too — MSVC deliberately returns +// nothing (cl.exe finds headers and libs through INCLUDE/LIB, not argv), but +// it must be a decision, not a crash or an accident. +TEST(HostFlags, ProducerAnswersForEveryFamily) { + HostFlagOptions opt; + for (auto id : kFamilies) { + auto tc = tc_for(id); + auto compile = mcpp::toolchain::host_compile_tokens( + tc, opt, mcpp::toolchain::no_escape); + auto link = mcpp::toolchain::host_link_tokens( + tc, opt, mcpp::toolchain::no_escape); + if (id == CompilerId::MSVC) { + EXPECT_TRUE(compile.empty()); + EXPECT_TRUE(link.empty()); + } + // No empty tokens anywhere: an empty argv element is an argument the + // driver still has to interpret. + for (auto const& t : compile) EXPECT_FALSE(t.empty()); + for (auto const& t : link) EXPECT_FALSE(t.empty()); + } +} + +// ── The rendered string must not move ─────────────────────────────────────── +// +// stdmod folds its compile command into `std_build_commands`, and the std +// cache DIRECTORY NAME is derived from the metadata containing it. Reordering +// a flag therefore invalidates every user's std BMIs for no behavioural gain +// — so the exact spelling is a compatibility surface, not an implementation +// detail. This pins it; the first attempt at this refactor moved +// `-stdlib=libc++` after the include flags and would have shipped exactly +// that invalidation. +TEST(HostFlags, ClangCfgBypassStringIsStable) { + mcpp::toolchain::ClangDriverModel dm; + dm.hasCfg = true; + dm.cxxIncludes = { "/llvm/include/c++/v1", "/llvm/include/tgt/c++/v1" }; + + EXPECT_EQ(dm.compile_flags(mcpp::toolchain::no_escape), + " --no-default-config -nostdinc++" + " -isystem/llvm/include/c++/v1" + " -isystem/llvm/include/tgt/c++/v1"); + + // With the stdlib selection the std module has always asked for, it lands + // immediately after -nostdinc++ — not at the end. + EXPECT_EQ(mcpp::toolchain::render_tokens( + dm.compile_tokens(mcpp::toolchain::no_escape, true)), + " --no-default-config -nostdinc++ -stdlib=libc++" + " -isystem/llvm/include/c++/v1" + " -isystem/llvm/include/tgt/c++/v1"); +} + +TEST(HostFlags, LinkModelStringsAreStable) { + mcpp::toolchain::ToolchainLinkModel lm; + lm.mode = mcpp::toolchain::CLibMode::PayloadFirst; + lm.clangDriver = true; + lm.crtDir = "/glibc/lib"; + lm.libDirs = { "/glibc/lib" }; + lm.loader = "/glibc/lib/ld.so"; + lm.systemIncludes = { "/glibc/include" }; + + EXPECT_EQ(lm.compile_flags(mcpp::toolchain::no_escape), + " -isystem/glibc/include"); + EXPECT_EQ(lm.link_flags(mcpp::toolchain::no_escape), + " -B/glibc/lib -L/glibc/lib -Wl,-rpath,/glibc/lib" + " -Wl,--dynamic-linker=/glibc/lib/ld.so"); + + // GCC takes -idirafter so libstdc++'s #include_next wrappers can still + // reach libc. + lm.clangDriver = false; + EXPECT_EQ(lm.compile_flags(mcpp::toolchain::no_escape), + " -idirafter/glibc/include"); +} + +// ── bmi_reference_tokens ──────────────────────────────────────────────────── +// +// The traits store these for the ninja STRING channel, where one word vs two +// makes no difference. argv consumers cannot be that relaxed. +TEST(HostFlags, BmiReferenceSplitsOnlyWhenTheSpellingHasASpace) { + auto gnu = mcpp::toolchain::bmi_reference_tokens( + " -fmodule-file=std=", std::filesystem::path("/tmp/std.pcm")); + ASSERT_EQ(gnu.size(), 1u); + EXPECT_EQ(gnu[0], "-fmodule-file=std=/tmp/std.pcm"); + + auto msvc = mcpp::toolchain::bmi_reference_tokens( + " /reference std=", std::filesystem::path("/tmp/std.ifc")); + ASSERT_EQ(msvc.size(), 2u); + EXPECT_EQ(msvc[0], "/reference"); + EXPECT_EQ(msvc[1], "std=/tmp/std.ifc"); +} + +// The general invariant, checked for every family: an argv element must never +// contain a space. This bug has now appeared three times in the same shape — +// `-x c++`, the mcpp module reference, the std reference — each time because +// a table entry written for the ninja STRING channel was concatenated into an +// argv element. cl.exe answers with "could not find module 'std'", which +// names neither the flag nor the reason. +TEST(HostFlags, BmiReferencesNeverProduceATokenWithASpace) { + for (auto id : kFamilies) { + auto tc = tc_for(id); + auto t = mcpp::toolchain::bmi_traits(tc); + for (auto prefix : { t.stdBmiUsePrefix, t.stdCompatBmiUsePrefix }) { + for (auto const& tok : mcpp::toolchain::bmi_reference_tokens( + prefix, std::filesystem::path("/tmp/x.bmi"))) { + EXPECT_EQ(tok.find(' '), std::string::npos) + << "family " << mcpp::toolchain::dialect_for(tc).id + << " token: " << tok; + } + } + } +} + +// Same invariant for the language-force spelling, which has both a positional +// and a per-file form. +TEST(HostFlags, LanguageForceTokensNeverContainASpace) { + for (auto const* d : { &mcpp::toolchain::gnu_dialect(), + &mcpp::toolchain::msvc_dialect() }) { + for (auto f : d->forceCxxLangArgv) + EXPECT_EQ(f.find(' '), std::string_view::npos) << d->id << ": " << f; + for (auto f : d->alwaysFlagsArgv) + EXPECT_EQ(f.find(' '), std::string_view::npos) << d->id << ": " << f; + EXPECT_EQ(d->perFileCxxPrefix.find(' '), std::string_view::npos) << d->id; + } +} + +TEST(HostFlags, BmiReferenceIsEmptyForAToolchainThatNamesNothing) { + // GCC finds BMIs implicitly under /gcm.cache — its prefix is empty + // and must not produce a stray token. + EXPECT_TRUE(mcpp::toolchain::bmi_reference_tokens( + "", std::filesystem::path("/tmp/x.gcm")).empty()); +} + +// ── HostFlagOptions divergences ───────────────────────────────────────────── +// +// These knobs encode documented differences between the three consumers. A +// test so that "why is this optional?" has an answer in code, not only prose. +TEST(HostFlags, CfgBypassLinuxOnlyDiffersFromAlwaysOffLinux) { + auto tc = tc_for(CompilerId::Clang); + HostFlagOptions always; always.cfgBypass = HostFlagOptions::CfgBypass::Always; + HostFlagOptions linuxOnly; linuxOnly.cfgBypass = HostFlagOptions::CfgBypass::LinuxOnly; + + // Without a real clang payload there is no cfg to bypass, so both are + // empty here; the assertion that matters is that the option exists and is + // honoured identically on Linux, where the host helper does bypass. + if constexpr (mcpp::platform::is_linux) { + EXPECT_EQ(mcpp::toolchain::host_compile_tokens(tc, always, mcpp::toolchain::no_escape), + mcpp::toolchain::host_compile_tokens(tc, linuxOnly, mcpp::toolchain::no_escape)); + } +} + +TEST(HostFlags, DeploymentTargetOnlyOnMacos) { + auto tc = tc_for(CompilerId::GCC); + HostFlagOptions opt; + opt.macosDeploymentTarget = "14.0"; + auto tokens = mcpp::toolchain::host_compile_tokens( + tc, opt, mcpp::toolchain::no_escape); + bool found = std::ranges::any_of(tokens, [](auto const& t) { + return t.starts_with("-mmacosx-version-min="); + }); + EXPECT_EQ(found, mcpp::platform::is_macos); +}