Uh oh!
There was an error while loading. Please reload this page.
feat: mcpp test capability batch — isolation, parallel, filter, list, timeout, JSON (0.0.104) - #274
Merged
Merged
Conversation
… — nested mcpp no longer segfaults its tools
… checks the disk, not just scanned sources
…leading mid-run Finished banner
…tion, roadmap P1-P6
…ll time 29s -> 3.7s on 49 tests
…N records for tooling
…s, naming exemption note (zh+en)
…lter, list, timeout, json, signal fix)
… with pre-existing scripts)
…antics, documented Windows limitation
Uh oh!
There was an error while loading. Please reload this page.
Sunrisepeak added a commit
that referenced
this pull request
Jul 25, 2026
….0.107 (#281) A shared library that declares `soname` is written as bin/libX11.so but records SONAME libX11.so.6. Both the linker resolving a transitive NEEDED and the loader starting a binary look for the alias, never for the plain name — so the alias is a prerequisite of anything that links the library, not a by-product of building it. It was modelled as a standalone ninja edge that nothing depends on, reachable only through `default`. The 0.0.104 test batch (#274) made `mcpp test` pass explicit goal targets so it could isolate per-test compiles, and that silently stopped producing the alias. `mcpp build` and `mcpp run` drive `default` and were unaffected, which is why this survived three releases: only a project driven by `mcpp test` hits it. The symptom lands three levels from the cause — a consumer failing to link with `libX11.so: undefined reference to xcb_connect` (alongside `libxcb.so.1 ... not found`), or a test binary exiting 127. mcpp-index CI has been failing this way on gui-stack, imgui-module and imgui-window since 0.0.104; two probe PRs pinning only MCPP_VERSION over unmodified descriptors reproduced it at both 0.0.104 and 0.0.106, which is what ruled out the package-identity migration as the cause. Hang the alias off the consumer's implicit inputs, beside the .so it already lists. Correct under `default` and under explicit goals alike, and it pulls in nothing that was not already being built. e2e 64 covered only `build` + `run` — precisely the two paths that still worked. It now also runs `mcpp test` from a clean target dir and asserts the alias exists; without the fix that test binary fails with exit 127. Verified: unit 37/37; e2e 64 red before / green after; the three real mcpp-index members pass with `mcpp test` alone (gui-stack 5/5, imgui-module 1/1, imgui-window 1/1). Remaining local e2e failures (22, 98, 141) reproduce identically on the released 0.0.106 binary under the same MCPP_HOME — they are this machine's gcc 15.1.0, not this change.
Sunrisepeak added a commit
that referenced
this pull request
Jul 25, 2026
… line (0.0.108) (#285) `mcpp test` names every shared prerequisite as a ninja goal so a broken package source fails once, as a package error, instead of N times as identical per-test compile failures. For a large package that is thousands of object paths: FFmpeg's 2281 translation units produced a 50,781-character argv. Windows joins argv into a single command string for cmd.exe, which caps at 8191 characters, so the command never ran at all — the 127 came back from cmd.exe, before ninja or mcpp could print anything. That silence is what made it hard to place. mcpp-index's Windows job has failed on the ffmpeg member since 0.0.104, and the log's last line was an unrelated download progress bar: no "build failed" diagnostic, no ninja output, and target/.build_cache never written. ffmpeg is the only member large enough to trip the limit, and `mcpp build` / `mcpp run` drive `default` with no goal arguments, so both stayed green throughout. Emit the goal set as a phony edge in build.ninja and pass its one-word name. The manifest has no length limit, and ninja still builds the whole set in a single invocation, so parallelism is unchanged. Same origin as the soname-alias regression fixed in 0.0.107: both come from the explicit-goal-targets change in #274 — one design change, two symptoms that only appear under conditions the test suite did not cover. e2e 164 asserts the mechanism (goals live in the manifest) rather than the 8191 limit: reproducing the limit needs a package far too slow to build here, and the limit does not exist on Linux or macOS. It is red on 0.0.107 and green here. Verified: unit 37/37; e2e 155 passed / 1 failed / 5 skipped, the one failure (22_doctor_cache_publish) reproducing identically on the released binary under the same MCPP_HOME. Found by publishing an instrumented Windows build to a throwaway prerelease and pinning it from a stripped-down mcpp-index probe PR — four rounds of black-box probing could not see inside the process.
speak-agent added a commit
that referenced
this pull request
Aug 5, 2026
mcpp-index 的 opencv-module / opencv-module-dnn 在 windows 上编译完 795s / 1166s 之后死在链接: fatal error LNK1170: line in command file contains 135135 or more characters 一条链接边和操作系统之间有**两道**上限,而「改用响应文件」只拆掉了第一道: 1. **命令行** —— Windows CreateProcess 32 KiB;POSIX 下 ninja 用 `sh -c "<整条命令>"`,整条命令是**一个** argv 项,撞的是 MAX_ARG_STRLEN 128 KiB(不是谁都会去查的 2 MiB ARG_MAX)。这道 #344 / PR#345 已经拆掉。 2. **响应文件的单行长度** —— link.exe 上限 128 KiB。`rspfile_content = $in` 让 ninja 把所有对象写成**一行**,于是对象一多就 LNK1170。 改成 `$in_newline`:ninja 用换行分隔。之后**没有任何上限再随对象数增长**。 全平台同一条规则形状,不需要按方言分支:GNU 与 LLVM 的响应文件解析把任何空白 (含换行)当分隔符,而 link.exe / lib.exe 要的正是这种写法。 ── 测试 ───────────────────────────────────────────────────────────────────── 新增 e2e 190,**两面都钉**: - 结构面:生成的规则必须是 `$in_newline`,且不得再出现裸 `$in`; - 可观测面:`ninja -d keeprsp` 保留下来的响应文件必须一行一个对象,且最长行远 低于 128 KiB。 只钉前者的话,ninja 哪天改了 `$in_newline` 的展开方式,测试仍然会绿。 结构断言已在 2026.8.5.2 上验证为红(它那里是 `rspfile_content = $in`)。 链接相关 e2e 全绿:28 / 47 / 86 / 07 / 183 / 190。 ── 同一族的第四次 ────────────────────────────────────────────────────────── #274、#247、#344,再加本条。前三次的教训写的是「一条命令有多长,不该有人放在 心上」;这次补上的是它的孪生兄弟:**一行有多长同样不该**。构建系统不该有一个 「靠崩溃才发现」的工程规模上限。
speak-agent added a commit
that referenced
this pull request
Aug 5, 2026
* fix(link): 命令长度上限从架构上消掉,不再补第八个洞 (2026.8.5.4) mcpp-index 的 opencv-module 在 windows 上 LNK1170 —— 这是同一族缺陷的**第七次**。 架构分析见 .agents/docs/2026-08-06-command-length-architecture.md。 前七次:#247(CreateProcess 32 KiB)、#261 两处(cmd.exe 8191)、#274(argv 50781 字符,失败是裸 127)、#344(POSIX MAX_ARG_STRLEN 128 KiB)、2026.8.5.3(link.exe 响应文件单行 128 KiB)、本次。共同形状: - **发现方式永远是崩溃**,且崩在构建最后一步(本次:编译完 356 秒之后); - **失败不可归因** —— 三种报错谁都不说是哪条边; - **触发者从来不是「写了很长的命令」**,而是无关改动:#344 修缓存正确性、#274 改 错误粒度、本次是**把 CI 的 pin 抬过 2026.8.3.4**。 每次修完都在注释里写「构建系统不该有靠崩溃才发现的规模上限」,然后换个地方再犯。 ── 为什么现在才出现 ────────────────────────────────────────────────────── #344 给每个依赖的对象加了一层包目录(缓存正确性要求),路径因此变长 —— 同一条边 在 linux 上从 56 840 涨到 161 687 字节。而 mcpp-index 的 CI 一直 pin 在 **2026.8.3.3**,正好是那之前一版,windows 腿从没用长路径链接过。抬 pin 才第一次撞到。 ── P1:让长度不再是变量 ───────────────────────────────────────────────── 2026.8.5.3 把 mcpp**自己**写的响应文件改成按行分隔。但 clang 作为 driver 时会 **再生成一个**响应文件转发给链接器,那个是单行的 —— 我们改不到它。所以 windows 上的 clang 链接改用 `-fuse-ld=lld`。 **不是 workaround**:lld 用 LLVM 的 tokenizer 解析响应文件,**没有单行上限**,消掉 的是一整类而不是把数字调大;路径也缩不短(那层包目录正是 #344 需要的);而且 **linux 与 macOS 早就在用 lld**,windows 是唯一还在用系统链接器、也是唯一有单行 上限的平台 —— 这是消除平台不一致。原生 cl.exe 保持 link.exe(那里响应文件是我们 自己写的,2026.8.5.3 已覆盖)。 ── P2:上限进表(新模块 cmdlimits.cppm)────────────────────────────────── 根因是命令构造层对「这条命令要穿过哪些通道、各自上限多少」一无所知,而这份知识 只在注释和 CHANGELOG 里 —— 是「同一决策 N 处推导」的镜像:**一个关键约束在零处 被表达**。 表里除字节数还记**症状**:这一族最贵的从来不是修,是**认出**。 `Argument list too long` / `LNK1170` / 裸 `127` 三种表现毫无共同点。新增通道必须 回答「你的上限是多少」,与 directives::kTable 里「Scope 是必填字段」同一手法。 ── P3:计划期拦截并指名道姓 ──────────────────────────────────────────── 生成完 build.ninja 统一扫描,超限时报出边名/通道/实测字节/上限/解法/文档路径, 且发生在**还没编译任何东西**的时候。 实施中纠正两处判断: - 校验点选「manifest 生成后统一扫描」而非「逐 emit site 插桩」—— 后者在新增一种边 时没人会想起来加,而那正是前七次的漏法; - **phony 必须排除**,否则会误报到 **#274 的修复本身**:它为解决 argv 超限,正是把 几千个目标收进一条 phony 聚合边,而 phony 根本没有 command。走 rspfile 的规则 同样豁免。判据是「rule 有 command 且不走 rspfile」。 ── 测试 ──────────────────────────────────────────────────────────────── 单测 test_cmdlimits 8 条:每个通道都在表里、数字是实测的那些(MAX_ARG_STRLEN 是 32 页,不是谁都会先想到的 2 MiB ARG_MAX)、每条都记了症状与解法、诊断含边名/字节/ 上限/解法/文档路径。 **没做超限 e2e**:P1 之后本地造不出自然超限的边,要造只能人为破坏 rspfile 规则, 那测的是被破坏的代码而不是真实路径。windows 的真实验证由 mcpp-index CI 承担。 全量单测 58/58;mcpp 自身 351 条边零误报。 ── 其他 ──────────────────────────────────────────────────────────────── 内带 xlings 升到 2026.8.5.2,.github/ 下 16 处 pin 由 check_version_pins.sh 校验同步。 (校验脚本当场抓到 7 处不同步,包括带 v 前缀的 5 处;并拦下了我一次误改 —— 全局 替换差点把它自己注释里的**历史记录** available: 2026.8.5.1 也改掉。) * revert: 内带 xlings 回退到 2026.8.5.1 —— 2026.8.5.2 有回归 `mcpp builds & runs xlings` 集成 CI 在 2026.8.5.2 上挂掉,**重跑可复现**: error: xlings install_packages failed (exit 1) for 'mcpplibs.xpkg@0.0.48' 归因:同一条 job 在 PR #360(xlings 仍是 2026.8.5.1)上是绿的;两个 xlings 版本 之间只有一个代码提交(xlings#481,把 runtime_deps 的版本匹配从字符串相等换成 semver 范围满足),而失败正好发生在依赖安装阶段。同一批里 cmdline / tinyhttps / capi.lua 都装成功,所以不是全部包受影响。 已报 openxlings/xlings#486。等对方发新版再升 —— 用一个已知有回归的版本去满足 「用最新版」不是升级,是把回归带进来。 .xlings.json 的 mcpp bootstrap pin(2026.8.5.3)不受影响,不动。 本 PR 其余部分(命令长度架构)与此无关,照常。 * docs: #359 的架构设计 —— 两个缺口是同一个形状 在等 xlings 修回归的间隙做 #359 的架构分析。 **关键发现:模型早就存在,新东西没接进去。** mcpp 有一套完整的「依赖提供什么 × 提供给谁」模型 —— `UsageRequirements` × {privateBuild, publicUsage, linkUsage}, include dirs / defines / ldflags / modules 全走它。而 #355 引入的两种新提供物 (host 工具、host 模块)**没有进这个模型**,各自硬编码成「只给发出请求的那条边」: prepare.cppm:4113 toolEnvByConsumer[edge.consumerPackageIndex] prepare.cppm:4016 只遍历 m->dependencies(只认 root 的直接依赖) 所以「库代用户拉起整条 codegen 工具链」在架构上不可能:工具被构建了,但环境变量 记在库的账上,消费者看不见。实测确认过,不是推断。 **根因不是少了一次传播,而是:新增一种提供物时,没有任何地方逼你回答「它怎么 传播」。** 这是「同一决策 N 处推导」的镜像 —— 一个必答问题在**零处**被表达。 缺口 B 同构:build.mcpp 的输入只有「文件内容哈希」与「环境变量」两种形态, `hash_file` 读的是内容,于是「我的输出取决于这个目录里有哪些文件」无法表达 —— 新增 .proto 静默不生成。同样是「新增一种输入时,没地方回答它的指纹怎么取」。 设计主张:两个都收敛成「表 + 必答字段」,与 directives::kTable 同一范式,而不是 各打一个补丁。三条语义写死:传播的是可见性不是自动执行、必须显式声明不能默认 传播(否则是供应链问题)、目录指纹只取成员集合不取内容(否则一次重跑放大成全量 重编)。 两条必须一起做:只做 A 仍要逐个列 proto,只做 B 仍要写 4 条依赖。 * revert: 撤回 xlings 回退 —— 我的归因是错的,与 xlings 无关 真因是 **mcpp-index 的描述符**:pkgs/x/xpkg.lua 里一个格式错误的 0.0.49 条目把 0.0.47 / 0.0.48 一起吞掉了,那两个版本根本不可解析。已由 mcpplibs/mcpp-index#160 修复。 时间线是决定性的:我最后一次失败在 **17:41:31**,#160 合并在 **17:51:19** —— 失败早于修复 10 分钟。 我的归因链有两处错误,记下来: 1. 先怪 xlings 2026.8.5.2,依据是「#360(.5.1)绿、#361(.5.2)红,且两版之间 只有一个代码提交」。**相关性是真的,因果是假的** —— 那段时间 mcpp-index 的 xpkg.lua 也刚好坏了。 2. 回退 xlings 后**仍然失败**,这本该立刻推翻结论,我却先去怀疑自己新加的命令 长度校验(本地复现证明它没误报)。 内带 xlings 恢复到 2026.8.5.2;已在 openxlings/xlings#486 更正并说明。 另记一条待办:mcpp 调 xlings 用 `install_packages ... 2>/dev/null`,把对方的 报错吞了,失败只剩一行「install_packages failed (exit 1)」,分不清是「版本不存在」 还是「构建失败」。这是我误判的助力之一,应当改掉。
speak-agent added a commit
that referenced
this pull request
Aug 5, 2026
…#346) (#362) Every mcpp CI job builds either mcpp itself (tens of TUs) or a synthetic e2e project (single digits). Link-line length, the response-file path and ninja graph size over a large object set therefore had no coverage at all, and the whole command-length defect family surfaced in the ecosystem instead: #274 ninja goals argv 50781 chars vs cmd.exe 8191 #247 Windows CreateProcess 32 KiB #345 POSIX MAX_ARG_STRLEN 128 KiB #360 link.exe LNK1170, response-file line capped at 128 KiB 190 asserts the shape of the generated rule at 25 objects. 191 asserts the scale: 1400 C TUs whose object list is 140 KiB, past the largest command-line ceiling in cmdlimits.cppm, required to link and to run. The regime is asserted rather than assumed. Object naming, the disambiguation prefix and the file count all influence how large the list actually is, so the response file's size is checked against the ceiling directly: if it ever falls back under, the test reports that it has stopped covering the axis instead of passing quietly. Verified to fail without the fix — reverting the generated cxx_link rule to its pre-#345 inline form on this project reproduces the original symptom verbatim: `ninja: fatal: posix_spawn: Argument list too long`. Cost: 1.3s wall on a developer machine. Padded file names carry the object paths to ~100 bytes so the ceiling is reached at a file count this small, bounded on the other side by Windows MAX_PATH.
speak-agent added a commit
that referenced
this pull request
Aug 6, 2026
* test(e2e): cover the link-command SCALE axis, which no CI job reached (#346) Every mcpp CI job builds either mcpp itself (tens of TUs) or a synthetic e2e project (single digits). Link-line length, the response-file path and ninja graph size over a large object set therefore had no coverage at all, and the whole command-length defect family surfaced in the ecosystem instead: #274 ninja goals argv 50781 chars vs cmd.exe 8191 #247 Windows CreateProcess 32 KiB #345 POSIX MAX_ARG_STRLEN 128 KiB #360 link.exe LNK1170, response-file line capped at 128 KiB 190 asserts the shape of the generated rule at 25 objects. 191 asserts the scale: 1400 C TUs whose object list is 140 KiB, past the largest command-line ceiling in cmdlimits.cppm, required to link and to run. The regime is asserted rather than assumed. Object naming, the disambiguation prefix and the file count all influence how large the list actually is, so the response file's size is checked against the ceiling directly: if it ever falls back under, the test reports that it has stopped covering the axis instead of passing quietly. Verified to fail without the fix — reverting the generated cxx_link rule to its pre-#345 inline form on this project reproduces the original symptom verbatim: `ninja: fatal: posix_spawn: Argument list too long`. Cost: 1.3s wall on a developer machine. Padded file names carry the object paths to ~100 bytes so the ceiling is reached at a file count this small, bounded on the other side by Windows MAX_PATH. * fix(pm): install_path 跨命名空间返回了别的包的目录 (2026.8.6.1) `Fetcher::install_path(ns, shortName, version)` 回答的是"**这个**包装在哪"。它最后 那道 legacy 扫描却匹配任何以 `-x-<shortName>` 结尾的目录、不看命名空间,于是查 `ocornut:imgui@1.92.8` 会返回 `compat-x-imgui/1.92.8` —— 另一个仅仅短名相同的包。 调用方随后(a)认为"已安装"而跳过安装,(b)把那个包的源码树当作本包读取。 **危险形态是静默的。** 我是在一个 Form B 邻居上撞见的,所以错误的 verdir 里没有 mcpp.toml,构建停在 "index entry has no `mcpp = ...` field" —— 一个指向错误原因的 诊断。而当两个包的源码都在 verdir 里时,没有任何报错:构建**编译了另一个包的源码 并成功**。e2e 192 在旧二进制上正是这么红的: Compiling acme.widget v1.5.0 Finished dev [unoptimized + debuginfo] in 0.07s FAIL: build succeeded, so acme:widget was satisfied from somewhere — the only widget payload on disk belongs to compat. **为什么一直够不到。** install_path 同时匹配版本,所以同短名的两个包只有在版本也相 同的时候才会撞;而生态里 module 层用的是打包计数(`imgui@0.0.6`),compat 用的是上 游版本(`compat.imgui@1.92.8`),永远不撞。把 module 层的版本对齐到上游 (mcpp-index#163)之后它们就重合了 —— 这个 bug 是被那次对齐**逼出来**的,不是新引入的。 修法:bare `-x-<shortName>` 这一支现在要求目录自身的命名空间前缀是调用方确实问过的 (请求的 ns,或旧式 index-prefixed 布局里的 index 名)。`-x-<ns>.<name>` 那一支把命 名空间带在后缀里,无需前缀约束。这保住了它存在的两种旧布局,拒掉的正是它从不该服务 的那一种。 顺带修诊断:`<verdir>` 是个字面占位符,无法区分"包是 Form B 而你忘了写 mcpp 字段" 与"mcpp 解析到的 verdir 根本不是这个包的"。现在打印真实路径,并在后一种情况下点名。 验证: * 单测 4 条(tests/unit/test_fallback_legacy_dirs.cpp),含"仍要找得到自己命名空间的 短名布局"与"index-prefixed 旧布局"两条防止修过头。 * e2e 192,实测旧二进制红、修复后绿。 * 真实复现同环境 A/B:store 里有 compat-x-imgui/1.92.8 时,ocornut:imgui@1.92.8 在 旧二进制上失败、在修复后正常安装并构建。 * 59 个单测全过。
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
mcpp test 能力批次 — 0.0.104
mcpp test从「单次构建 + 顺序运行」升级为练习级/CI 级可用的测试运行器。两轮批次合并交付,设计评审与路线图见.agents/docs/2026-07-24-mcpp-test-design-review.md,实施计划见.agents/docs/2026-07-23-test-isolation-json-plan.md与2026-07-24-test-batch2-plan.md。新增
tests/00-a/0.cpp→00-a/0,子目录布局可用mcpp test <pattern>:按名字子串过滤(只作用于构建/运行,compile_commands 保持完整)mcpp test --list:枚举测试不构建;JSON 记录供工具消费mcpp test --timeout <secs>:每测试运行截止(POSIX)--message-format json:逐测试 NDJSON(status/exit_code/signal/duration_ms/timed_out/compile_output/run_output)+ 包级错误记录 + summary[build].flagsglob 覆盖测试 TU;死 glob 警告改为查磁盘BuildOptions::ninjaTargets/keepGoing:后端 goal 子集构建的通用接口修复
WIFSIGNALED → 128+WTERMSIG(此前 SIGTERM 死亡伪装成 exit 15)LD_LIBRARY_PATH段错误根治:merged_environ剥离私有 glibc loader 条目(下游 d2x/d2mcpp 的 unsetenv workaround 可删)Compiling → 结果 → 诊断连续块,删除误导性中场 Finished 横幅/的命名豁免验证