From 7e93b5f176b233b1f3014845a492e625359203ec Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Thu, 20 Aug 2026 03:56:25 +0800 Subject: [PATCH 1/3] docs(openkal): record the layering correction that produced 0.2 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Version 0.1 let the implementation provide the module a consumer imports, with the specification package providing openkal.decl. for it to re-export. Review identified this as a contradiction of what a specification is for, and the identification was correct. The justification does not survive examination. That arrangement existed solely to make optional operations detectable through argument-dependent lookup, which requires an implementation's declarations to be visible to the consumer — and version 0.1 defined no optional operation. The mechanism served a requirement that did not exist, and it was paid for with a real cost: the name a consumer relies upon sat outside the specification's control. 0.2 restores the intended layering, verified: a backend that exports no module builds and runs, and a missing implementation is reported by the linker naming the undefined functions. This is the third occurrence of one shape in this round. The first was openkal.namespace, which merged two kinds of resource in order to unify naming; the second was extending cfg() with capability predicates, which carried an openarch conclusion into openkal. Each designed a mechanism for a requirement that did not yet exist. A requirement that arrives later can be met by deferring the decision and recording the constraints, which costs far less than implementing the wrong mechanism early. --- .agents/docs/2026-08-20-openkal-design.md | 53 +++++++++++++++++- .../2026-08-20-openkal-implementation-plan.md | 56 ++++++++++++------- 2 files changed, 87 insertions(+), 22 deletions(-) diff --git a/.agents/docs/2026-08-20-openkal-design.md b/.agents/docs/2026-08-20-openkal-design.md index 0b7063e1..82453172 100644 --- a/.agents/docs/2026-08-20-openkal-design.md +++ b/.agents/docs/2026-08-20-openkal-design.md @@ -1,8 +1,8 @@ # openkal 设计方案:通用内核 ABI 规范 -**状态**:**0.1 已实施并发布**。规范与声明模块在 +**状态**:**0.2 已实施并发布**(0.1 的分层已被 review 推翻,见 §21)。规范与接口模块在 [`mcpplibs/openkal`](https://github.com/mcpplibs/openkal),Linux 参考实现在 -[`mcpplibs/openkal-linux`](https://github.com/mcpplibs/openkal-linux),两者均为 `0.1.0`。 +[`mcpplibs/openkal-linux`](https://github.com/mcpplibs/openkal-linux),两者均为 `0.2.0`。 实施计划与结果见 [`2026-08-20-openkal-implementation-plan.md`](2026-08-20-openkal-implementation-plan.md)。 @@ -1196,3 +1196,52 @@ extern "C" { | ⚠️ 不做的事 | 它**不算**第三方后端,也不算第三个后端(linux/bare 是官方的两个) | ⇒ 判据仍然是**别人来不来**,而这份实现让「来」这件事从一个季度变成一个周末。 + +--- + +## 21. ⚠️ 0.1 的分层被推翻:实现不该拥有应用 import 的名字 + +0.1 让**实现**提供 `openkal.stream`,规范包提供 `openkal.decl.stream` 由实现再导出。 +review 指出这与「openkal 是接口」矛盾 —— **应用 import 的那个名字落在了规范管不到的一方手里**。 + +### 21.1 我的理由为什么站不住 + +那套安排的**唯一**用途是让可选能力可经 ADL 探测,而 ADL 要求实现的声明对消费者可见。 +⚠️ **而 0.1 一个可选能力都没有** —— `write_vectored` 是我为了演示机制放进去的。 + +⇒ **机制服务的是一个当时不存在的需求**,却付出了「实现拥有接口名」这个真实代价。 + +### 21.2 0.2 的分层(实测成立) + +``` +应用 ──import──► openkal ◄──import── 实现 + │ │ + └──────────── 链接 ─────────────────┘ +``` + +- 规范包提供 `openkal.types` / `.abort` / `.stream` / `.memory` +- **实现零模块**,只贡献定义;它 import 与消费者**同一个**接口,因为它要定义自己声明的东西 +- ⇒ **实现无法扩展接口** —— 这不是需要执行的规则,是这个安排的推论 + +ⓘ 实测:后端零模块的工程编译运行通过;缺实现时报 +`undefined reference to kal_stream_write`(链接期,可读)。 + +### 21.3 代价与去向 + +| | 0.1 | 0.2 | +|---|---|---| +| 缺实现 | 编译期,点名模块 | **链接期**,点名函数 | +| 可选能力探测 | ADL,编译期 `if constexpr` | **无**(0.2 无可选能力) | +| 实现能否扩展接口 | 能加重载,靠 conformance 封 | **不能**,由安排本身排除 | +| 碎片化风险 | 有,需 surface diff 兜底 | **无** | + +⇒ 可选能力的机制**推迟到真的出现时再定**,SPEC 6.3 记录了两个候选与各自的约束 +(以及那三条实测:限定名 vs 非限定名、ADL 到不了未 import 的模块、点号延伸导致模块自环)。 + +### 21.4 元结论 + +这是本轮第三次同一形状:**我为一个尚不存在的需求设计了机制,并为它付出了真实代价。** + +前两次是 `openkal.namespace`(为统一命名合并了两种资源)与 `cfg(mmu)`(把 openarch 的 +结论搬进 openkal)。⇒ **判据应当是「今天有没有这个需求」,而不是「将来会不会有」** —— +将来的需求可以用「推迟决定并记录约束」来接,代价远低于提前实现一个错的机制。 diff --git a/.agents/docs/2026-08-20-openkal-implementation-plan.md b/.agents/docs/2026-08-20-openkal-implementation-plan.md index 3cbdc90b..2408eda1 100644 --- a/.agents/docs/2026-08-20-openkal-implementation-plan.md +++ b/.agents/docs/2026-08-20-openkal-implementation-plan.md @@ -10,9 +10,9 @@ specification itself, which is maintained in the `mcpplibs/openkal` repository. | Repository | Contents | Version | | --- | --- | --- | -| `mcpplibs/openkal` | the specification, the modules that declare it, the surface checker, and a substitution example | 0.1.0 | -| `mcpplibs/openkal-linux` | the reference implementation for Linux, its conformance suite, and an example | 0.1.0 | -| `mcpplibs/mcpp-index` | descriptors for both packages | pull request 220 | +| `mcpplibs/openkal` | the specification, the modules that declare it, the surface checker, and a substitution example | 0.2.0 | +| `mcpplibs/openkal-linux` | the reference implementation for Linux, its conformance suite, and an example | 0.2.0 | +| `mcpplibs/mcpp-index` | descriptors for both packages | pull requests 220 and 221 | Both packages are mirrored to GitCode, and the mirrored archives were verified to be byte-identical to those served by GitHub. @@ -51,16 +51,30 @@ the resolution is recorded. ### 3.1 Architecture +⚠️ **A correction applied after 0.1 was published.** Version 0.1 placed the +module a consumer imports under the control of the implementation: the +specification package provided `openkal.decl.` and the implementation +re-exported it as `openkal.`. Review identified this as a contradiction of +what a specification is for, and the identification was correct. The arrangement +existed to support detection of optional operations through argument-dependent +lookup, which requires an implementation's declarations to be visible to the +consumer — and version 0.1 defined no optional operation, so the mechanism +served nothing that existed. + +Version 0.2 restores the intended layering: the specification package provides +the interface, and an implementation contributes definitions and exports no +module. The cost is that a missing implementation is reported by the linker +rather than by the compiler, and the diagnostic names the undefined functions. + An interface is the unit of provision and of versioning. An implementation provides an interface in whole or not at all, and an interface that is not provided is absent as a module rather than present and refusing. The consequence for the module layout is stated in clause 4 of the -specification: the implementation owns the name a consumer imports, because -argument-dependent lookup does not reach a module the translation unit has not -imported. The alternative arrangement, in which the specification package owns -that name, was rejected because it would have made optional capabilities -undetectable. +specification: the specification package owns every module, and an +implementation exports none. An implementation therefore cannot extend the +interface, and this is not a rule that must be enforced — it follows from the +arrangement. ### 3.2 Stability @@ -76,22 +90,24 @@ reproduced by a test suite. ### 3.3 Simplicity -The capability mechanism uses argument-dependent lookup and a fallback overload. -An earlier design used a record of capability flags together with a separate -configuration file, and both were removed. A record can disagree with the code -it describes; a declaration cannot. The removal also eliminated a second -configuration format from the ecosystem, in which every other fact about a -package resides in `mcpp.toml`. +Version 0.2 defines no optional operation, and therefore no mechanism for +expressing one. Two designs were built and removed before this was recognised: a +record of capability flags with a separate configuration file, and a set of +fallback overloads detected through argument-dependent lookup. Each solved a +problem the specification did not yet have. + +Clause 6.3 records the alternatives and the measurements that constrain them, so +that the choice is informed when an optional operation is first defined. ### 3.4 User experience -A consumer that calls an operation the implementation does not provide is -rejected during compilation, and the diagnostic carries the wording the -specification supplies. This is the default behaviour and requires nothing of -the consumer: no capability test, no configuration, and no annotation. +A consumer imports the interface and names no implementation. Which +implementation supplies the definitions is decided in the manifest, and changing +it is a change to one line. -A consumer that wishes to adapt rather than fail uses the concept the interface -provides, which evaluates to false when the operation is absent. +A consumer that depends upon no implementation compiles and fails to link, with +the undefined functions named. That is later than a compilation failure and is +legible; clause 4.2 records it rather than concealing it. ### 3.5 Compatibility From 747154fd18dcc28941b03a52167fa3404fa55479 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Thu, 20 Aug 2026 04:09:09 +0800 Subject: [PATCH 2/3] docs(openkal): plan for industrial completeness, with musl and the compiler as validation The core set is published; four questions were deferred and each blocks hosting a C library. This plan settles them and derives the interfaces that follow. The method is stated first because it was nearly got wrong. Deriving the interface set from what a compiler calls would have been the third instance of the error the specification already records: POSIX shaped by the C library of its time, WASI preview one shaped by POSIX, openkal shaped by a compiler. Programs validate a derivation; they do not produce one. Two measurements are load-bearing, and they point in opposite directions. GCC creates processes by spawning and calls neither fork nor execve. A specification that had copied the POSIX decomposition would have obliged an implementation on Windows to reproduce fork, which cannot be done faithfully and would have failed the specification's own admission criterion. The resource-derived form and the observed usage agree. GCC also assumes a global namespace of paths, which capability-based kernels do not provide. That is evidence about the program's portability assumptions rather than a reason to admit a global namespace, and it locates the path-resolution work where it belongs: in the C library, performed once. The two validation subjects form a stack rather than a pair. Porting musl is the enabling work; the compiler above it is then an end-to-end check that porting a C library once causes the software above it to run on every implementation. The capability mechanism is settled rather than deferred, by a rule that follows from a distinction the specification already draws: an operation that may be absent becomes an interface of its own, so that its absence is reported by the linker, while a property that varies is reported by a capability word, because a property cannot be called and an operation that is present and always fails is the defect clause 6.4 rejects. Also settles concurrency, ownership, and the interface inventory, and records why openkal.event stays reserved: readiness notification is where environments differ most, and a C library can be hosted without it. --- .../2026-08-20-openkal-completeness-plan.md | 332 ++++++++++++++++++ 1 file changed, 332 insertions(+) create mode 100644 .agents/docs/2026-08-20-openkal-completeness-plan.md diff --git a/.agents/docs/2026-08-20-openkal-completeness-plan.md b/.agents/docs/2026-08-20-openkal-completeness-plan.md new file mode 100644 index 00000000..db8b880f --- /dev/null +++ b/.agents/docs/2026-08-20-openkal-completeness-plan.md @@ -0,0 +1,332 @@ +# openkal: plan for industrial completeness + +This document plans the extension of openkal from its core set to a +specification sufficient to host a C library, and through it the ordinary +software of a hosted system. It does not restate the decisions already taken; +those are recorded in [`2026-08-20-openkal-design.md`](2026-08-20-openkal-design.md) +and in the specification itself, and this plan treats them as given. + +## 1. Method + +The interfaces are derived from the kinds of resource an execution environment +provides. They are not derived from what a particular program calls. + +The distinction is not stylistic, and the specification already records why: a +boundary shaped by one client acquires that client's assumptions and obliges +every other environment to reproduce them. POSIX was shaped by the C library of +its time; the first preview of the WebAssembly System Interface was shaped by +POSIX; an openkal shaped by a compiler would be the third instance of the same +error. + +Programs are therefore used to **validate** the derivation and never to produce +it. A program that cannot be hosted indicates that a resource kind has been +omitted or decomposed incorrectly. A program that can be hosted only by +reproducing its own assumptions indicates nothing about openkal, and something +about the program. + +### 1.1 The two validation subjects, and why they form a stack + +``` + an ordinary POSIX program, such as the GCC toolchain + │ the C standard library and POSIX + ▼ + musl, ported to openkal + │ the openkal C application binary interface + ▼ + an openkal implementation: Linux, Windows, a microkernel, bare metal +``` + +The two subjects are not parallel. Porting a C library is the enabling work, +and porting it once causes the software above it to run on every openkal +implementation without further porting. The compiler is then an end-to-end +check of that claim rather than a separate exercise. + +The arrangement also settles a question the derivation raises. A compiler +assumes a global namespace of paths; that assumption is not universally +satisfiable, since capability-based kernels and the second preview of the +WebAssembly System Interface have no such namespace. The resolution of a path +against a root the environment supplies is therefore work that belongs in the C +library, performed once, rather than in every program or in the specification. + +### 1.2 Evidence gathered + +Measurements taken while preparing this plan. They inform coverage, which is a +question about sufficiency; they do not inform shape. + +| Subject | Measurement | +| --- | --- | +| GCC driver and `cc1plus` | 189 undefined symbols, of which 149 face the environment | +| Process creation in GCC | `posix_spawn` and `posix_spawnp`; neither `fork` nor `execve` appears | +| Path operations in GCC | `open`, `stat`, `lstat`, `mkdir`, `unlink` alongside `fstatat`: a global namespace is assumed | +| musl, static archive | file and directory operations 48, process and signal 21, memory 7, threads 81, time 8, network 23 | +| musl thread construction | the pthread layer is built upon a wait-and-wake primitive rather than upon kernel mutexes | + +Two of these are load-bearing. + +The first is that GCC creates processes by spawning rather than by duplicating +the calling image. A specification that copied the POSIX decomposition would +have obliged an implementation on Windows to reproduce `fork`, which cannot be +done faithfully, and would therefore have failed the specification's own +admission criterion. The resource-derived form and the observed usage agree, +which is corroboration rather than derivation. + +The second is that musl constructs mutexes and condition variables from a +primitive that suspends an execution context until another context wakes it. +The kernel boundary is therefore the primitive, and the synchronisation objects +are library constructions above it. An interface offering mutexes would be +placing a library at the kernel boundary. + +## 2. The four decisions this plan settles + +The core set is specified and published. Four questions were deferred, and each +must be answered before the specification can host a C library. + +### 2.1 Optional capabilities + +An earlier draft deferred the mechanism. The rule adopted here follows from a +distinction the specification already draws for another purpose. + +> An **operation** that an implementation may lack becomes an interface of its +> own. A **property** that varies between implementations is reported by a +> capability word. + +The justification is that the two are not alike. An operation that is present +and always fails is the defect the specification rejects in clause 6.4; the +remedy is that its absence is expressed by its absence, which at the granularity +of an interface means the linker reports it. A property, by contrast, cannot be +called. Whether paths are compared case-sensitively, whether a clock advances +while the machine is suspended, what the granularity of an allocation is: these +are facts, and a program adapts to them rather than invoking them. + +Capability words are the ordinary mechanism of every system that has faced this +question, including `sysconf`, `pathconf`, the auxiliary vector and the +processor identification instruction. They are also the only mechanism available +once the implementation is chosen at link time, which the specification's +layering requires. + +Information therefore becomes available at three times, each being the earliest +at which it exists. + +| Time | Mechanism | What it answers | +| --- | --- | --- | +| Dependency resolution | the implementation package declares the interfaces it provides | may this program be built against this implementation | +| Link | an undefined symbol | was an interface used that the implementation does not provide | +| Run | a capability word | how does this implementation behave within an interface it provides | + +### 2.2 Interfaces beyond the core + +Derived from resource kinds. The core set is unchanged. + +| Interface | Resource | Class | +| --- | --- | --- | +| `openkal.abort` | termination | core | +| `openkal.memory` | a region of the address space | core | +| `openkal.stream` | a byte stream | core | +| `openkal.env` | the parameters a program receives at inception | standard | +| `openkal.time` | a time source | standard | +| `openkal.fs` | a directory, and an open file | standard | +| `openkal.process` | a program image that has been started | standard | +| `openkal.task` | an execution context, and a suspension primitive | standard | +| `openkal.net` | an endpoint | optional | +| `openkal.module` | a code image that has been loaded | optional | +| `openkal.entropy` | a source of unpredictable bits | optional | +| `openkal.event` | readiness of a set of resources | reserved | + +*Standard* denotes an interface an implementation hosting a C library provides. +*Optional* denotes one it may omit without ceasing to host a C library, at the +cost of the facilities built upon it. *Reserved* denotes a name whose contents +are not yet normative. + +`openkal.event` is reserved rather than specified because readiness +notification is the interface at which environments differ most, and because a +C library can be hosted without it: blocking operations suffice, and the +facilities that require readiness are those a program uses when it declines to +block. Specifying it prematurely would produce an interface shaped by whichever +environment was consulted first. + +#### 2.2.1 `openkal.fs` + +The resource is a directory or an open file, and operations are relative to a +directory the program holds. There is no global namespace of paths. + +The reason is the admission criterion. A global namespace is not available in a +capability-based kernel, and an implementation on such a kernel would have to +construct one. Relative operations, by contrast, are natural on every +environment considered: they are the primitive on capability systems, and on +POSIX they are the `…at` family, which musl already uses to implement the +global forms. + +The consequence is that the resolution of an absolute path is work performed by +the C library against a root directory the environment supplies at inception. +This is the arrangement the second preview of the WebAssembly System Interface +adopts, and it is what allows a program to be confined without its cooperation. + +#### 2.2.2 `openkal.process` + +The resource is a program image that has been started. The operations are to +start one, to wait for it, and to request its termination. + +Duplication of the calling image is not among them. It is not universally +implementable, and the observed behaviour of the validation subject does not +require it. + +#### 2.2.3 `openkal.task` + +The resource is an execution context sharing the address space, together with a +primitive that suspends a context until another wakes it. + +Mutexes and condition variables are not among the operations. They are +constructions above the primitive, as the measurement of musl demonstrates, and +placing them at the kernel boundary would place a library there. + +### 2.3 Concurrency + +An implementation shall permit concurrent operations upon distinct handles. + +Concurrent operations upon one handle shall not damage the implementation's own +state. The order in which they take effect, and whether the bytes of one +transfer may be separated by those of another, are unspecified. + +Atomicity below a threshold, which POSIX guarantees for pipes, is not required. +It is not universally implementable, and a specification that required it would +oblige an implementation to introduce buffering it does not otherwise need. + +### 2.4 Ownership + +Handles obtained from the core interfaces are borrowed and are not released. + +Handles obtained from `openkal.fs`, `openkal.process`, `openkal.net` and +`openkal.module` are owned, and each of those interfaces provides the operation +that releases one. An implementation shall not treat a released handle as valid. + +The recommended construction divides the handle word into an index and a +generation, incrementing the generation on release. The specification does not +require it: it requires only the property, which this construction achieves +without a lookup table and therefore without the compatibility layer that clause +7.1 excludes. + +## 3. Work breakdown + +### 3.1 Groups and their dependencies + +``` +S. specification: clauses for the four decisions ─┬─► I. interface modules ─┬─► L. Linux implementation ─┬─► M. musl port ──► G. compiler validation + │ │ │ + └─► C. conformance ──────┴────────────────────────────┘ +``` + +| Group | Tasks | Depends upon | +| --- | --- | --- | +| S | capability rule, interface inventory, concurrency, ownership | this plan | +| I | `env`, `time`, `fs`, `process`, `task` declaration modules | S | +| L | the same five for Linux | I | +| C | behavioural suites, surface comparison, capability-word agreement | I | +| M | musl retargeted onto openkal | L, C | +| G | the compiler built and run above the ported library | M | + +Groups I and C are independent of each other and both depend upon S. Group L +may begin as soon as the declarations of a given interface exist, so the five +interfaces proceed in parallel rather than in sequence. + +### 3.2 Sequence within an interface + +Each interface is taken through the same five steps, and the order is not +interchangeable. + +1. State the resource and the operations that follow from it. +2. Determine which operations may be absent, and separate those into their own + interface. +3. Determine which properties vary, and assign them positions in the capability + word. +4. Write the Linux implementation, which is where a decomposition error becomes + visible. +5. Write the conformance suite, which is where a claim becomes checkable. + +Step 4 precedes step 5 deliberately. The implementation of the core set +demonstrated that a decomposition error is visible to an implementer and not to +a reader of the specification: positioning was excluded from the stream +interface by argument, and the exclusion was confirmed when the Linux +implementation showed that positioning is a property of the individual +descriptor. + +## 4. Assessment + +### 4.1 Architecture + +The interface remains the unit of provision and of versioning, and the +specification retains ownership of every module. The extension adds interfaces; +it does not add mechanisms, with the single exception of the capability word, +which is confined to properties. + +### 4.2 Stability + +Structure layouts remain frozen. The capability word introduces a new obligation: +a position once assigned retains its meaning, and a property that ceases to vary +is not reclaimed. Unassigned positions are reserved and read as zero, so that a +program compiled against a later specification behaves correctly against an +earlier implementation. + +### 4.3 Simplicity + +The rule distinguishing operations from properties replaces the three mechanisms +that were drafted and withdrawn: a record of capability flags, a configuration +file that accompanied it, and a set of fallback overloads detected through +argument-dependent lookup. Each was more elaborate than the rule that replaces +them, and each solved a problem that a correct decomposition does not have. + +### 4.4 Consumer experience + +A program declares the interfaces it requires. A program built against an +implementation that lacks one fails during dependency resolution, with the +interface named. A program that reaches an interface it did not declare fails +at link, with the function named. Neither failure is deferred to run time. + +### 4.5 Compatibility + +The core set does not change. An implementation of the core set that predates +this extension remains conforming, and a program that uses only the core set is +unaffected. + +### 4.6 Portability + +The interfaces are derived rather than borrowed, and each is checked against +four environments before it is specified: a hosted system with a global +namespace, a hosted system without one, a capability-based kernel, and a +freestanding target. An interface that any of the four could satisfy only by +constructing a compatibility layer is decomposed again rather than admitted. + +### 4.7 Upgrade + +An implementation adds interfaces without altering those it already provides. A +program observes the addition through dependency resolution. No existing +manifest requires modification. + +## 5. Criteria for completeness + +The extension is complete when the following hold. Each is an observation rather +than a judgement. + +1. musl, retargeted onto openkal, builds and passes its own test suite against + the Linux implementation. +2. The compiler toolchain, built against that library, compiles and links a + program, and the program runs. +3. A second implementation exists for an environment without a global path + namespace, and the same library binary interface is satisfied by it. +4. The conformance suite verifies, for every interface, the behaviour of the + operations provided, the absence of those not provided, and the agreement + between each capability word and the behaviour it describes. +5. No implementation requires a table, a registry or a name resolver in order to + satisfy the specification. + +Criterion 3 is the one that distinguishes this work from a portable C library. +A specification satisfied only by environments resembling the one it was written +against has not been validated, however many programs it hosts. + +## 6. Matters this plan does not settle + +| Matter | Status | +| --- | --- | +| `openkal.event` | reserved; see 2.2 | +| Symbol versioning | the evolution rule prohibits change rather than permitting coexistence, and an ecosystem that outgrows the prohibition will require a mechanism this specification does not define | +| Locale and character encoding | properties of a C library rather than of a kernel boundary; the specification does not address them | +| Signals as a general mechanism | the process interface provides termination; asynchronous delivery to a running program is deferred with `openkal.event` | From 5657410623609e227e623b5541298c428ecf0a09 Mon Sep 17 00:00:00 2001 From: speak-agent Date: Thu, 20 Aug 2026 05:46:36 +0800 Subject: [PATCH 3/3] docs(openkal): what one portable program found, and what it says about the method MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A program above all eight interfaces was written and run. It exposed two points on which the specification was silent — whether the argument vector supplied to a spawn includes the started program's own name, and how enquiry reports a name that does not exist — and on both, two conforming implementations could have differed. Both were found in under an hour. Neither had been found by reading the text, twice, across two rounds. It exposed three defects in continuous integration alongside them, sharing one shape: a version stated twice and drifted, an assertion naming two of five suites, and a final step that had never executed because the step before it failed first. All four pipelines were green throughout. The document also records what the program had to be redesigned to avoid. Its first version started itself with a marker to distinguish parent from child; the marker did not arrive, because of the very defect the program existed to find, and it started itself without end. A conformance program must not have an unbounded failure mode, and least of all one armed by the defect it looks for. --- .../2026-08-20-openkal-implementation-plan.md | 8 + ...08-20-openkal-portable-program-findings.md | 188 ++++++++++++++++++ 2 files changed, 196 insertions(+) create mode 100644 .agents/docs/2026-08-20-openkal-portable-program-findings.md diff --git a/.agents/docs/2026-08-20-openkal-implementation-plan.md b/.agents/docs/2026-08-20-openkal-implementation-plan.md index 2408eda1..8df55648 100644 --- a/.agents/docs/2026-08-20-openkal-implementation-plan.md +++ b/.agents/docs/2026-08-20-openkal-implementation-plan.md @@ -14,6 +14,14 @@ specification itself, which is maintained in the `mcpplibs/openkal` repository. | `mcpplibs/openkal-linux` | the reference implementation for Linux, its conformance suite, and an example | 0.2.0 | | `mcpplibs/mcpp-index` | descriptors for both packages | pull requests 220 and 221 | +⚠️ **This table records 0.2 and is not the current state.** The specification is +at 0.4.0 and covers eight interfaces; the implementations are `openkal-linux` +0.4.0, `openkal-macos` 0.2.0 and `openkal-libc` 0.2.0. See +[the portable-program findings](2026-08-20-openkal-portable-program-findings.md) +for what 0.4 changed and why, and +[the completeness plan](2026-08-20-openkal-completeness-plan.md) for the +interfaces 0.3 added. + Both packages are mirrored to GitCode, and the mirrored archives were verified to be byte-identical to those served by GitHub. diff --git a/.agents/docs/2026-08-20-openkal-portable-program-findings.md b/.agents/docs/2026-08-20-openkal-portable-program-findings.md new file mode 100644 index 00000000..204314f1 --- /dev/null +++ b/.agents/docs/2026-08-20-openkal-portable-program-findings.md @@ -0,0 +1,188 @@ +# openkal 0.4: what one portable program found, and what it says about the method + +This document records the second round of work on openkal: the writing of a +program above all eight interfaces, the two defects in the specification it +exposed within minutes, the three defects in continuous integration it exposed +alongside them, and the conclusions about method that follow. It continues +[`2026-08-20-openkal-implementation-plan.md`](2026-08-20-openkal-implementation-plan.md) +and [`2026-08-20-openkal-design.md`](2026-08-20-openkal-design.md). + +## 1. What was published + +| Repository | Version | Change | +| --- | --- | --- | +| `mcpplibs/openkal` | 0.4.0 | clauses 7.6 and 7.7; `examples/portable` | +| `mcpplibs/openkal-linux` | 0.4.0 | argv passed unaltered; an assertion that observes it; three CI defects | +| `mcpplibs/openkal-macos` | 0.2.0 | the same, and CI that had never run a step it appeared to run | +| `mcpplibs/openkal-libc` | 0.2.0 | its tests given an implementation; its example asserted against `wc` | +| `mcpplibs/mcpp-index` | PR #223 | four descriptors, merged and propagated | + +No declaration changed, so `SURFACE.txt` is unchanged and an implementation +conforming to 0.3 exports exactly what 0.4 requires. What changed is behaviour +the earlier version left open. + +## 2. The two defects in the specification + +Both were found by writing one program against the specification and running it. +Neither was visible in the specification text, and neither would have been found +by reading it more carefully. + +### 2.1 The argument vector (clause 7.6) + +`kal_process_spawn` takes a path and an argument vector. Version 0.3 did not say +whether the vector includes the started program's own name. Both implementations +prepended the path, so a caller's `argv[0]` arrived at index 1. + +The defect surfaced as a shell reporting `cannot open sh`: given +`argv = {"sh", "-c", "exit 0"}`, the prepending implementation delivered +`{"bin/sh", "sh", "-c", "exit 0"}`, and the shell read `sh` as a script to open. + +The rule adopted is that the vector is complete and is passed unaltered. Three +reasons, in order of weight: + +1. The two sides must agree. A started program reads its own name through + `kal_env_arg(0)`, which does include it. A caller that did not supply it could + not predict what the program would read. +2. The name a program observes as its own is behaviour on every environment that + has an argument vector. The choice belongs to the caller. +3. It is what every adjacent interface does — `posix_spawn`, `fdio_spawn`, and + the conventional form of `CreateProcess`. The outlier needed a reason and did + not have one. + +⭐ **Both implementations agreed, and their agreement was worth nothing.** They +were written by one author from one reading. A second implementation by the same +author tests less than a second implementation by another; this is the limit of +what the two can establish between them, and it is now recorded in the macOS +implementation's own history so that the limit is not mistaken for evidence. + +### 2.2 Absence as an answer (clause 7.7) + +`kal_fs_info` on a name that does not exist: version 0.3 declared +`kal_node_absent` and did not say when it is reported. The implementations +return `kal_ok` with `kind = kal_node_absent`. A third implementer could as +reasonably have returned an error, and a portable program cannot be written +against a point on which two conforming implementations may differ. + +The rule adopted is that enquiry succeeds and reports absence, while opening the +same name reports `kal_err_not_found`. Enquiry and access are different +operations: a caller that asks what a name refers to has been answered when told +that it refers to nothing. It is the same distinction `openkal.env` already draws +between a variable that is absent and one whose value is empty. + +⚠️ **The test I wrote first was wrong, and the implementation was right.** I +asserted that enquiry after removal fails. It succeeded, and my first reading was +that removal had not worked. The file was gone from the disk. Had I trusted the +assertion over the artefact, I would have "fixed" a correct implementation. + +## 3. The three defects in continuous integration + +All four pipelines were green before this round, on packages containing the two +defects above. They share one shape. + +### 3.1 A fact stated twice, which drifted + +`openkal-linux`'s workflow pinned `OPENKAL_VERSION: 0.2.0` while its manifest +named 0.3.0. The surface comparison therefore fetched the 0.2 list, and reported +the four `kal_time_*` names — which that implementation is *required* to export — +as unspecified. The step's own comment said the list was fetched "so that the +comparison has one source rather than a copy that can drift", and the version +selecting it was the copy that drifted. + +The fix is to read it from the manifest, which is where it is declared. + +### 3.2 An assertion naming a subset + +Each implementation asserted that `conformance_stream` and `conformance_memory` +had run. Five suites were present. The assertion exists precisely because *a +suite that discovered nothing reports success* — and a hand-written list defeats +that purpose the moment a suite is added. + +The list is now derived from `tests/*.cpp`. ⭐ The general form: **an assertion +about coverage must be derived from what exists, not from what existed.** + +### 3.3 A step that had never run + +Every implementation's final step ran an example and grepped for +`openkal: vectored writes unavailable` — a line from a 0.1-era optional +operation that no example prints. In `openkal-macos` and `openkal-libc` it also +named a directory the repository does not contain. + +It had never passed, and nobody could have noticed: the step before it failed +first, so it was never reached. ⚠️ **A pipeline whose steps fail in order hides +every later defect, and reports the first as though it were the only one.** +`openkal-macos`'s failure was reported as an inability to start `/bin/bash` in a +working directory, which reads as an infrastructure fault rather than as a +missing example. + +### 3.4 The one that only linking could find + +`openkal-libc`'s three test suites all failed to link, naming sixteen undefined +operations. The package declares against the interface and links against no +implementation — correct for the package, wrong for its own tests, which run. + +This is the first occasion on which clause 4.2's claim (that a missing +implementation is reported by the linker, late but legible) was tested by +something other than a demonstration built to test it. The claim held: the +diagnostic named the operations, and the fix was one manifest section. + +## 4. The portable program + +`openkal/examples/portable` exercises the eight interfaces and prints one line +per observation plus a count of those that did not hold. Each implementation's +continuous integration checks out the specification at the version its manifest +names and builds the program from there. **The program is fetched, never +copied** — a copy in each implementation is a copy that can diverge, and the +whole value of the program is that it does not. + +Two properties were designed in after the first version had neither. + +**It does not start itself.** The first version spawned its own executable with a +`--child` marker. The marker did not arrive — because of the argv defect the +program existed to find — and the program started itself without end. ⚠️ **A +conformance program must not have an unbounded failure mode, and least of all one +armed by the defect it is looking for.** It now starts a shell, and reports the +observation as unobservable if the environment supplies no directory to start it +from. + +**It asserts in both directions.** The workflow requires the summary line *and* +`observations that did not hold: 0` *and* the absence of `NOT HELD`. Asserting +only that the program reported would pass for a program that reported failures. + +## 5. Method + +⭐ **The specification's silences are not visible in the specification.** Two +rules were missing. Reading the text, twice, over two rounds, found neither. +Writing one program and running it found both in under an hour. Clause 9 requires +a conformance procedure for this reason, and the requirement was under-served by +a suite that started programs without observing what they received. + +⭐ **A test that does not observe the thing cannot detect the thing.** The suite +started `/bin/true` and read its status. `/bin/true` ignores its arguments, so +the same status was produced whether the vector arrived intact or shifted. The +replacement starts a shell, whose behaviour depends on the vector. + +⚠️ **Green is a property of the assertions, not of the software.** Four +pipelines were green across two published packages containing an ABI defect, a +CI comparison against the wrong version, an assertion covering two of five +suites, and a step that had never executed. + +⚠️ **The revert probe is what separates an assertion from a decoration.** The new +argv assertion was confirmed to fail against the previous behaviour before the +behaviour was changed. On restoring the fix it still failed — because `mv` +restored an older mtime and nothing rebuilt. Had I read that as "the fix does not +work", I would have chased a defect that was not there. + +⚠️ **One edit, several places.** Writing the index descriptors, a regex that +matched the first version block updated one of three platform tables per file, +leaving `openkal-macos` advertising the new version for Linux and the old one for +macOS. It parsed, and it would have installed. The check that caught it enumerated +what the file *says* rather than what the edit *intended*. + +## 6. What remains + +| Matter | State | +| --- | --- | +| A third implementation by another author | the gate that governs everything beyond; unchanged, and now with a specification worth implementing against | +| `openkal.net`, `openkal.channel` | reserved; no program above the stack needs them yet | +| Symbol versioning | clause 8 protects the interface by prohibiting change; two rules were added by prohibition-compatible means this round, which does not prove the next will be | +| A shared conformance package | the portable program is now the shared artefact in practice; formalising it needs a way for a test package to be compiled against an implementation a third project chooses |