从真实创业失败案例中检索可用的历史先例，而不把语义相似误当作证据。
Historical precedent retrieval for products, ideas, and business models.
简体中文 · English ↓
Counterexample 是一个历史失败先例检索层。它从 1,749 条真实创业失败案例中检索与当前产品、商业模式或创业想法相关的历史先例，并区分可用案例、相邻证据与不足以支撑判断的弱证据。
它不是成功率预测器，也不把相似案例自动写成因果结论。Counterexample 可以通过独立 Skill / CLI 直接使用，也可以通过稳定的 machine-readable contract 向任何兼容的 agent、Skill 或 workflow 提供结构化历史证据。
Semantic similarity ≠ useful precedent.
普通向量检索擅长找到“听起来很像”的记录，但一个案例能否成为有用的 historical precedent，还取决于产品类型、商业机制、客户与市场语境、失败相关性以及证据强度。Counterexample 在语义召回之后继续检查这些一致性，不只返回一个 similarity score。
| 普通语义检索常见输出 | Counterexample 继续回答的问题 |
|---|---|
| 哪些文本与 query 相似？ | 产品与核心机制是否真正一致？ |
| 哪个 cosine score 更高？ | 这是 usable_candidate、adjacent_candidate 还是 weak_candidate？ |
| 返回 Top-K neighbors | 这一轮是否应该使用、降级表达，或 abstain？ |
Counterexample 提供历史证据，不预测未来。历史相似性不等于因果关系，一个过去失败的 startup 也不意味着当前想法必然失败。
需要 Python 3.10 或更高版本。从仓库根目录安装 Retrieval / Skill 运行依赖：
python -m pip install -r retrieval/requirements.txt然后直接查询预构建索引：
python skill/counterexample/scripts/retrieve_cases.py "AI code review tool for pull requests" --top-k 5默认输出（以及显式 --format json）是供程序与 agent 使用的
machine-readable JSON。直接面向人类阅读时，可使用同一 retrieval payload
的 evidence summary：
python skill/counterexample/scripts/retrieve_cases.py "AI code review tool for pull requests" --top-k 5 --format human仓库已经包含 Derived 数据和 Retrieval index，普通查询不需要先 rebuild。首次运行时，如果本机尚未缓存 intfloat/multilingual-e5-small，Sentence Transformers 会从 Hugging Face 下载模型。需要明确指定设备时可使用 --device cpu、--device cuda 或 --device mps。
对以下 query：
AI code review tool for pull requests
v1.1.0 adapter 的真实 Top-5 结果为 2 个 usable candidates 和 3 个 adjacent candidates；运行级 assessment 是 usable_candidates_found。以下仅保留理解结果所需的字段：
| Rank | Historical case | Product type | Candidate class |
|---|---|---|---|
| 1 | Sider | Developer Tools | usable_candidate |
| 2 | Dev Tools AI | Developer Tools | usable_candidate |
| 3 | Kite DevTool | Developer Tools | adjacent_candidate |
| 4 | Codeball | Developer Tools | adjacent_candidate |
| 5 | Ansaro | SaaS (B2B) | adjacent_candidate |
这里的差异是刻意保留的：Sider 与 query 具有直接的核心产品一致性，而语义上接近的通用开发工具仍可能只被标记为 adjacent。usable_candidate 表示更强的先例候选，不表示该案例已经成为独立验证过的“直接先例”。
flowchart LR
Q["Product / Idea / Business Model"]
D[("Historical Failure Cases")]
R["Counterexample Retrieval"]
A["Evidence Assessment<br/>usable · adjacent · weak"]
H["Human / CLI / Skill"]
C["Machine-readable Contract<br/>0.1.2"]
Q --> R
D --> R
R --> A
A --> H
A --> C
Counterexample 本身在图中完成从检索、证据评估到人类与机器输出的完整闭环。它的 Retrieval layer：
- 将输入保留为 deterministic
full_query、core_query与mechanism_queryviews； - 使用
intfloat/multilingual-e5-small对本地 1,749 条记录进行检索； - 在 value proposition、description 与 taxonomy 字段上执行 field-aware rank fusion；
- 结合核心产品一致性、机制覆盖与跨 view 支持，生成 candidate class 和 reason codes；
- 通过 standalone adapter 或 machine contract 输出结果。
这是 deterministic、local、LLM-free 的检索流程。内部分类不是单一 cosine threshold，ranking_score 也只是当前算法版本下的 retrieval relevance，不是置信度、precedent quality、成功率或失败率。
Candidate-level class 描述每条返回记录：
| Contract 值 | 含义 | 建议的 consumer 行为 |
|---|---|---|
usable_candidate |
产品与商业机制具有较强一致性，是更可用的先例候选 | 可以使用，但仍需核查差异与来源边界 |
adjacent_candidate |
有相关性，但存在明显的产品、机制或语境距离 | 只能作为 adjacent evidence，不能升级为直接先例 |
weak_candidate |
当前检索支持不足 | 不应称为 precedent；应 abstain 或只说明证据不足 |
Run-level assessment 描述整个结果集，而且作用域始终是 this_dataset_this_retrieval_run：
assessment.status |
精确含义 |
|---|---|
usable_candidates_found |
至少找到一个 usable_candidate |
weak_candidates_only |
只有 adjacent / weak candidates，没有 usable support |
no_strong_precedent_found_in_this_dataset_run |
此数据集、此次运行没有找到强先例候选；不是全局不存在声明 |
Counterexample 不只服务人类 CLI 用户。任何兼容的 agent、Skill 或 workflow 都可以消费它的 machine-readable contract；该接口不绑定某个特定 consumer。稳定 adapter skill/counterexample/scripts/retrieve_cases.py 同时暴露 CLI 和 Python callable，并输出 UTF-8 machine-readable JSON。下面是上述真实查询的精简 contract 片段；未展示的 lifecycle、failure provenance、relevance signals、quality flags 与 duplicate context 仍存在于完整响应中。
{
"contract_version": "0.1.2",
"release_version": "1.1.0",
"query": "AI code review tool for pull requests",
"result_semantics": "retrieval_candidates_not_verified_precedents",
"assessment": {
"status": "usable_candidates_found",
"scope": "this_dataset_this_retrieval_run",
"candidate_class_counts": {
"usable_candidate": 2,
"adjacent_candidate": 3,
"weak_candidate": 0
}
},
"results": [
{
"rank": 1,
"record_id": 2494,
"name": "Sider",
"product_type": "Developer Tools",
"candidate_class": "usable_candidate",
"reason_codes": [
"strong_mechanism_support",
"broad_product_form_support",
"taxonomy_auxiliary_support",
"core_product_agreement"
]
}
]
}| Version surface | Current version |
|---|---|
| Counterexample Release | 1.1.0 |
| Machine Contract | 0.1.2 |
项目版本与 machine contract 版本有意独立演进：Counterexample 进入 v1.1.0 不等于 JSON schema 必须同步升级。完整字段与解释见 Retrieval 文档 和 Skill contract。
Counterexample 的历史失败案例可以作为创业分析中的辅助证据。当产品定位、目标用户、商业模式或市场判断已经形成后，可以调用 Counterexample 检索相似的历史失败案例，用于补充反例、识别已经出现过的失败模式，并为后续判断提供更多历史依据。
创业分析 Skill / Workflow
↓
形成产品、市场或商业模式判断
↓
调用 Counterexample 检索历史失败案例
↓
作为反例与辅助证据
↓
帮助形成更有历史依据的决策
Counterexample 辅助决策，而不替代决策。任何兼容的 agent、Skill 或 workflow 都可以通过 Machine Contract 0.1.2 取得 structured historical precedent candidates，并根据明确的 evidence semantics 决定使用、降级表达或 abstain。
Counterexample 已对 self-refuting assessment、problem-first decomposition、
load-bearing claim extraction 与 failure-attribution bridge 等潜在增强进行
isolated evaluation。这些机制没有证明对 production baseline 产生足够的
incremental value，因此均未被吸收。当前 architecture 的简单性是实验选择
的结果，而不是遗漏了这些方向。完整状态与保留的 NO-GO 证据见
experiments/README.md。
v1.0.0 的稳定状态来自实际的 failure discovery、focused fix 与 targeted revalidation，而不是版本号自称：
Expanded Validation → Focused Fix → Targeted Revalidation
| Validation | Result |
|---|---|
| Expanded Validation regressions | 6/6 PASS |
| Retrieval / query-view tests | 5/5 PASS |
| Machine contract tests | 6/6 PASS |
| Existing evaluation | 21 queries / 59 checks PASS |
| Mixed-script retrieval smoke | PASS |
| Non-UTF-8 machine output | PASS |
这些是 v1.0.0 release-candidate 的静态验证记录，不是伪装成 CI 的动态 badge。当前仓库中可直接审计相应的 regression tests 与 retrieval/eval-results.json。
v1.1.0 在该稳定基线上增加 presentation-only human format，并公开保留的 research provenance。新增 deterministic tests 验证默认 JSON 兼容、candidate class / evidence boundary、空或弱 assessment 以及非 UTF-8 console 下的中文 输出；retrieval ranking 与 machine contract 均未改变。
counterexample/
├─ raw/ Upstream records and public-data snapshots
├─ derived/ Deterministic enrichment and audit metadata
├─ retrieval/ Local index, runtime, evaluation, and tests
├─ experiments/ Isolated enhancement research and retained NO-GO evidence
├─ skill/
│ └─ counterexample/ Standalone Skill and stable adapter
├─ metadata/ Source discovery, schema, and integrity records
├─ reports/ Collection report and boundaries
└─ analysis/ Dataset and evidence-quality analysis
主要入口：
skill/counterexample/scripts/retrieve_cases.py：稳定 CLI / Python adapter；retrieval/README.md：检索与 contract 细节；derived/README.md：Raw → Derived 的确定性处理边界。
上游数据来自 Loot Drop 的匿名可访问、只读公共前端资源。raw/ 尽量按抓取结果保留记录与相关 snapshots；实际 endpoints、fetch metadata、文件哈希、访问边界与未执行的写操作记录在 reports/crawl-report.md 和 metadata/ 中。
Counterexample 是独立项目，不是 Loot Drop 官方项目。仓库保留上游 attribution，是在说明数据 provenance，而不是声称对上游记录拥有原创作者身份。数据集内的失败描述是 Loot Drop upstream attribution；仓库没有 case-level supporting citations，也没有完成 independent verification。
derived/ 保留全部 27 个 Raw 字段及原值，只增加 _derived namespace，用于：
- deterministic normalization；
- quality flags；
- duplicate-candidate groups 与冲突字段；
- computed / retrieval-supporting metadata；
- transformation provenance。
Derived 不会自动纠正年份或 funding，不会重新分类公司，也不会生成新的分析文本。quality flag 是审计上下文，不是“原记录错误”的证明。
重复候选只被 grouping 与 flagging，不会被自动 merge 或删除。query-time retrieval 会在已知 duplicate group 中保留当前 query 下排名最高的一个代表，原始记录仍保留在数据层。详见 derived/duplicate-groups.json 与 analysis/duplicate-report.json。
项目代码采用 GPL-3.0；上游记录仍按其来源明确 attribution。项目许可证说明不替代对上游内容来源与授权边界的判断。
| Surface | v1.1.0 状态 |
|---|---|
| Raw | unchanged |
| Derived | unchanged |
| Retrieval indexes | unchanged |
| Machine contract schema | unchanged |
| Machine contract version | 0.1.2 |
| Consumer semantics | unchanged |
已经兼容 contract 0.1.2 的 consumer 不需要因为 Counterexample 项目进入 v1.1.0 而执行 schema migration。项目 release 追踪整体能力与稳定性；contract version 只在 machine interface 的兼容性需要时独立变化。完整版本历史由 Git history 与 GitHub Releases 承担。
Counterexample 以 GNU General Public License v3.0 发布。兼容 consumer 的独立许可证不会改变 Counterexample 自身的 GPL-3.0 许可。
Counterexample is a historical precedent retrieval layer. Given a product, idea, or business model, it searches 1,749 real historical startup failure cases and separates stronger precedent candidates from adjacent evidence and weak matches that do not support a conclusion.
It does not predict whether a venture will succeed or fail, and it does not turn similarity into causality. Counterexample is complete as a standalone Skill / CLI and can also provide structured historical evidence to any compatible agent, Skill, or workflow through its machine-readable contract.
Semantic similarity is not the same as a useful precedent.
Vector retrieval can find records that sound alike. A useful historical precedent also requires agreement in product type, business mechanism, customer and market context, failure relevance, and evidence strength. Counterexample evaluates those signals after semantic retrieval and returns explicit candidate semantics rather than only a similarity score.
| A typical semantic search answers | Counterexample also asks |
|---|---|
| Which text resembles the query? | Does the core product and mechanism actually agree? |
| Which cosine score is higher? | Is the case usable, adjacent, or weak? |
| What are the Top-K neighbors? | Should a consumer use the evidence, qualify it, or abstain? |
Historical similarity does not establish causality. A failed historical startup does not imply that a current idea will fail.
Counterexample requires Python 3.10 or later. From the repository root, install the Retrieval / Skill runtime dependencies:
python -m pip install -r retrieval/requirements.txtQuery the bundled prebuilt index:
python skill/counterexample/scripts/retrieve_cases.py "AI code review tool for pull requests" --top-k 5The default output, and explicit --format json, is machine-readable JSON for
programs and agents. For a human-readable evidence summary of the same
retrieval payload, use:
python skill/counterexample/scripts/retrieve_cases.py "AI code review tool for pull requests" --top-k 5 --format humanNo index rebuild is required for normal use. On the first run, Sentence Transformers may download intfloat/multilingual-e5-small from Hugging Face if the model is not cached locally. Use --device cpu, --device cuda, or --device mps only when an explicit device is needed.
For AI code review tool for pull requests, the v1.1.0 adapter returns two usable candidates and three adjacent candidates in the Top 5. The run-level assessment is usable_candidates_found.
| Rank | Historical case | Product type | Candidate class |
|---|---|---|---|
| 1 | Sider | Developer Tools | usable_candidate |
| 2 | Dev Tools AI | Developer Tools | usable_candidate |
| 3 | Kite DevTool | Developer Tools | adjacent_candidate |
| 4 | Codeball | Developer Tools | adjacent_candidate |
| 5 | Ansaro | SaaS (B2B) | adjacent_candidate |
The distinction is deliberate: Sider has direct core-product agreement, while semantically related developer tools can remain adjacent. A usable_candidate is a stronger candidate for further evaluation, not an automatically verified direct precedent.
Counterexample's core loop is complete within the project:
Product / Idea / Business Model + Historical Failure Cases
↓
Counterexample Retrieval
↓
Evidence Assessment
├─ Human / CLI / Skill
└─ Machine-readable Contract 0.1.2
Counterexample creates deterministic full_query, core_query, and mechanism_query views; retrieves over all local records with intfloat/multilingual-e5-small; fuses ranks across value proposition, description, and taxonomy fields; and evaluates core-product agreement, mechanism coverage, and cross-view support before assigning candidate classes and reason codes.
The retrieval path is deterministic, local, and LLM-free. Classification is not a single cosine threshold. ranking_score is retrieval relevance for the current algorithm version—not confidence, precedent quality, success probability, or failure probability.
Candidate-level classes apply to individual records:
| Contract value | Meaning | Consumer behavior |
|---|---|---|
usable_candidate |
Stronger product and mechanism agreement | May be used, with differences and provenance kept explicit |
adjacent_candidate |
Relevant, but with material product, mechanism, or context distance | Keep it labeled as adjacent; never upgrade it to a direct precedent |
weak_candidate |
Insufficient retrieval support | Do not call it a precedent; abstain or report insufficient evidence |
Run-level assessment applies to the entire result set and is always scoped to this_dataset_this_retrieval_run:
assessment.status |
Exact meaning |
|---|---|
usable_candidates_found |
At least one usable candidate was found |
weak_candidates_only |
Only adjacent and/or weak candidates were found |
no_strong_precedent_found_in_this_dataset_run |
No strong candidate was found in this dataset and run; this is not a global absence claim |
Any compatible agent, Skill, or workflow can consume Counterexample through the machine-readable contract; the interface is not tied to a specific consumer. The stable adapter, skill/counterexample/scripts/retrieve_cases.py, exposes both a CLI and a Python callable and writes UTF-8 machine-readable JSON. This is a compact excerpt from the real example above; the full response also includes lifecycle data, failure provenance, relevance signals, quality flags, and duplicate context.
{
"contract_version": "0.1.2",
"release_version": "1.1.0",
"query": "AI code review tool for pull requests",
"result_semantics": "retrieval_candidates_not_verified_precedents",
"assessment": {
"status": "usable_candidates_found",
"scope": "this_dataset_this_retrieval_run",
"candidate_class_counts": {
"usable_candidate": 2,
"adjacent_candidate": 3,
"weak_candidate": 0
}
},
"results": [
{
"rank": 1,
"record_id": 2494,
"name": "Sider",
"product_type": "Developer Tools",
"candidate_class": "usable_candidate",
"reason_codes": [
"strong_mechanism_support",
"broad_product_form_support",
"taxonomy_auxiliary_support",
"core_product_agreement"
]
}
]
}| Version surface | Current version |
|---|---|
| Counterexample Release | 1.1.0 |
| Machine Contract | 0.1.2 |
The project release and machine contract are intentionally versioned independently. The v1.1.0 project release does not require a corresponding schema change. See the Retrieval documentation and Skill contract for the complete field semantics.
Counterexample can add historical counterexamples and supporting evidence to startup analysis. After a product position, target user, business model, or market judgment has been formed, a Skill or workflow can call Counterexample to retrieve similar historical failures, identify recurring failure patterns, and inform the next decision with additional historical context.
Counterexample assists decision-making; it does not replace it. Any compatible agent, Skill, or workflow can obtain structured historical precedent candidates through Machine Contract 0.1.2 and use its explicit evidence semantics to decide whether to use, qualify, or abstain from the results.
Counterexample has isolated and evaluated potential enhancements including self-refuting assessment, problem-first decomposition, load-bearing claim extraction, and a failure-attribution bridge. They did not demonstrate enough incremental value over the production baseline and were not absorbed. The current architecture's simplicity is an experimental choice, not an omission of those directions. See the public research index and its retained NO-GO evidence.
The v1.0.0 stability claim follows real failure discovery, a focused fix, and targeted revalidation:
| Validation | Result |
|---|---|
| Expanded Validation regressions | 6/6 PASS |
| Retrieval / query-view tests | 5/5 PASS |
| Machine contract tests | 6/6 PASS |
| Existing evaluation | 21 queries / 59 checks PASS |
| Mixed-script retrieval smoke | PASS |
| Non-UTF-8 machine output | PASS |
These are static v1.0.0 release-candidate results, not a fabricated dynamic CI claim. The regression tests and retrieval/eval-results.json remain available for inspection.
On that stable baseline, v1.1.0 adds a presentation-layer human format and publishes the retained research provenance. New deterministic tests cover default JSON compatibility, candidate class and evidence boundaries, empty or weak assessments, and Chinese output through a non-UTF-8 console. Retrieval ranking and the machine contract are unchanged.
counterexample/
├─ raw/ Upstream records and public-data snapshots
├─ derived/ Deterministic enrichment and audit metadata
├─ retrieval/ Local index, runtime, evaluation, and tests
├─ experiments/ Isolated enhancement research and retained NO-GO evidence
├─ skill/
│ └─ counterexample/ Standalone Skill and stable adapter
├─ metadata/ Source discovery, schema, and integrity records
├─ reports/ Collection report and boundaries
└─ analysis/ Dataset and evidence-quality analysis
Primary entry points are the stable adapter, Retrieval documentation, and Derived data documentation.
The upstream source is Loot Drop. Records and related snapshots were collected from anonymously accessible, read-only public frontend resources. Endpoints, fetch metadata, hashes, collection boundaries, and deliberately skipped write paths are documented in the crawl report and metadata/.
Counterexample is an independent project, not an official Loot Drop project. Attribution records provenance; it does not claim original authorship of the upstream records. Failure descriptions are Loot Drop upstream attributions. The dataset contains no case-level supporting citations, and independent verification has not been assessed.
derived/ preserves all 27 Raw fields and their values, adding only an _derived namespace for deterministic normalization, quality flags, duplicate-candidate groups, computed retrieval-supporting metadata, and transformation provenance. It does not silently correct years or funding, reclassify companies, or generate new analytical text. A quality flag is audit context, not proof that a source record is wrong.
Duplicate candidates are grouped and flagged, not automatically merged or deleted. Query-time retrieval keeps the highest-ranked representative of a known duplicate group for that query, while the underlying records remain in the data layer. See derived/duplicate-groups.json and analysis/duplicate-report.json.
The project code is GPL-3.0, while upstream records retain explicit source attribution. The project license statement does not replace evaluation of upstream content provenance and authorization boundaries.
| Surface | v1.1.0 status |
|---|---|
| Raw | unchanged |
| Derived | unchanged |
| Retrieval indexes | unchanged |
| Machine contract schema | unchanged |
| Machine contract version | 0.1.2 |
| Consumer semantics | unchanged |
Consumers already compatible with contract 0.1.2 do not need a schema migration merely because Counterexample reached v1.1.0. Project releases track overall capability and stability; the contract version changes independently when the machine interface requires it. Git history and GitHub Releases carry the full changelog.
Counterexample is released under the GNU General Public License v3.0. A compatible consumer's independent license does not change Counterexample's GPL-3.0 license.