Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 56 additions & 0 deletions bounty-ai-assistance-disclosure-guard/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# Bounty AI Assistance Disclosure Guard

This module adds a focused solver AI-assistance disclosure and provenance guard for the Scientific Bounty System in issue `#18`.

Scientific bounty platforms increasingly receive submissions that mix human research, generated code, generated reports, model-assisted analysis, and tool-produced artifacts. Sponsors and reviewers need a deterministic pre-scoring gate that answers a narrow question: is this AI-assisted submission safe and transparent enough to score?

## Scope

- Checks whether AI assistance is allowed by the challenge policy.
- Requires explicit AI-use disclosure when the policy requires it.
- Checks approved AI/tool lists and tool license compatibility.
- Flags prohibited data classes sent through AI workflows.
- Holds packets with private sponsor data, prompt leakage, or secret-like text.
- Checks human review attestation for AI-assisted deliverables.
- Checks generated components for disclosure, human review, provenance hashes, and submission-license fit.
- Requires reproducibility commands or environment notes for AI-assisted work.
- Produces JSON, Markdown, SVG, and MP4 reviewer artifacts.
- Uses synthetic data only.
- Uses no external AI APIs, credentials, private submissions, payment systems, or network calls.

## Requirement Map

| Issue #18 capability | Implementation |
| --- | --- |
| Submission engine | Validates solver submission packets before sponsor/reviewer scoring. |
| Secure private workspaces | Detects private data and prompt/secret leakage before reviewer handoff. |
| Arbitration and reviewer checklists | Emits severity-coded findings, decisions, confidence, evidence, and remediation. |
| IP and licensing options | Checks generated component and tool licenses against challenge policy. |
| Audit logs and reproducibility | Requires provenance hashes, human review, and reproducibility commands. |
| Multi-phase challenge support | Decisions can be run before proposal, prototype, final scoring, award release, or sponsor handoff. |

## Run

```bash
npm test
npm run demo
npm run video
npm run check
```

Demo artifacts are written to `reports/`:

- `summary.json`
- `reviewer-packet.md`
- `summary.svg`
- `demo.mp4`

## Decisions

- `RELEASE`: disclosure, provenance, human review, and reproducibility evidence are scoring-ready.
- `REVIEW`: missing or incomplete provenance can be repaired before scoring.
- `HOLD`: critical policy violations such as undisclosed AI, prohibited private data, prompt leakage, or AI use in a no-AI challenge block scoring and award release.

## AI Assistance Disclosure

Implemented with Codex assistance and locally verified with deterministic tests and generated artifacts.
Original file line number Diff line number Diff line change
@@ -0,0 +1,224 @@
[
{
"id": "release-transparent-ai",
"title": "Transparent AI-assisted climate forecast submission",
"challenge": {
"title": "Regional climate forecasting bounty"
},
"challengePolicy": {
"allowsAiAssistance": true,
"requiresDisclosure": true,
"requiresAiToolList": true,
"approvedTools": ["Codex", "OpenModelLab"],
"allowedToolLicenses": ["MIT", "Apache-2.0"],
"allowedSubmissionLicenses": ["MIT", "CC-BY-4.0"],
"prohibitedDataClasses": ["private sponsor data", "participant health data", "export controlled data"]
},
"aiAssistance": {
"used": true,
"disclosed": true,
"tools": [
{
"name": "Codex",
"type": "coding agent",
"version": "local",
"license": "MIT",
"purpose": "test generation",
"disclosed": true
}
],
"dataClassesUsed": ["public benchmark data"],
"privateDataUsed": false,
"promptLeakage": false
},
"humanReview": {
"completed": true,
"reviewer": "team lead",
"reviewedComponents": ["model.py", "report.md"]
},
"generatedComponents": [
{
"path": "model.py",
"type": "code",
"aiGenerated": true,
"disclosed": true,
"humanReviewed": true,
"provenanceHash": "sha256:111111",
"license": "MIT"
}
],
"reproducibility": {
"commands": "python -m pytest && python run_forecast.py --seed 17",
"environment": "python 3.12",
"artifactHash": "sha256:222222"
},
"submission": {
"license": "MIT",
"includesPrivateData": false,
"promptLeakage": false,
"notes": "AI assistance disclosed in solver packet."
}
},
{
"id": "review-missing-component-provenance",
"title": "AI-assisted materials model with incomplete provenance",
"challenge": {
"title": "Materials discovery bounty"
},
"challengePolicy": {
"allowsAiAssistance": true,
"requiresDisclosure": true,
"requiresAiToolList": true,
"approvedTools": ["Codex"],
"allowedToolLicenses": ["MIT"],
"allowedSubmissionLicenses": ["MIT"],
"prohibitedDataClasses": ["private sponsor data"]
},
"aiAssistance": {
"used": true,
"disclosed": true,
"tools": [
{
"name": "Codex",
"type": "coding agent",
"license": "MIT",
"purpose": "feature extraction",
"disclosed": true
}
],
"dataClassesUsed": ["public crystal dataset"],
"privateDataUsed": false,
"promptLeakage": false
},
"humanReview": {
"completed": true,
"reviewer": "principal investigator"
},
"generatedComponents": [
{
"path": "features.py",
"type": "code",
"aiGenerated": true,
"disclosed": true,
"humanReviewed": false,
"license": "MIT"
}
],
"reproducibility": {
"commands": "python -m pytest",
"environment": "python 3.12"
},
"submission": {
"license": "MIT",
"includesPrivateData": false,
"promptLeakage": false,
"notes": "Needs final review signoff."
}
},
{
"id": "hold-undisclosed-ai",
"title": "Undisclosed AI-generated solver report",
"challenge": {
"title": "Single-cell biomarker bounty"
},
"challengePolicy": {
"allowsAiAssistance": true,
"requiresDisclosure": true,
"requiresAiToolList": true,
"approvedTools": ["Codex"],
"allowedSubmissionLicenses": ["MIT"],
"prohibitedDataClasses": ["private sponsor data", "participant health data"]
},
"aiAssistance": {
"used": true,
"disclosed": false,
"tools": [
{
"name": "Unknown LLM",
"type": "LLM",
"purpose": "drafted report",
"disclosed": false
}
],
"dataClassesUsed": ["public benchmark data"],
"privateDataUsed": false,
"promptLeakage": false
},
"humanReview": {
"completed": false
},
"generatedComponents": [
{
"path": "report.md",
"type": "whitepaper",
"aiGenerated": true,
"disclosed": false,
"humanReviewed": false,
"license": "MIT"
}
],
"reproducibility": {},
"submission": {
"license": "MIT",
"includesPrivateData": false,
"promptLeakage": false,
"notes": "No disclosure included."
}
},
{
"id": "hold-private-data-leak",
"title": "Private sponsor data sent through AI workflow",
"challenge": {
"title": "Protein design bounty"
},
"challengePolicy": {
"allowsAiAssistance": true,
"requiresDisclosure": true,
"approvedTools": ["Codex"],
"allowedSubmissionLicenses": ["Apache-2.0"],
"prohibitedDataClasses": ["private sponsor data", "export controlled data"]
},
"aiAssistance": {
"used": true,
"disclosed": true,
"tools": [
{
"name": "Codex",
"type": "coding agent",
"license": "MIT",
"purpose": "analysis rewrite",
"disclosed": true
}
],
"dataClassesUsed": ["private sponsor data"],
"privateDataUsed": true,
"promptLeakage": true,
"promptExcerpt": "analysis with bearer abcdefghijklmnopqrstuvwxyz"
},
"humanReview": {
"completed": true,
"reviewer": "compliance reviewer"
},
"generatedComponents": [
{
"path": "analysis.ipynb",
"type": "notebook",
"aiGenerated": true,
"disclosed": true,
"humanReviewed": true,
"provenanceHash": "sha256:333333",
"license": "MIT"
}
],
"reproducibility": {
"commands": "python rerun.py",
"environment": "python 3.12"
},
"submission": {
"license": "MIT",
"includesPrivateData": true,
"promptLeakage": true,
"notes": "Reviewer packet includes prompt material."
}
}
]
13 changes: 13 additions & 0 deletions bounty-ai-assistance-disclosure-guard/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
{
"name": "bounty-ai-assistance-disclosure-guard",
"version": "1.0.0",
"description": "Dependency-free AI assistance disclosure and provenance guard for scientific bounty submissions.",
"main": "src/index.js",
"scripts": {
"test": "node --test",
"demo": "node scripts/demo.js",
"video": "node scripts/render-demo-video.js",
"check": "node --check src/index.js && node --check scripts/demo.js && node --check scripts/render-demo-video.js"
},
"license": "MIT"
}
Binary file not shown.
72 changes: 72 additions & 0 deletions bounty-ai-assistance-disclosure-guard/reports/reviewer-packet.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
# Bounty AI Assistance Disclosure Review

Generated: 2026-08-15T20:59:43.334Z

## Decision Counts

- RELEASE: 1
- REVIEW: 1
- HOLD: 2

## Packet Reviews

### Transparent AI-assisted climate forecast submission
Challenge: Regional climate forecasting bounty
Decision: RELEASE
Confidence: 0.96
AI used: true
Generated components: 1
Reviewer note: Regional climate forecasting bounty: AI-assistance disclosure, provenance, human review, and reproducibility evidence are scoring-ready.

### AI-assisted materials model with incomplete provenance
Challenge: Materials discovery bounty
Decision: REVIEW
Confidence: 0.7
AI used: true
Generated components: 1
Reviewer note: Materials discovery bounty: REVIEW before scoring because an ai-generated component lacks human review attestation.
- MAJOR GENERATED_COMPONENT_NOT_HUMAN_REVIEWED: An AI-generated component lacks human review attestation.
Remediation: Add human review evidence before the component can be scored or released to a sponsor.
- MINOR GENERATED_COMPONENT_MISSING_PROVENANCE_HASH: An AI-generated component does not include a provenance or artifact hash.
Remediation: Add a stable artifact hash, prompt digest, or build provenance digest.

### Undisclosed AI-generated solver report
Challenge: Single-cell biomarker bounty
Decision: HOLD
Confidence: 0.1
AI used: true
Generated components: 1
Reviewer note: Single-cell biomarker bounty: HOLD before scoring because the submission used ai assistance without the disclosure required by the challenge policy.
- CRITICAL AI_ASSISTANCE_UNDISCLOSED: The submission used AI assistance without the disclosure required by the challenge policy.
Remediation: Require an explicit AI-use disclosure before scoring, award release, or sponsor handoff.
- MAJOR MISSING_HUMAN_REVIEW_ATTESTATION: AI-assisted work lacks a completed human review attestation.
Remediation: Add named reviewer, review date, reviewed components, and residual-risk notes.
- MAJOR AI_ASSISTED_WORK_NOT_REPRODUCIBLE: The submission lacks reproducibility commands or environment notes for AI-assisted deliverables.
Remediation: Add commands, environment, seeds, and artifact hashes sufficient for sponsor-side reproduction.
- MAJOR UNAPPROVED_AI_TOOL_USED: The submission used an AI/tooling system that is not listed in the challenge-approved tool policy.
Remediation: Add sponsor approval evidence for the tool or rerun the affected work with an approved toolchain.
- MAJOR AI_TOOL_NOT_LISTED_IN_DISCLOSURE: An apparent AI tool is present in provenance records but not listed as disclosed.
Remediation: Update the solver disclosure packet so reviewers can see which AI tools touched the deliverable.
- MAJOR GENERATED_COMPONENT_NOT_DISCLOSED: An AI-generated component is not clearly disclosed in the submission packet.
Remediation: Mark the generated component and explain how it was produced and reviewed.
- MAJOR GENERATED_COMPONENT_NOT_HUMAN_REVIEWED: An AI-generated component lacks human review attestation.
Remediation: Add human review evidence before the component can be scored or released to a sponsor.
- MINOR GENERATED_COMPONENT_MISSING_PROVENANCE_HASH: An AI-generated component does not include a provenance or artifact hash.
Remediation: Add a stable artifact hash, prompt digest, or build provenance digest.

### Private sponsor data sent through AI workflow
Challenge: Protein design bounty
Decision: HOLD
Confidence: 0.1
AI used: true
Generated components: 1
Reviewer note: Protein design bounty: HOLD before scoring because ai assistance appears to have used data classes prohibited by the challenge policy.
- CRITICAL PROHIBITED_DATA_SENT_TO_AI: AI assistance appears to have used data classes prohibited by the challenge policy.
Remediation: Hold the packet, notify the sponsor, and require a clean-room rerun or documented exception.
- CRITICAL PRIVATE_DATA_IN_AI_WORKFLOW: The AI workflow or final packet includes private or sponsor-restricted data.
Remediation: Remove private data, regenerate affected artifacts, and provide reviewer-safe provenance evidence.
- CRITICAL PROMPT_OR_SECRET_LEAKAGE: The packet appears to expose prompts, secrets, bearer tokens, or private workflow instructions.
Remediation: Redact the packet, rotate any exposed credentials if real, and resubmit reviewer-safe evidence only.
- MAJOR GENERATED_COMPONENT_LICENSE_MISMATCH: A generated component uses a license outside the challenge submission policy.
Remediation: Relicense, remove, or obtain sponsor approval before scoring the submission.

Loading