Add paper ingestion quality gate

paper-ingestion-quality-gate/README.md

@@ -0,0 +1,48 @@
+# Paper Ingestion Quality Gate
+
+This contribution adds a focused intake guard for the AI Paper Summarizer in SCIBASE's AI-Assisted Research Tools MVP. It checks whether an uploaded or repository-linked paper has been extracted cleanly enough before an AI summarizer generates key findings, implications, or next steps.
+
+The slice is intentionally narrow. It does not score the generated summary, validate citation context, check journal venue quality, assess statistics, run reporting guidelines, detect plagiarism, or review disclosure statements. It sits earlier in the workflow: before summarization, when broken PDF/OCR/table/reference extraction can quietly poison every downstream AI output.
+
+## What It Checks
+
+- Dropped pages between the source PDF and extracted text.
+- Required scientific sections: abstract, methods, results, and discussion.
+- OCR confidence and unrecognized glyph noise.
+- Table extraction completeness before numeric key findings are generated.
+- Figure caption extraction before figure-derived summary bullets are allowed.
+- Reference parsing coverage before citation-based implications are generated.
+- Equation extraction coverage for mathematical or methods summaries.
+- Required extraction provenance for text, sections, tables, figures, references, and digests.
+- Whether key-finding generation is incorrectly enabled while intake findings are unresolved.
+
+## Running Locally
+
+```bash
+npm test
+npm run demo
+npm run check
+npm run video
+```
+
+The demo uses only synthetic intake packets in `data/sample_ingestion_packets.json`.
+
+Generated files:
+
+- `reports/summary.json`
+- `reports/reviewer-packet.md`
+- `reports/summary.svg`
+- `reports/demo.mp4`
+
+The MP4 is generated from an FFmpeg color source and drawtext slate. It is not a screen recording and contains no desktop capture or private information.
+
+## Why This Matters
+
+AI summarizers can sound confident even when source extraction is broken. A missing methods section, low-quality OCR, dropped pages, unparsed tables, or failed references can lead to unsupported key findings and misleading implications.
+
+This guard creates a deterministic intake review packet so SCIBASE can:
+
+1. Block summarization when the source is incomplete.
+2. Route questionable intake packets for manual review.
+3. Suppress table, figure, reference, or equation-derived summary bullets when extraction is incomplete.
+4. Preserve extraction provenance for reviewers and collaborators.

paper-ingestion-quality-gate/data/sample_ingestion_packets.json

@@ -0,0 +1,150 @@
+[
+  {
+    "id": "ingest-summarize-001",
+    "title": "Clean paper intake for abstract and executive summary generation",
+    "source": {
+      "fileType": "pdf",
+      "pageCount": 12
+    },
+    "expected": {
+      "tables": 2,
+      "figures": 3,
+      "references": 42,
+      "minEquationCoverage": 0.9
+    },
+    "extraction": {
+      "pagesExtracted": 12,
+      "sectionsPresent": ["abstract", "introduction", "methods", "results", "discussion", "references"],
+      "ocrConfidence": 0.98,
+      "unrecognizedGlyphRate": 0.002,
+      "textCoverage": 0.96,
+      "tablesExtracted": 2,
+      "figureCaptionsExtracted": 3,
+      "referencesParsed": 41,
+      "equationCoverage": 0.93
+    },
+    "summarizer": {
+      "generateKeyFindings": true,
+      "requireCleanIntake": true
+    },
+    "provenance": {
+      "textLayer": true,
+      "sectionMap": true,
+      "tableMap": true,
+      "figureCaptionMap": true,
+      "referenceMap": true,
+      "extractionDigest": true
+    }
+  },
+  {
+    "id": "ingest-hold-002",
+    "title": "OCR-only scan with missing methods and dropped pages",
+    "source": {
+      "fileType": "pdf_scan",
+      "pageCount": 18
+    },
+    "expected": {
+      "tables": 4,
+      "figures": 6,
+      "references": 61,
+      "minEquationCoverage": 0.9
+    },
+    "extraction": {
+      "pagesExtracted": 15,
+      "sectionsPresent": ["abstract", "introduction", "discussion", "references"],
+      "ocrConfidence": 0.61,
+      "unrecognizedGlyphRate": 0.092,
+      "textCoverage": 0.62,
+      "tablesExtracted": 1,
+      "figureCaptionsExtracted": 3,
+      "referencesParsed": 25,
+      "equationCoverage": 0.41
+    },
+    "summarizer": {
+      "generateKeyFindings": true,
+      "requireCleanIntake": false
+    },
+    "provenance": {
+      "textLayer": true,
+      "sectionMap": false,
+      "tableMap": false,
+      "figureCaptionMap": false,
+      "referenceMap": false,
+      "extractionDigest": false
+    }
+  },
+  {
+    "id": "ingest-review-003",
+    "title": "Good text extraction with incomplete tables and captions",
+    "source": {
+      "fileType": "pdf",
+      "pageCount": 9
+    },
+    "expected": {
+      "tables": 3,
+      "figures": 4,
+      "references": 30,
+      "minEquationCoverage": 0.85
+    },
+    "extraction": {
+      "pagesExtracted": 9,
+      "sectionsPresent": ["abstract", "introduction", "methods", "results", "discussion", "references"],
+      "ocrConfidence": 0.94,
+      "unrecognizedGlyphRate": 0.006,
+      "textCoverage": 0.91,
+      "tablesExtracted": 2,
+      "figureCaptionsExtracted": 2,
+      "referencesParsed": 28,
+      "equationCoverage": 0.87
+    },
+    "summarizer": {
+      "generateKeyFindings": false,
+      "requireCleanIntake": true
+    },
+    "provenance": {
+      "textLayer": true,
+      "sectionMap": true,
+      "tableMap": true,
+      "figureCaptionMap": false,
+      "referenceMap": true,
+      "extractionDigest": true
+    }
+  },
+  {
+    "id": "ingest-hold-004",
+    "title": "Reference and equation extraction failure before implications summary",
+    "source": {
+      "fileType": "pdf",
+      "pageCount": 14
+    },
+    "expected": {
+      "tables": 1,
+      "figures": 2,
+      "references": 48,
+      "minEquationCoverage": 0.9
+    },
+    "extraction": {
+      "pagesExtracted": 14,
+      "sectionsPresent": ["abstract", "introduction", "methods", "results", "discussion", "references"],
+      "ocrConfidence": 0.9,
+      "unrecognizedGlyphRate": 0.01,
+      "textCoverage": 0.88,
+      "tablesExtracted": 1,
+      "figureCaptionsExtracted": 2,
+      "referencesParsed": 20,
+      "equationCoverage": 0.45
+    },
+    "summarizer": {
+      "generateKeyFindings": true,
+      "requireCleanIntake": false
+    },
+    "provenance": {
+      "textLayer": true,
+      "sectionMap": true,
+      "tableMap": true,
+      "figureCaptionMap": true,
+      "referenceMap": false,
+      "extractionDigest": true
+    }
+  }
+]

paper-ingestion-quality-gate/package.json

@@ -0,0 +1,13 @@
+{
+  "name": "paper-ingestion-quality-gate",
+  "version": "1.0.0",
+  "description": "Dependency-free AI summarizer intake guard for broken paper/OCR/table/reference extraction.",
+  "main": "src/index.js",
+  "scripts": {
+    "test": "node --test",
+    "demo": "node scripts/demo.js",
+    "video": "node scripts/render-demo-video.js",
+    "check": "node --check src/index.js && node --check scripts/demo.js && node --check scripts/render-demo-video.js"
+  },
+  "license": "MIT"
+}

paper-ingestion-quality-gate/reports/reviewer-packet.md

@@ -0,0 +1,69 @@
+# Paper Ingestion Quality Gate Report
+
+Generated: 2026-08-15T22:05:11.925Z
+Packets analyzed: 4
+Decision counts: SUMMARIZE 1, REVIEW 1, HOLD 2
+
+## Findings
+
+### ingest-summarize-001: Clean paper intake for abstract and executive summary generation
+
+Decision: SUMMARIZE
+Completeness: 0.984
+
+- No intake quality risk detected.
+
+Recommended actions:
+- Allow AI summarization with normal extraction provenance attached.
+
+### ingest-hold-002: OCR-only scan with missing methods and dropped pages
+
+Decision: HOLD
+Completeness: 0.528
+
+- CRITICAL PDF_PAGE_DROP_DETECTED: The source paper has pages that were not extracted.
+- CRITICAL REQUIRED_SECTION_MISSING: Required scientific sections are missing from the extracted text map.
+- CRITICAL OCR_OR_ENCODING_QUALITY_LOW: OCR confidence or glyph quality is too poor for a reliable AI-generated summary.
+- CRITICAL TABLE_EXTRACTION_INCOMPLETE: One or more expected tables were not extracted for summarizer grounding.
+- MAJOR FIGURE_CAPTION_MAP_INCOMPLETE: Figure captions are missing from the summarizer intake packet.
+- MAJOR REFERENCE_PARSE_COVERAGE_LOW: Reference parsing coverage is too low for reliable related-work or implication summaries.
+- MAJOR EQUATION_EXTRACTION_COVERAGE_LOW: Equation extraction coverage is below the configured summarizer threshold.
+- MAJOR EXTRACTION_PROVENANCE_INCOMPLETE: The intake packet lacks required extraction provenance flags.
+- CRITICAL INTAKE_COMPLETENESS_BELOW_THRESHOLD: Overall source extraction completeness is below the AI summarizer safety threshold.
+- CRITICAL SUMMARIZER_NOT_GATED_ON_CLEAN_INTAKE: AI key-finding generation is enabled while source-ingestion quality findings are unresolved.
+
+Recommended actions:
+- Disable key-finding generation until intake quality is clean.
+- Attach the intake findings to the AI summarizer queue.
+- Regenerate extraction artifacts before key findings or implications are generated.
+- Show reviewers which source sections, tables, figures, or references are unsafe.
+
+### ingest-review-003: Good text extraction with incomplete tables and captions
+
+Decision: REVIEW
+Completeness: 0.877
+
+- MAJOR TABLE_EXTRACTION_INCOMPLETE: One or more expected tables were not extracted for summarizer grounding.
+- MAJOR FIGURE_CAPTION_MAP_INCOMPLETE: Figure captions are missing from the summarizer intake packet.
+- MAJOR EXTRACTION_PROVENANCE_INCOMPLETE: The intake packet lacks required extraction provenance flags.
+
+Recommended actions:
+- Attach the intake findings to the AI summarizer queue.
+- Regenerate extraction artifacts before key findings or implications are generated.
+- Show reviewers which source sections, tables, figures, or references are unsafe.
+
+### ingest-hold-004: Reference and equation extraction failure before implications summary
+
+Decision: HOLD
+Completeness: 0.824
+
+- MAJOR REFERENCE_PARSE_COVERAGE_LOW: Reference parsing coverage is too low for reliable related-work or implication summaries.
+- MAJOR EQUATION_EXTRACTION_COVERAGE_LOW: Equation extraction coverage is below the configured summarizer threshold.
+- MAJOR EXTRACTION_PROVENANCE_INCOMPLETE: The intake packet lacks required extraction provenance flags.
+- CRITICAL SUMMARIZER_NOT_GATED_ON_CLEAN_INTAKE: AI key-finding generation is enabled while source-ingestion quality findings are unresolved.
+
+Recommended actions:
+- Disable key-finding generation until intake quality is clean.
+- Attach the intake findings to the AI summarizer queue.
+- Regenerate extraction artifacts before key findings or implications are generated.
+- Show reviewers which source sections, tables, figures, or references are unsafe.