Skip to content

Detect unknown MIME types without copying uploads - #579

Open
CyMule wants to merge 4 commits into
mainfrom
perf/no-copy-mime-detection
Open

CyMule wants to merge 4 commits into
mainfrom
perf/no-copy-mime-detection

Conversation

@CyMule

@CyMule CyMule commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Summary

Detect unknown upload MIME types through a proxy of the existing upload stream, avoiding a full-file BytesIO copy. Preserve the uploaded filename as a detection hint and rewind the original stream in a finally block, including when detection fails.

Compatibility

No new file-size, format, or processing limit is introduced. Explicit MIME hints and unsupported-type handling are unchanged. The proxy keeps the request-owned stream open; upload lifecycle management remains with FastAPI. Detection results match the prior copied-stream path on tested text, HTML, CSV, JSON, PDF, XLSX, and PPTX inputs, with both memory-backed and disk-backed spools.

Validation

Synced with current main on September 14, 2026.

  • MIME detection tests: 18 passed, including real-detector comparisons, filename preservation, stream reuse, and rewind on success/failure.
  • Ruff import checks and diff whitespace checks passed.
  • Fresh full CI pending.

@CyMule
CyMule marked this pull request as ready for review July 21, 2026 01:35

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 4 files

Shadow auto-approve: would not auto-approve because issues were found.

Re-trigger cubic

Comment thread prepline_general/api/filetypes.py Outdated

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0 issues found across 2 files (changes from recent commits).

Shadow auto-approve: would auto-approve. Optimizes MIME detection to avoid a full-body BytesIO copy by proxying the spooled upload, preserving behavior and filename. Includes regression tests verifying the original stream is used and rewound. A bounded, performance-improving refactor with no observable external change.

Re-trigger cubic

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant