Blob storage: `Blob` state machine with a pluggable gRPC data plane by rjhuijsman · Pull Request #109 · reboot-dev/reboot

rjhuijsman · 2026-07-23T15:41:41Z

Before this change, Reboot had no first-class way to store large binary objects: state machines hold protobuf state, which is unsuited to multi-megabyte payloads, so applications had nowhere to put user uploads like images or videos.

This PR introduces the Blob state type as the control plane for one immutable-once-committed binary object: its state holds only metadata (content type, size, upload progress, lifecycle, authorization) while the bytes travel directly between the client and a data plane. Data is uploaded to the data plane in parts, each via PUT to a URL minted by the control plane - compatible with S3 as a data plane, if we ever want that. Uploads are resumable, sizes are enforced against the real bytes at commit time, and never-committed blobs expire automatically.

The data plane is a plain gRPC service, BlobDataPlane (data_plane.proto — no Reboot options, implementable by anything), discovered via REBOOT_BLOB_DATA_PLANE_URL. Locally (rbt dev run / rbt serve run / unit tests) a filesystem-based data plane server is run out-of-the-box.

Authorization: Blob has two mechanisms that combine:

Blob creation is always application-mediated; it can't be done directly from a frontend, creation must go through a backend call. The backend can then use whatever existing auth mechanism they'd like. Assuming the Blob is given a random ID, knowing that state ID acts as a capability: to upload or download you must first know the ID.
Blobs can be created with an uploader_id and downloader_ids; if given these will limit uploads and downloads to only users whose IDs are in those lists.

Reviewer hint: review commits in-order.

TESTED: with new unit tests, and by manually running rbt dev run.

aviator-app · 2026-07-23T15:41:44Z

Current Aviator status

Aviator will automatically update this comment as the status of the PR changes.
Comment /aviator refresh to force Aviator to re-examine your PR (or learn about other /aviator commands).

This pull request is currently open (not queued).

How to merge

To merge this PR, comment /aviator merge or add the mergequeue-ready label.

See the real-time status of this PR on the Aviator webapp.

Use the Aviator Chrome Extension to see the status of your PR within GitHub.

Copilot

Pull request overview

Introduces first-class blob storage with a Reboot control plane, pluggable gRPC data plane, local filesystem implementation, and browser SDK.

Changes:

Adds blob protocols, lifecycle, authorization, multipart storage, and tests.
Integrates local data-plane startup into CLI and test harnesses.
Demonstrates attachments through the chat-room example and React hooks.

Reviewed changes

Copilot reviewed 50 out of 50 changed files in this pull request and generated 14 comments.

Show a summary per file

File	Description
`tests/reboot/std/blob/v1/BUILD.bazel`	Configures blob tests.
`tests/reboot/std/blob/v1/blob_tests.py`	Tests blob lifecycle and authorization.
`tests/reboot/examples/chat-room/serve_expected_output.txt`	Updates structured message output.
`tests/reboot/examples/chat-room/BUILD.bazel`	Adds frontend blob packages.
`reboot/std/react/package.json`	Exports React blob helpers.
`reboot/std/react/BUILD.bazel`	Builds React blob helpers.
`reboot/std/react/blob/package.json`	Configures the blob submodule.
`reboot/std/react/blob/index.tsx`	Implements browser upload/download helpers.
`reboot/std/react/blob/BUILD.bazel`	Builds the React blob module.
`reboot/std/BUILD.bazel`	Includes the blob TypeScript API.
`reboot/std/blob/v1/package.json`	Configures the blob module.
`reboot/std/blob/v1/index.ts`	Exposes Node.js blob integration.
`reboot/std/blob/v1/BUILD.bazel`	Builds blob Python and TypeScript libraries.
`reboot/std/blob/v1/blob.py`	Implements the Blob control plane.
`reboot/std/blob/v1/_store.py`	Implements filesystem blob storage.
`reboot/std/blob/v1/_proxy.py`	Proxies application byte routes.
`reboot/std/blob/v1/_http.py`	Serves filesystem upload/download requests.
`reboot/std/blob/v1/_filesystem_server.py`	Hosts the local data plane.
`reboot/std/blob/v1/_data_plane.py`	Creates data-plane connections.
`reboot/routing/cors_settings.py`	Exposes upload ETags through CORS.
`reboot/examples/chat-room/frontend/web/src/App.tsx`	Adds attachment UI and uploads.
`reboot/examples/chat-room/frontend/web/src/App.module.css`	Styles attachment UI.
`reboot/examples/chat-room/frontend/web/package.json`	Adds blob frontend dependencies.
`reboot/examples/chat-room/frontend/reboot-non-react-web/src/main.ts`	Handles structured messages.
`reboot/examples/chat-room/frontend/mobile/src/App.tsx`	Handles structured messages.
`reboot/examples/chat-room/frontend/.tests/type_check.sh`	Installs local blob packages.
`reboot/examples/chat-room/backend/tests/chat_room_servicer_test.py`	Updates message assertions.
`reboot/examples/chat-room/backend/src/main.py`	Registers the blob library.
`reboot/examples/chat-room/backend/src/chat_room_servicer.py`	Creates attachment blobs.
`reboot/examples/chat-room/api/chat_room/v1/chat_room.proto`	Defines attachment APIs.
`reboot/examples/chat-room/.tests/serve_test.sh`	Updates response validation.
`reboot/cli/common/BUILD.bazel`	Builds data-plane startup support.
`reboot/cli/common/blob_data_plane.py`	Spawns the local data plane.
`reboot/cli/commands/serve.py`	Starts blobs under `serve run`.
`reboot/cli/commands/dev.py`	Starts blobs under `dev run`.
`reboot/cli/commands/BUILD.bazel`	Adds CLI blob dependencies.
`reboot/BUILD.bazel`	Packages blob runtime artifacts.
`reboot/aio/tests.py`	Runs a data plane in tests.
`reboot/aio/http.py`	Adds PUT route support.
`reboot/aio/BUILD.bazel`	Adds test-harness blob dependencies.
`rbt/std/BUILD.bazel`	Packages generated blob APIs.
`rbt/std/blob/v1/package.json`	Configures generated blob modules.
`rbt/std/blob/v1/data_plane.proto`	Defines the gRPC data-plane contract.
`rbt/std/blob/v1/BUILD.bazel`	Generates blob protocol bindings.
`rbt/std/blob/v1/blob.proto`	Defines Blob state and methods.
`documentation/docs/learn_more/testing.md`	Updates testing examples.
`documentation/docs/learn_more/define/protobuf.mdx`	Updates protobuf examples.
`documentation/docs/learn_more/call/from_within_your_app.mdx`	Updates internal-call examples.
`documentation/docs/learn_more/call/from_react.mdx`	Updates React examples.
`documentation/docs/learn_more/applications.mdx`	Documents blob registration and PUT routes.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Before this change, an application's custom HTTP routes (`application.http`) could only be registered for `GET`, `POST`, and `OPTIONS`; the docs listed the `PUT`/`DELETE`/... gap as a known limitation. This blocked serving a plain-HTTP upload endpoint, where `PUT` is the natural verb. Add `application.http.put(...)`, a sibling of the existing `post(...)` that forwards `methods=["PUT"]` to the underlying FastAPI route (the route-capture machinery already supports arbitrary methods; only the public sugar was missing). Update the custom-HTTP-routes documentation to list `PUT` among the supported methods. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Before this change, `Application.run()` invoked each library's `pre_run(application)` hook, but the `Reboot` in-process test harness (`reboot.aio.tests`) did not. A library that performs application setup in `pre_run` — for example, registering custom HTTP routes — therefore behaved differently under test than in a real run, and its routes were simply absent when brought up via the harness. Call `library.pre_run(...)` for every library in `Reboot.up()`, before deciding whether a local Envoy is needed, mirroring what `Application.run()` does. Libraries must already tolerate being `pre_run` more than once (a test may bring the same application up again after a `down`). This harness behavior is covered by unit tests introduced in a later commit (`reboot/std: add a `Blob` state machine with a gRPC blob data plane`): `blobs_tests.py` brings an application up with the `BlobsLibrary`, whose `pre_run` hook connects to the blob data plane and registers the byte-proxying HTTP routes the tests then exercise — which succeeds only when the harness has invoked `pre_run`. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Reboot had no first-class way to store large binary objects: state machines hold protobuf state, which is unsuited to multi-megabyte payloads, so applications had nowhere to put user uploads like images or videos. Add `rbt.std.blobs.v1.Blob`, a state machine that is the *control plane* for one immutable-once-committed binary object. Its state holds only metadata — content type, expected/maximum size, upload progress, lifecycle status — while the bytes live in a *data plane* and travel directly between the client and that data plane via URLs minted per part. Uploads are resumable (parts are idempotent by number), sizes are enforced against the real bytes at commit time, and blobs that are never committed expire automatically. The data plane is a gRPC service, `BlobDataPlane` (`data_plane.proto`), deliberately free of Reboot options so that anything can implement it; the control plane discovers it via `REBOOT_BLOB_DATA_PLANE_URL` and calls it to provision uploads, mint URLs, finalize objects, and delete bytes. Keeping the implementation behind a bare gRPC URL means developers can write their own data planes to fit any environment. A data plane whose URLs are not directly reachable by clients (e.g. when run adjacent to an `rbt dev run` that's used over an `ngrok` tunnel) asks, via its `Configuration`, for certain requests to the app's HTTP server (under the reserved `/__/reboot/blob/`) to be forwarded to it; the `Blob` library then registers reverse-proxying routes on the application, so a single application origin serves both control plane and bytes and no second port needs exposing. A data plane whose URLs _are_ directly reachable (e.g. presigned S3) asks for nothing and is never in the application's path. This commit contains just one implementation of a data plane: a filesystem-based server. It binds loopback only and relies on the forwarded-path proxying above, so it works anywhere a Reboot app may be deployed. The `reboot.aio.tests.Reboot` test harness runs the filesystem data plane in-process for every test, so applications using `reboot.std.blobs` work in unit tests out of the box — which is also how this commit is tested. A later commit has `rbt dev run`/`rbt serve run` provide the same data plane for local runs. Authorization model: blob creation is application-mediated (the application enforces quota and size policy), after which the blob's framework-generated random ID acts as a capability (only callers who know the ID can call, and thus read or write, the blob). In addition the backend may limit the identities of callers by setting `uploader_id` and `downloader_ids` at create-time. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

With the blob data plane behind a gRPC interface, an application that uses `reboot.std.blobs` needs a data-plane service to talk to — but local development must keep working out of the box, with no external service to configure. Have `rbt dev run` and `rbt serve run` start the open-source filesystem data-plane server as a background subprocess whenever `REBOOT_BLOB_DATA_PLANE_URL` is not already set (in Reboot Cloud, or via `--env`, it is — and then nothing is spawned), and point the application at it on localhost. The server picks its own ports and reports them through a ready file only once both its gRPC and HTTP endpoints are listening, so there is no port-allocation race and the application can never observe a half-started data plane. Blob bytes live under the application's state directory, so `rbt dev expunge` removes them along with the rest of the state. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Uploading a file to a `Blob` from the browser means driving the whole multipart protocol — fetching upload instructions, `PUT`ting each part to its URL, reporting ETags, committing, and polling for the result — plus resuming after a dropped connection. Before this change an application author had to write all of that by hand against the generated client. Add `@reboot-dev/reboot-std-react/blobs`: - `useBlobUpload()` — the dead-simple case: given a blob id (from an application RPC, since creation is app-mediated) and a `File`, it uploads every part directly to the data plane, resumes already- confirmed parts, reports progress, and commits. - `BlobUploader` — the same machinery for bytes that don't come from a `File` (media recorders, transforms), with explicit `putPart`/`commit` and a `writable()` stream. - `useBlobDownloadUrl()` — resolves to a URL for a committed blob (e.g. for an `<img src>`), plus a re-exported reactive `useBlob` so any participant — not just the uploader — can render live progress. The part-`PUT` response carries the part's ETag, which the browser must read to report it back; expose the `etag` response header through Envoy's CORS configuration so cross-origin uploads (including direct-to-S3 uploads on the Cloud) can see it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

The chat-room example only sent text, so it demonstrated nothing about storing binary data. Give it file attachments, wired the way a real app would: message-first, with attachment bytes uploaded after the message is already visible. `Send` now takes a list of attachment descriptors; the servicer checks each against a per-attachment size limit, creates a `Blob` per attachment, embeds the blob ids in the immediately-published message, and returns them. The web frontend then uploads each file into its blob via `useBlobUpload`, and every participant renders the attachment off its reactive blob status — a progress bar while it uploads (visible to everyone, since progress lives on the blob's state), the image once committed. No end-user auth here, so blobs are created with an empty `owner_id` (anyone in the room may upload). The documentation snippets that embed the chat-room proto are regenerated to match. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

rjhuijsman force-pushed the rjh.blob-storage-v2 branch from 1574235 to 6f904f0 Compare August 3, 2026 15:49

rjhuijsman commented Aug 3, 2026

View reviewed changes

rjhuijsman force-pushed the rjh.blob-storage-v2 branch from 842a135 to b0c684d Compare August 24, 2026 15:47

rjhuijsman commented Aug 25, 2026

View reviewed changes

rjhuijsman force-pushed the rjh.blob-storage-v2 branch from 263b706 to 23d4ee1 Compare August 25, 2026 12:17

rjhuijsman commented Aug 25, 2026

View reviewed changes

Comment thread documentation/docs/learn_more/call/from_react.mdx Outdated

rjhuijsman force-pushed the rjh.blob-storage-v2 branch 2 times, most recently from ef66db2 to c306853 Compare August 25, 2026 13:10

rjhuijsman marked this pull request as ready for review August 25, 2026 13:17

rjhuijsman self-assigned this Aug 25, 2026

rjhuijsman requested review from benh and a balanced review from Copilot August 25, 2026 13:18

Copilot started reviewing on behalf of rjhuijsman August 25, 2026 13:18 View session

Copilot AI reviewed Aug 25, 2026

View reviewed changes

github-actions Bot reviewed Aug 25, 2026

View reviewed changes

Comment thread reboot/cli/commands/dev.py Outdated

github-actions Bot reviewed Aug 25, 2026

View reviewed changes

Comment thread reboot/examples/chat-room/backend/src/chat_room_servicer.py Outdated

github-actions Bot reviewed Aug 25, 2026

View reviewed changes

Comment thread reboot/std/blob/v1/_http.py

github-actions Bot reviewed Aug 25, 2026

View reviewed changes

Comment thread reboot/std/BUILD.bazel

github-actions Bot reviewed Aug 25, 2026

View reviewed changes

Comment thread documentation/docs/learn_more/call/from_within_your_app.mdx Outdated

rjhuijsman force-pushed the rjh.blob-storage-v2 branch from 53f04f6 to 5fd032e Compare August 25, 2026 20:07

github-actions Bot reviewed Aug 25, 2026

View reviewed changes

Comment thread reboot/cli/commands/dev.py Outdated

github-actions Bot reviewed Aug 25, 2026

View reviewed changes

Comment thread reboot/examples/chat-room/backend/src/chat_room_servicer.py Outdated

github-actions Bot reviewed Aug 25, 2026

View reviewed changes

Comment thread reboot/std/blob/v1/index.ts

github-actions Bot reviewed Aug 25, 2026

View reviewed changes

Comment thread reboot/examples/chat-room/backend/src/chat_room_servicer.py Outdated

rjhuijsman force-pushed the rjh.blob-storage-v2 branch from 7d89404 to 6468f39 Compare August 25, 2026 22:40

rjhuijsman commented Aug 25, 2026

View reviewed changes

Comment thread reboot/examples/chat-room/backend/src/chat_room_servicer.py Outdated

Comment thread rbt/std/blob/v1/BUILD.bazel Outdated

Comment thread rbt/std/blob/v1/BUILD.bazel

Comment thread reboot/std/package.json

rjhuijsman force-pushed the rjh.blob-storage-v2 branch 2 times, most recently from 2ec1f54 to 978f9de Compare August 25, 2026 22:53

rjhuijsman force-pushed the rjh.blob-storage-v2 branch from 978f9de to c2a6ca5 Compare August 25, 2026 22:58

github-actions Bot reviewed Aug 25, 2026

View reviewed changes

Comment thread reboot/std/blob/v1/_store.py Outdated

github-actions Bot reviewed Aug 26, 2026

View reviewed changes

Comment thread reboot/std/blob/v1/_http.py

rjhuijsman and others added 4 commits August 26, 2026 12:22

rjhuijsman force-pushed the rjh.blob-storage-v2 branch from 7cda34a to f02a820 Compare August 26, 2026 12:23

Uh oh!

Conversation

rjhuijsman commented Jul 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aviator-app Bot commented Jul 23, 2026

Current Aviator status

How to merge

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

rjhuijsman commented Jul 23, 2026 •

edited

Loading