Skip to content

refactor: abstract object storage backend - #2

Open
dylayed wants to merge 1 commit into
mainfrom
storage-backend-refactor
Open

refactor: abstract object storage backend#2
dylayed wants to merge 1 commit into
mainfrom
storage-backend-refactor

Conversation

@dylayed

@dylayed dylayed commented Aug 7, 2026

Copy link
Copy Markdown
Owner

Summary

This refactors Celld's S3 storage code so the same runtime paths can support another object-store backend. Storage setup is currently spread across bucket access, fleet startup, and LTX replication. This patch moves that setup behind a private storage_backend module and passes configured object_store::ObjectStore instances to callers.

This patch is S3-only and is intended to preserve existing S3 behavior.

Approach

  • Keep bucket, credentials, retry, and client setup in Celld, where daemon configuration already lives.
  • Use the existing object_store::ObjectStore trait instead of adding another provider abstraction.
  • Treat the conditional-write identifier as an opaque version string. S3 still maps it to an ETag; the next patch can map GCS to a generation without changing ownership logic.
  • Inject the configured store into celld-ltx instead of moving Celld's provider policy into that crate.

Most of the remaining diff is mechanical propagation through callers.

Review

The main things to review are the boundary in crates/celld/storage_backend.rs and whether existing S3 construction, credentials, retries, endpoint behavior, CAS behavior, and LTX configuration remain unchanged.

Review this PR before #1. The two PRs are linked in native stack #3.

Validation

  • Celld library and test checks pass.
  • The focused S3 object-storage tests pass.
  • The four-scenario CAS contract passes against real MinIO.
  • The existing MinIO LTX suite passes all 3 tests, including multipart.
  • The PR refactor: abstract object storage backend #2 range passes git diff --check.

@dylayed
dylayed force-pushed the storage-backend-refactor branch 20 times, most recently from fbe603d to 0ac1830 Compare August 10, 2026 18:23
Parse S3 configuration once and share it across deployment, runtime, ownership, and LTX replication. Keep object versions and conditional-write preconditions provider-neutral while preserving existing S3 behavior.
@dylayed
dylayed force-pushed the storage-backend-refactor branch from 0ac1830 to 186eddf Compare August 10, 2026 18:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant