Skip to content

feat: support chunked transfers and consolidate relay hardening - #28

Open
sidmorizon wants to merge 6 commits into
mainfrom
codex/ok-63101-transfer-chunks
Open

sidmorizon wants to merge 6 commits into
mainfrom
codex/ok-63101-transfer-chunks

Conversation

@sidmorizon

@sidmorizon sidmorizon commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

The relay's three-second per-method limit rejects consecutive Prime Transfer chunks. This change adds bounded chunk forwarding for actual transfer progress, closes message-envelope and response-channel abuse paths, fixes disconnect-during-join admission, combines #26/#27, and consolidates all ten dependency update PRs.

Transfer progress and compatibility

  • Advertise chunkedTransferVersion: 1 and the deployment maxMessageSize in successful room-join responses. New App senders use these for transport selection and preflight size checks; legacy clients ignore the added fields.
  • Allow up to 512 sendTransferChunk requests per connection per second, with at most 64 KiB Base64 data and 72 KiB for the entire JSON envelope. Validate transfer ID, index, and parameters. Invalid chunk input returns 1001; actual chunk throttling returns 1100.
  • Keep the existing single-message transfer API. Clients enable chunking only when both the relay and peer advertise support; otherwise updated clients fall back to the existing loading flow. Percentage progress requires both clients and the relay to support it.
  • Preserve the default rate limit for other methods, with the bounded room-query allowance described below. Encryption algorithms and client key derivation are unchanged.

Historical client binaries have not been tested across every version combination.

Relay security: #26 and #27

  • Combine the overlapping authorization checks into one guard: the sender must belong to the room in both Socket.IO and the authoritative RoomManager. This covers client-to-client requests, responses, and transfer chunks.
  • Return the same empty list for non-member and missing-room user queries, preserving legacy missing-room behavior without exposing room existence. Bound getRoomUsers with a per-connection token bucket (capacity 10, refill 5/second), allowing shipped App pairing/UI queries to overlap with one-second CLI polling. Notify existing room members with user-joined after the peer has joined.
  • Return relay rejections on the originating C2S or C2C response channel, preserving request correlation fields even when the two bridges reuse an ID. Do not reply to one-way requests without an ID. Accept the bridge contract for remoteId: null, finite numbers, or bounded strings.
  • Refresh room activity only after authorized requests pass validation and rate limits, or after authorized responses pass envelope validation. Active transfers survive the idle timeout; invalid and rejected traffic does not renew rooms.
  • Strip server stacks at the bridge's actual outbound response path, including plain-object errors, and omit them from direct E2eeError serialization. Server crash logging remains available.
  • Preserve permissive origin reflection and credentials: true, including credentialed browser polling; remove the unused allowlist configuration. Retain server-generated room-key fields for wire compatibility.
  • Bind relay events to matching REQUEST/RESPONSE types and validate bounded bridge metadata. Cap complete response envelopes at 256 KiB and 1024 responses/second. Both relay directions, including rejected traffic, share 1600 messages/second and 48 MiB/second per connection (the byte budget is at least the configured legacy packet limit). The 48 MiB budget covers 512 complete 72 KiB envelopes plus 12 MiB for ACKs/control traffic. Sustained budget abuse disconnects the source; ordinary valid full-size bursts remain connected.
  • Within the transport packet limit, convert a single oversized or overly complex response with valid membership/metadata into a small same-ID 1001 error for the original caller. Never return it to the responder. Similarly reject correlatable, authorized requests that exceed JSON complexity bounds with an immediate error; discard uncorrelatable or unauthorized input. The legacy request transport limit remains MAX_MESSAGE_SIZE (default 10 MiB).
  • On idle expiry, emit the existing user-left event and remove Socket.IO membership for that room while preserving unrelated rooms on the connection. Make departure from a removed room idempotent so existing client cleanup can finish.
  • Bound supplied join metadata while accepting absent legacy fields. Count JSON escapes/UTF-8 without allocating another copy of large payload strings; extract request rate policies and remove unused protocol types.
  • Reserve room capacity while adapter admission is pending, publish members only after success, and release reservations on disconnect/failure. Keep existing joins idempotent even in a full room.
  • Bound debug room IDs, derive the 1024-chunk limit from named v1 size constants, remove unused ESLint tsconfigs, share join-result construction, and align event declarations/README with the implemented protocol and limits.

The security fixes are integrated without duplicating their membership checks or bypassing the chunk size/rate limits.

Dependency consolidation and ESLint migration

Source PR Package Resolved version in this PR
#7 diff 4.0.4
#12 eslint 10.10.0
#14 ajv 6.15.0
#16 minimatch (3.x) 3.1.5
#18 tar 7.5.22
#20 socket.io-parser 4.2.7
#21 flatted 3.4.4
#22 handlebars 4.7.9
#23 lodash-es 4.18.1
#24 lodash 4.18.1

Several resolved versions include newer compatible patches than the original PR targets. ESLint is intentionally upgraded to the current stable 10.10.0 rather than the older target in #12. Migrate to a shared flat config and typescript-eslint 8.70.0, adapt source typing and obsolete suppressions, and lint source/tests across all workspaces with zero warnings. Pin Yarn 4.14.1 and regenerate the existing lockfile format without copying the old Dependabot lockfile-format churn. No permanent resolution overrides are added.

Validation

  • yarn install --immutable passes.
  • yarn lint passes on ESLint 10.10.0, with zero warnings.
  • yarn workspaces foreach --all --exclude @onekeyhq/monorepo run build passes.
  • Cloud-sync tests: 15 passed. Mock-app tests: 6 passed.
  • Transfer-server TypeScript build, real Socket.IO smoke suite (63 assertions), and crash-logging suite (8 assertions) pass. Coverage includes unauthorized injection through all relay paths, room-query privacy/rate limits, normal one-second polling, peer-join notification, stack removal, consecutive chunks, malformed/oversized chunks, and the 512 requests/second cap.
  • The policy, real Socket.IO compatibility, and admission lifecycle suite passes 30 Node tests (including two parent tests). Coverage includes a same-window burst of 512 full 72 KiB chunk envelopes plus begin/finish/ACKs with both endpoints connected; 9 MiB padding rejection; valid 9 MiB legacy forwarding; numeric/null remote IDs; no-ID rejection; bounded oversized-response errors; complexity rejection; response flood disconnection; mixed-direction budgets; Unicode/JSON escaping; room-query bursts; C2S/C2C ID isolation; idle expiry notifications and membership removal; join metadata, adapter races and reservations.
  • Re-ran 12 old/new App bridge combinations across polling, WebSocket, and upgrade transports against this relay tree; bidirectional legacy/chunk transfers and RPC interactions pass. Old bridge source is fixed at App commit 78cbfd9008a7418abdbff114b7ea07c71ba6878e. Six Chrome cross-origin connection/room-creation checks pass with withCredentials both false and true. The real App chunk sender also completes 32 MiB in each direction. These use actual bridge source and synthetic data, not every historical app binary.
  • Companion App transport/capability/preflight suites pass 48 Jest tests. Chunk totals above 64 MiB fail before beginChunkedTransfer; the legacy fallback checks the full encoded message before emitting wallet data, using the advertised limit or the historical 10 MiB default. Live new-to-old oversize probes deliver zero wallet packets and leave both connections open. The App implementation and localized error copy are maintained separately in the App worktree, outside this server PR; already shipped clients cannot gain local preflight from a server update. Earlier capability tests also cover a v5.18 receiver without getTransferType, with real errors/cancellation preserved.
  • git diff --check passes.

The full recursive dependency audit is not clean: it reports 62 entries across 23 packages (including deprecation notices). Every reported package/version was already present in the pre-consolidation lockfile; this PR does not resolve the repository's entire existing dependency backlog. Examples include engine.io 6.6.4, mongoose 8.17.1, and older minimatch 5.x/9.x paths. Install also retains existing third-party peer warnings.

Consolidates #7, #12, #14, #16, #18, #20, #21, #22, #23, #24, #26, and #27. The original PRs have been closed after consolidation, with references to this PR. This PR remains pending merge.

Related requirement: OK-63101. This PR is not a production deployment.

@socket-security

socket-security Bot commented Sep 17, 2026

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Addedtypescript-eslint@​8.70.01001007498100
Updatedlodash@​4.17.21 ⏵ 4.18.1100100 +1987 +182100
Updatedeslint-config-prettier@​8.10.2 ⏵ 10.1.8100 +1100100 +2983100
Updatedglobals@​13.24.0 ⏵ 17.12.010010086 +195100
Added@​eslint/​js@​10.0.110010010087100
Updatedeslint@​7.32.0 ⏵ 10.10.097 +110010096 +46100

View full report

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 17, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
🔒 Security Review Completed 2026-09-17T08:50:03.228702Z f9edf38 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant