Skip to content

docs: add VPC ingress control-plane plan - #342

Open
privateip wants to merge 2 commits into
mainfrom
docs/vpc-ingress-control-plane-plan
Open

docs: add VPC ingress control-plane plan#342
privateip wants to merge 2 commits into
mainfrom
docs/vpc-ingress-control-plane-plan

Conversation

@privateip

@privateip privateip commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Summary

Issue #856 asked NSO to reconcile a VPCAttachment resource, but that design was rejected — the accepted design (datum-cloud/enhancements#851) keeps Envoy on the cluster overlay and disambiguates tenants via a per-tenant VRF on Envoy's own node instead. This plan documents NSO's actual slice of that design: referencing (not synthesizing) the EndpointSlice a VPC pod publishes, skipping Service synthesis for it downstream, and patching the resulting Envoy cluster with a socket-bind option naming the tenant's VRF device.

Revised after an exhaustive review that verified every code claim against current main and the VRF/socket-bind mechanism against actual kernel/Envoy behavior. That review surfaced several corrections now folded in:

  • The Envoy container needs CAP_NET_RAW for its SO_BINDTODEVICE call — the sidecar's CAP_NET_ADMIN alone doesn't cover this.
  • The VRF device name must be bounded to IFNAMSIZ - 1 (15 chars); the original vrf-<tenant-id> scheme is unbounded.
  • Extension-server RBAC (config/extension-server/rbac/role.yaml) is hand-maintained — make manifests won't add the EndpointSlice grant this plan needs.
  • The HTTPProxy and Gateway controller steps needed real expansion: both touch unconditional existing code (URL parsing, an upstream Get + finalizer stamp, BackendTLSPolicy synthesis) that has to be bypassed, not just the synthesis lines originally cited.
  • The tenant-id label's trust boundary is now called out as an open blocker to resolve with #854/#855 — NSO's own manager already holds unrestricted RBAC on downstream EndpointSlice objects.
  • The downstream-native, no-upstream-owner watch in step 3 has zero precedent in this repo; sequencing now calls for prototyping it before the rest of that step is built on it.

Test plan

  • Plan reflects current code structure (httpproxy_controller.go, gateway_controller.go, extensionserver) accurately enough to implement from

Related to datum-cloud/enhancements#856

Add the implementation plan for NSO's slice of HTTP ingress for VPC
networks, scoped against the accepted design in PR #851 rather than
#856's original (rejected) VPCAttachment framing.

Key changes:
- Document what NSO owns: HTTPProxy backend API + translation,
  extension server VRF socket-bind mutation, sidecar wiring
- Ground each step in existing code (httpproxy_controller.go,
  gateway_controller.go, extensionserver package)
- Flag coordination points with #854/#855 (tenant-id label, VRF
  device naming) that still need byte-for-byte confirmation
- Recommend same-namespace-only resolution and cutting TPP
  integration from this pass, both already applied to issue #856
@privateip
privateip force-pushed the docs/vpc-ingress-control-plane-plan branch from 466481a to 161a6e6 Compare August 20, 2026 11:23
An exhaustive review verified every code claim in the plan against
current main and the VRF/socket-bind mechanism against kernel/Envoy
behavior. Fold the findings back into the plan so implementation
starts from corrected, grounded detail instead of the original draft.

Key changes:
- Grant CAP_NET_RAW to the Envoy container for its SO_BINDTODEVICE
  call; the sidecar's CAP_NET_ADMIN alone does not cover this
- Bound the VRF device name to IFNAMSIZ - 1 (15 chars) instead of an
  unbounded vrf-<tenant-id> string
- Call out the manual config/extension-server/rbac/role.yaml edit
  make manifests does not generate
- Expand the HTTPProxy and Gateway controller steps to cover
  unconditional existing code (URL parsing, upstream Get + finalizer,
  BackendTLSPolicy synthesis) that must be bypassed, not just the
  synthesis lines originally cited
- Flag the tenant-id label's trust boundary as an open blocker, since
  NSO's own manager already holds unrestricted downstream EndpointSlice
  RBAC
- Correct the EG version-pin-comment target to Taskfile.test-infra.yml
  instead of go.mod
- Note the downstream-native, no-upstream-owner watch has zero
  precedent in this repo and should be prototyped before the rest of
  the Gateway controller branch is built on it
- Refresh drifted line numbers and the Coraza sidecar mechanism
  description (native EnvoyDeployment fields, not patch-gated)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

NSO: Control plane integration for VPC ingress attachment

1 participant