feat(kubernetes): add template-driven warm pooling

architecture/compute-runtimes.md

@@ -397,6 +397,66 @@ credentials. The driver removes its scheduling gate only after the companions
 exist; measured confirmation and supervisor-session registration gate public
 readiness.
 
+Kubernetes supervisor startup uses `RegisterSupervisor` before any tenant RPC.
+The driver validates the projected token, live proxy/workload Pods and Sandbox
+ownership, and returns the prepared runtime coordinates to the gateway. The
+driver requires Secret read permission in sandbox namespaces to validate the
+proxy descriptor and its owner UID during registration, plus list/delete
+permission for generation Secret cleanup during stop and rollback. The
+gateway checks the persisted runtime generation and delivers the operational
+authentication bundle and launch configuration. Proxy Secrets retain physical
+transport and proxy-CA material, but contain no operational auth bundle.
+
+The workload listener can wait without a logical sandbox ID. Its first token
+must carry a signed binding matching all protected runtime resource claims;
+only then does it pin the logical sandbox identity. Later tokens must match that
+identity, generation and epoch. Transport retries preserve assignment and the
+existing agent-launch replay protection. Other compute drivers retain their
+file-provisioned authentication path.
+
+Optional Kubernetes warm pooling uses the same provisioning path.
+The gateway periodically supplies complete template snapshots and public trust;
+the driver elects a controller using a Kubernetes Lease. Templates from unavailable
+operator workspaces are excluded without blocking snapshot refreshes for eligible
+workspaces. Template startup hints set desired spare capacity, bounded by a
+driver-wide spare-pair cap. Preparing and ready pairs count toward capacity.
+A Kubernetes Sandbox owns each pair and
+records its physical identity, preparation revision and state without a logical
+sandbox ID. Pool inventory selects managed resources by gateway and pool labels;
+it must not require the logical sandbox-ID label. Ordinary sandbox listing and lifecycle reconciliation exclude it.
+An immutable parent-owned Secret journals both bootstrap Secrets so interrupted
+preparation preserves TLS material. Recovery accepts an already released workload
+only when its recorded Pod, Service and network-policy identities still match;
+it requires the existing journal and validates both bootstrap Secrets before
+finishing release. Generation cleanup also removes this journal.
+The authentication generation is also used in Sandbox metadata and generation-scoped
+resource names, so allocation and proxy registration refer to the same runtime.
+
+Standby readiness requires a local workload-listener probe, a live paired proxy
+with recent authenticated registration, the boundary Service and network fence.
+It does not mean the agent has started. The leader conditionally retires spare
+pairs when preparation stalls, standby age expires, templates or trust change,
+or pooling is disabled. Retirement competes with allocation through resource
+versions. Claimed resources remain under ordinary lifecycle management.
+
+Template-based creation selects a compatible ready pair and persists its Sandbox
+UID and prepared generation in the logical record before attempting a conditional
+claim. The claim removes availability and records the logical ID and command on
+the parent Sandbox. Registration checks that parent UID against the persisted
+assignment before delivering credentials. Both Pods retain their physical identity;
+stop/start and cleanup derive companion names from that identity.
+
+Uncertain claims remain pending against the same candidate across gateway restarts.
+Recovery fences a rejected, still-unassigned candidate before recording another;
+it adopts an already committed claim instead of retiring it. With no eligible
+candidate, creation uses the cold path. Readiness and terminal main-process results
+finalize claim recovery. Explicit restart also clears pending claim recovery
+atomically with the new provisioning attempt, retaining the physical assignment
+and its reservation so recovery cannot replace a restarting runtime's identity.
+Pending deletion fences allocation before removing the logical record. Assigned
+pairs are single-use and are deleted with the sandbox; template changes or removal
+retire only spare inventory.
+
 ## Images
 
 The gateway image and Helm chart are built from this repository. Sandbox images

architecture/gateway.md

@@ -7,6 +7,10 @@ attachments; and asks compute runtimes to create or delete sandbox workloads.
 ## Responsibilities
 
 - Authenticate clients and sandbox supervisor sessions.
+- Authorize Kubernetes proxy registration against driver-verified physical
+  runtime identity and persisted logical assignment. `RegisterSupervisor`
+  supplies initial credentials; it does not bypass generation checks or recover
+  a credential lineage that has already advanced through refresh.
 - Serve gRPC APIs for sandbox lifecycle, provider management, policy updates,
   settings, logs, watch streams, and relay forwarding.
 - Serve HTTP endpoints for health, WebSocket tunnels, and edge-auth flows.
@@ -314,7 +318,9 @@ identity inspection without client-side token decoding.
 Sandbox secrets are gateway-signed JWTs bound to a single sandbox ID. Docker,
 Podman, and VM drivers deliver the initial token through supervisor-only
 runtime material; Kubernetes supervisors exchange a projected ServiceAccount
-token through `IssueSandboxToken`. The gateway delegates that opaque credential
+token through `RegisterSupervisor` to receive their runtime assignment and
+in-memory authentication bundle. `IssueSandboxToken` remains available for
+existing bootstrap clients. The gateway delegates that opaque credential
 to the selected compute driver's `AuthenticateSandbox` RPC. A capable driver is
 trusted to return the authenticated sandbox ID, while the gateway still requires
 a matching durable sandbox record before minting a JWT. The Kubernetes driver
@@ -397,6 +403,11 @@ The storage schema is intentionally narrow:
 
 Public RPC contracts and durable protobuf formats have separate ownership. The `openshell.v1.OpenShell` service's request and response roots, streaming flags, and transitive message closure come from the public descriptor set generated by `openshell-core`. The `public_and_durable_schema_inventories_are_complete` test in `openshell-server` owns the counts and fingerprints and requires this inventory to be reviewed whenever it changes. Compute-driver, credential-driver, gateway-interceptor, and supervisor-middleware services are compiled contracts for internal extension boundaries.
 
+`RegisterSupervisor` adds request and response messages to the public closure.
+Its assignment response carries initial runtime credentials but is not persisted
+as a protobuf storage root. Warm-pool selection and synchronization extend the
+internal compute-driver contract; neither changes the durable schema inventory.
+
 `ReportEndpointStatus` is a sandbox-authenticated public gateway RPC. Its request, response, and `EndpointObservation` messages belong only to the public closure. `EndpointStatus` and `EndpointResult` also belong to the durable closure because `Sandbox.status.endpoint_statuses` persists them. The repeated status field uses a new wire tag; stored sandboxes without it decode with an empty endpoint list and retain their lifecycle fields. A fixed payload encoded with the earlier sandbox schema verifies that no database rewrite is required.
 
 Allow and deny append requests carry `L7RuleTarget` to declare the rule, endpoint, and complete affected scope. The removed `host` and `port` fields remain reserved by number and name, and requests without a target are rejected. These mutation requests are not persisted formats.
@@ -548,6 +559,20 @@ revisions commit in one database transaction. SQLite serializes this operation
 with an immediate transaction, while Postgres locks the sandbox row. A failed
 resource-version check or revision insert rolls back the entire operation.
 
+Warm allocation intent and its exclusive target reservation commit in one
+transaction. The `allocation_claims` table keys ownership by configured driver,
+namespace, and physical UID; the owner is a sandbox provisioning attempt and
+runtime generation. A uniqueness conflict rolls back the sandbox write and
+allows immediate selection of another target. Reservations have no TTL and
+survive ambiguous driver outcomes and gateway restart. Only fenced or reclaimed
+allocations release ownership; deleting the owning sandbox cascades its claims.
+The driver's resource CAS still arbitrates physical assignment and retirement.
+Creates with attached providers hold the gateway-wide sandbox lock from provider
+validation through the sandbox and reservation commit, including reservation
+retries. They release it before submitting creation to the driver. Allocation
+completion and recovery then acquire their own synchronization and lifecycle
+locks.
+
 SQLite is the default local store; Postgres is supported for deployments that
 need an external database or multi-replica coordination. Both backends expose
 the same `Store` API and the same logical schema. Backend differences stay

architecture/sandbox.md

@@ -4,6 +4,26 @@ A sandbox is the runtime boundary where agent code executes. A compute driver
 creates it and connects two dedicated components: `openshell-sandbox` inside
 the workload boundary and `openshell-supervisor` outside it.
 
+Kubernetes warm pooling is enabled by default, but only templates with startup
+hints request spare workload/supervisor pairs. The driver caps total unassigned
+inventory; operators can disable pooling through driver configuration or Helm.
+Each claimed pair belongs to one logical sandbox and is never returned to the pool.
+The gateway reserves each physical target in its database atomically with the
+logical allocation intent before sending a claim to the driver. Concurrent
+requests that lose the reservation immediately select another candidate without
+retiring the winner's pair. Create and recovery use bounded retries for definite
+claim rejection and persist each replacement before claiming it. Exhausted warm
+capacity falls back to cold creation in the same invocation. Uncertain claims
+retain their target and generation for idempotent recovery.
+The Sandbox resource owns the assignment. Reconciliation copies its logical
+sandbox name and ID onto both Pods as display labels and updates their sandbox-ID
+annotations for SPIFFE identity. Physical pair selectors and registration
+bindings remain stable. A successful claim wakes
+registration waiters on the same gateway replica; polling durable state remains
+the fallback for other replicas and recovery. Every wake revalidates identity.
+Prepared pairs also carry template name and ID labels on their parent and both
+Pods. These display labels survive assignment and do not affect pool compatibility.
+
 ## Runtime Model
 
 Each sandbox has three trust levels:
@@ -27,6 +47,10 @@ TCP Service, or VM vsock channel. Independent bidirectional `Exchange` RPCs
 carry lifecycle, exec, TCP, and forwarding traffic, while one persistent
 bidirectional `Mediate` RPC carries multiplexed DNS traffic. General application
 UDP is unsupported; UDP DNS remains mediated by the supervisor.
+Boundary connection attempts bound transport establishment and TLS negotiation
+to two seconds per attempt within a 30-second retry window. The window also
+cancels an in-flight attempt at its deadline. Initial attachment can retry calls
+for up to five minutes while the workload boots.
 The sandbox probes HTTP/2 connection liveness every five seconds and closes
 connections that miss a ten-second acknowledgement deadline. Closing a
 connection freezes the owned workload process tree and cancels its stream
@@ -50,9 +74,13 @@ connections, including loopback aliases. Unix control listeners reject workload
 descendants using kernel peer credentials and process ancestry, while ordinary
 workload loopback and Unix services remain available.
 NetworkPolicy is an outer reachability fence, not a confidentiality boundary.
-Each sandbox generation receives a fresh CA and distinct server/client leaves;
-both endpoints bind the same workload identity and immutable driver resource
-claims. Driver crates do not appear in generic process, network, SSH, or
+Each sandbox generation receives a fresh CA and server certificate. The
+supervisor verifies server TLS; the workload authenticates the supervisor using
+a gateway-signed boundary JWT, not a TLS client certificate. Both endpoints bind
+the workload identity and immutable driver resource claims. Kubernetes can
+prepare the listener before logical assignment: the first JWT must also contain
+the exact protected resource binding before the workload accepts its sandbox ID.
+Driver crates do not appear in generic process, network, SSH, or
 session code.
 
 The supervisor exposes readiness only after the sandbox is confirmed and the
@@ -651,6 +679,14 @@ A leader-owned scan runs independently of driver inventory. Expiry records
 `Error`/`ProvisioningTimedOut` before reclaiming compute; cleanup progress and
 backoff survive restart. Late runtime reports cannot replace that result. The
 record and restartable storage survive cleanup, including for ephemeral creates.
+Allocation recovery checks the same persisted deadline before retrying compute.
+Timeout cleanup retires an unclaimed warm candidate through the driver's claim
+CAS before acknowledging reclamation; a candidate already claimed by the sandbox
+uses normal stop semantics to retain storage. Cleanup clears allocation intent
+only after retirement and reclamation succeed. Late allocation results are fenced
+by provisioning attempt and runtime generation, and cannot rearm an expired
+attempt. Records with leftover allocation intent require cleanup even if an older
+gateway already recorded cleanup completion.
 Explicit start is blocked while cleanup is pending, then creates a fresh attempt
 using the latest configuration. Configuration edits alone never restart an
 expired sandbox. Legacy provisioning records receive one persisted rollout

crates/openshell-cli/tests/ensure_providers_integration.rs

@@ -677,6 +677,13 @@ impl OpenShell for TestOpenShell {
         Err(Status::unimplemented("not implemented in test"))
     }
 
+    async fn register_supervisor(
+        &self,
+        _request: tonic::Request<openshell_core::proto::RegisterSupervisorRequest>,
+    ) -> Result<Response<openshell_core::proto::RegisterSupervisorResponse>, Status> {
+        Err(Status::unimplemented("not implemented in test"))
+    }
+
     async fn issue_sandbox_token(
         &self,
         _request: tonic::Request<openshell_core::proto::IssueSandboxTokenRequest>,

crates/openshell-cli/tests/mtls_integration.rs

@@ -529,6 +529,13 @@ impl OpenShell for TestOpenShell {
         Err(Status::unimplemented("not implemented in test"))
     }
 
+    async fn register_supervisor(
+        &self,
+        _request: tonic::Request<openshell_core::proto::RegisterSupervisorRequest>,
+    ) -> Result<Response<openshell_core::proto::RegisterSupervisorResponse>, Status> {
+        Err(Status::unimplemented("not implemented in test"))
+    }
+
     async fn issue_sandbox_token(
         &self,
         _request: tonic::Request<openshell_core::proto::IssueSandboxTokenRequest>,

crates/openshell-cli/tests/provider_commands_integration.rs

@@ -1405,6 +1405,13 @@ impl OpenShell for TestOpenShell {
         Err(Status::unimplemented("not implemented in test"))
     }
 
+    async fn register_supervisor(
+        &self,
+        _request: tonic::Request<openshell_core::proto::RegisterSupervisorRequest>,
+    ) -> Result<Response<openshell_core::proto::RegisterSupervisorResponse>, Status> {
+        Err(Status::unimplemented("not implemented in test"))
+    }
+
     async fn issue_sandbox_token(
         &self,
         _request: tonic::Request<openshell_core::proto::IssueSandboxTokenRequest>,

crates/openshell-cli/tests/sandbox_create_lifecycle_integration.rs

@@ -988,6 +988,13 @@ impl OpenShell for TestOpenShell {
         Err(Status::unimplemented("not implemented in test"))
     }
 
+    async fn register_supervisor(
+        &self,
+        _request: tonic::Request<openshell_core::proto::RegisterSupervisorRequest>,
+    ) -> Result<Response<openshell_core::proto::RegisterSupervisorResponse>, Status> {
+        Err(Status::unimplemented("not implemented in test"))
+    }
+
     async fn issue_sandbox_token(
         &self,
         _request: tonic::Request<openshell_core::proto::IssueSandboxTokenRequest>,

crates/openshell-cli/tests/sandbox_name_fallback_integration.rs

@@ -620,6 +620,13 @@ impl OpenShell for TestOpenShell {
         Err(Status::unimplemented("not implemented in test"))
     }
 
+    async fn register_supervisor(
+        &self,
+        _request: tonic::Request<openshell_core::proto::RegisterSupervisorRequest>,
+    ) -> Result<Response<openshell_core::proto::RegisterSupervisorResponse>, Status> {
+        Err(Status::unimplemented("not implemented in test"))
+    }
+
     async fn issue_sandbox_token(
         &self,
         _request: tonic::Request<openshell_core::proto::IssueSandboxTokenRequest>,

crates/openshell-core/src/grpc_client.rs

@@ -274,6 +274,60 @@ async fn build_plain_channel(endpoint: &str) -> Result<Channel> {
         .wrap_err("failed to connect to OpenShell server")
 }
 
+/// Register with a driver-native projected credential, before installing any
+/// operational token slots. Re-read the token on every reconnect for rotation.
+pub async fn register_supervisor(
+    endpoint: &str,
+) -> Result<crate::proto::RegisterSupervisorResponse> {
+    let path = std::env::var(sandbox_env::K8S_SA_TOKEN_FILE)
+        .into_diagnostic()
+        .wrap_err("registration requires a projected service-account token")?;
+    loop {
+        let token = tokio::fs::read_to_string(&path).await.into_diagnostic()?;
+        let result = match build_plain_channel(endpoint).await {
+            Ok(channel) => {
+                let mut client = OpenShellClient::new(channel);
+                let mut request = tonic::Request::new(crate::proto::RegisterSupervisorRequest {});
+                request.metadata_mut().insert(
+                    "authorization",
+                    format!("Bearer {}", token.trim())
+                        .parse()
+                        .into_diagnostic()?,
+                );
+                request.set_timeout(Duration::from_secs(45));
+                client
+                    .register_supervisor(request)
+                    .await
+                    .map(tonic::Response::into_inner)
+            }
+            Err(_) => Err(Status::unavailable("registration transport unavailable")),
+        };
+        match result {
+            Ok(assignment) => return Ok(assignment),
+            Err(error)
+                if matches!(
+                    error.code(),
+                    tonic::Code::Unavailable
+                        | tonic::Code::DeadlineExceeded
+                        | tonic::Code::Cancelled
+                        | tonic::Code::Aborted
+                        | tonic::Code::Unauthenticated
+                ) =>
+            {
+                // Log only the code, never credential-bearing response bodies.
+                debug!(code = ?error.code(), "supervisor registration unavailable; retrying");
+                tokio::time::sleep(Duration::from_secs(2)).await;
+            }
+            Err(error) => {
+                return Err(miette::miette!(
+                    "supervisor registration rejected ({:?})",
+                    error.code()
+                ));
+            }
+        }
+    }
+}
+
 /// Build a Bearer-authenticated channel to the gateway.
 ///
 /// First call per process resolves the sandbox JWT via the three-step