diff --git a/CHANGELOG.md b/CHANGELOG.md index 0d9104519..36d762f98 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -60,7 +60,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - guest-agent: `Worker.GetAttestationForAppKey` is **retained**, unchanged and frozen, and v1 ships no counterpart. The method attests the key v0's KDF derives at path `vms` with purpose `signing`, and no v1 `GetKey(domain, algorithm)` can return that key -- different salt, different `info`, no `purpose` input -- so a v1 counterpart would have handed a pure-v1 app an attestation of a public key whose private half it could not obtain, which is worse than having no method because it looks like it works. A v1 app attests its own key instead: derive it at `/v1/GetKey`, commit the public key into `report_data`, call `/v1/Attest`, and serve the result to relying parties itself. That is strictly more capable, since the app chooses which key and which commitment format rather than being limited to the one the agent would derive. Legacy flows keep using the frozen method; it remains Intel TDX only, because it returns a `GetQuoteResponse` - sdk: the Go SDK's v1 `IssueCert` defaults `usage_server_auth` to true, as the Rust, Python and JavaScript v1 clients already did. Go was the odd one out, so the same argument-free call produced a certificate that could serve TLS in three languages and one that could not in the fourth — and a certificate you cannot serve with is useless to most callers. `WithCertUsageServerAuth(false)` opts out. v0's `GetTlsKey` keeps its `false` default deliberately: that is what the released 0.5.x Go SDK sent, and `DstackClientV0` mirrors released behaviour rather than the better choice - sdk: the JavaScript v1 `issueCert` response no longer carries a raw-bytes accessor. `asUint8Array()` is **removed rather than renamed**: it existed to feed the private key into the blockchain adapters, and v1 has no chain-flavoured surface. `IssueCert` returns TLS material, PEM is the form a TLS stack takes, and a caller who genuinely needs DER converts it with a standard library. The Rust, Python and Go v1 clients already returned the PEM string and the chain alone, so all four now agree. v0's `GetTlsKeyResponse.asUint8Array` is untouched — released API, and the viem and solana adapters depend on its truncating behaviour -- sdk: the JavaScript v1 GPU evidence bundle's `asUint8Array()` is renamed `decodeEvidence()`, matching Python's and Rust's `decode_evidence` and Go, which hands back the decoded `Evidence` bytes directly. The name now says what the bytes are — the vendor's evidence, hex off the wire and decoded byte-exact, because sha256 over precisely those bytes is what the measured `gpu-attestation` event commits to +- sdk: the JavaScript v1 GPU evidence bundle's `asUint8Array()` accessor is gone, and `evidence` is the vendor's bytes directly, as Go's `Evidence` always was. Byte-exact off the wire, because sha256 over precisely those bytes is what the measured `gpu-attestation` event commits to +- sdk: every field the `dstack.guest.v1` proto declares `bytes` is now that language's byte type on the v1 clients — Rust `Vec`, Python `bytes`, JavaScript `Uint8Array`, Go `[]byte` — and the `decode_*` helpers are gone along with the hex strings they decoded. Eleven fields move: `GetKeyResponse`'s `key`, `public_key` and `signature_chain`; `AttestResponse.attestation`; `GpuEvidenceBundle.evidence`; and `InfoResponse`'s `app_id`, `instance_id`, `compose_hash`, `device_id`, `os_image_hash` and `mr_aggregated`. Rust's `AttestConfig.report_data` moves with them, so the public builder and `attest()` finally agree on a type. **The JSON wire is unchanged** — it still carries lowercase hex; the encoding moved into the serialization layer, as serde's `hex::serde` in Rust, an annotated pydantic type in Python, and the client's decode step in JavaScript. Go already did this and is untouched. + + The old typing did quiet damage. `docs/guest-api-v1.md` says of the v1 key claim that `public_key` is the raw derived public key, *not* a hex string, and its verification steps rebuild the claim from raw bytes — so a Rust or Python caller passing `response.public_key` straight into a claim builder built it over 66 ASCII characters instead of 33 bytes. No type error, no exception, just a chain that never verifies. `evidence` had the same shape of problem: three separate documents had to keep repeating "hash the decoded bytes, not the string as returned", precisely because the type did not say it. In Go the mistake was unspellable, and now it is unspellable everywhere -- which is why Rust's `report_data` is a `ReportData` newtype rather than a bare `Vec`: `&str` and `String` both implement `Into>`, so under the builder's `into` coercion `.report_data("00ff")` would still compile and attest the four ASCII bytes of that string. The newtype converts from a `Vec`, an array or a slice and from nothing else, so the ergonomics survive and the string does not. A `compile_fail` doctest keeps it that way. Nor was there one line to learn: before this, three of the eleven fields had no decoder at all in Rust, six had none in Python, and JavaScript had decoded three of them for a while. + + Decoding got stricter where it was silently lenient. JavaScript relied on `Buffer.from(value, 'hex')`, which stops at the first pair it cannot parse and returns the prefix, so a corrupted `app_id` became a short `Uint8Array` and a signature chain with one bad link came back quietly one link short; it now throws and names the field, as Rust, Python and Go already did. A required field that is absent altogether is an error rather than empty bytes -- `os_image_hash` and `mr_aggregated` are the two exceptions, read as empty so a degraded `Info` stays parseable, which is what Rust's `#[serde(default)]` already did and what Python now does instead of rejecting the response. Python also stops accepting hex with embedded whitespace, which `bytes.fromhex` skips and Rust refuses. + + The `borsh` encoding of these structs does change, since borsh writes a `Vec` as length-prefixed bytes where it wrote a hex `String` before. The v1 types shipped in no 0.5.x release, so the window is between 0.6 prereleases: a blob written by an earlier one deserializes without error into these types and yields the ASCII of the hex string. Only the JSON wire is compatible. + + The request direction follows: v1's `attest` and `attest_gpu` take bytes and nothing else in all four SDKs. Rust and Go always did; Python and JavaScript also accepted a string and UTF-8 encoded it, so `attest("deadbeef")` committed to the eight ASCII characters rather than the four bytes they spell, and `attestGpu` on a 32-character string passed the length check on its way to attesting the wrong nonce. Both now raise, and say whether to `encode()` the text or decode the hex. **Breaking for a v1 caller passing a string** -- but v1 has not shipped, and the v0 clients keep the old signature, so a 0.5.x program is unaffected. + + Decoding a malformed response now reaches the same verdict in all four SDKs. Differential testing -- 194 identical JSON bodies through four real clients -- found them agreeing on 143 and diverging on 51, every divergence in absence, `null`, or JSON type confusion rather than bad hex. Go read an absent or null `bytes` field as empty and returned a nil error, so an error body arriving with a 200 handed back a zero-length private key that looked like an answer; JavaScript's hex check stringified its argument, so `app_id: ["00112233"]` decoded to one attacker-chosen byte. Required fields are now required, an absent `os_image_hash` or `mr_aggregated` is still empty, an explicit `null` is malformed everywhere, and a bundle without the `vendor` a caller dispatches on is an error rather than evidence routed to no verifier. All 194 bodies now agree. None of them is reachable from a conforming agent, which emits every field, always lowercase hex, never `null` -- they are reachable from a compromised or non-dstack server, which is the threat model these fields already take seriously. + + **v0 deliberately keeps its hex strings and `decode_*` helpers.** That surface mirrors the released 0.5.x SDK so a 0.5.x program keeps working by changing only the class name; retyping every byte field would break that promise on an API that is frozen anyway. The blockchain adapters are v0-typed and unaffected - sdk: the v0 modules carry a `_v0` suffix, so the file a reader opens matches the client it holds. Rust's `dstack_sdk::dstack_client` becomes `dstack_sdk::dstack_client_v0` and `dstack_sdk_types::dstack` becomes `dstack_sdk_types::dstack_v0`; Python's `dstack_sdk.dstack_client` becomes `dstack_sdk.dstack_client_v0`; Go's `client.go`/`client_test.go` become `client_v0.go`/`client_v0_test.go`; and the JavaScript `index.ts`, which held both surfaces in one file, splits into `client-v0.ts`, `client-v1.ts` and a `shared.ts`, leaving `index.ts` as a barrel that re-exports exactly the names it always did. Until now the unsuffixed *file* meant v0 while the unsuffixed *class* meant v1, so a reader opening `dstack_client.rs` for the recommended client found the legacy one instead. **There are deliberately no backward-compat module aliases**: 0.6.0 is the loud-break release, and an import of an old module path fails at build time rather than silently binding the frozen surface under a name that now means something else. Package-level exports are untouched in every SDK — `dstack_sdk::DstackClient`, `from dstack_sdk import DstackClientV0` and `@phala/dstack-sdk`'s public surface are exactly what they were; only a deep import of the module path moves. In Go this is file naming alone, since it is all one `package dstack` - sdk: the v0 clients are deprecated in the way each language's tooling understands, not only in prose. Rust's `DstackClientV0` and `TappdClient` carry `#[deprecated(since = "0.6.0")]`, so a downstream build warns at every mention of the type — the `use`, the constructor, any signature naming it. Method calls on an already-built client stay silent, because Rust does not propagate the attribute to inherent methods. Python's `DstackClientV0` and `AsyncDstackClientV0` emit a `DeprecationWarning` on construction, through the same helper `TappdClient` already used, alongside the `.. deprecated:: 0.6.0` docstring note they already carried. JavaScript's `DstackClientV0` already had its `@deprecated` JSDoc and `TappdClient` gains one. Go's `// Deprecated:` markers were in place but seven sat mid-comment rather than as their own trailing paragraph, which is the only form gopls and pkg.go.dev recognise, and are repaired. diff --git a/docs/confidential-ai.md b/docs/confidential-ai.md index 8d4e2f8a3..f418b805f 100644 --- a/docs/confidential-ai.md +++ b/docs/confidential-ai.md @@ -144,7 +144,7 @@ from dstack_sdk import DstackClient client = DstackClient() info = client.info() -print(f"Compose hash: {info.compose_hash}") +print(f"Compose hash: {info.compose_hash.hex()}") # Data provider compares this against the docker-compose they reviewed ``` diff --git a/docs/guest-api-v1.md b/docs/guest-api-v1.md index dff079898..2d8b28172 100644 --- a/docs/guest-api-v1.md +++ b/docs/guest-api-v1.md @@ -687,6 +687,13 @@ explicitly named and marked legacy, and still carries its `Sign` and `Verify` RPCs. They are transport mirrors, not a compatibility layer: neither translates a call to the other, and each one's method set is exactly its surface's. +Every field this document declares `bytes` is that language's byte type in the +v1 clients -- `Vec`, `bytes`, `[]byte`, `Uint8Array` -- with hex confined to +serialization. `public_key` is why: the claim above is built over the raw key, +and a hex string handed to a claim builder silently produces a chain that never +verifies. The v0 clients keep their hex strings and `decode_*` helpers, because +that surface is frozen. + That alias flipped in 0.6.0. Code that used the unsuffixed client for v0 calls fails loudly on upgrade -- the v1 signatures differ and `GetKey` requires `algorithm` explicitly -- rather than silently deriving different keys under the diff --git a/sdk/README.md b/sdk/README.md index df1a05a99..eeb54c7f8 100644 --- a/sdk/README.md +++ b/sdk/README.md @@ -34,6 +34,11 @@ a call to the other, and each one's method set is exactly its surface's. v1 has no `sign` and no `verify`, because any caller that can reach the socket can ask `get_key` for the private key and do both locally. +Every field the v1 proto declares `bytes` is that language's byte type on the v1 +clients -- `Vec`, `bytes`, `[]byte`, `Uint8Array` -- with hex confined to +serialization; there are no `decode_*` helpers. The v0 clients keep their hex +strings and helpers, because that surface is frozen. + > **v1 keys are not v0 keys.** Deriving under the same name through `DstackClient` > returns *different key material* than `DstackClientV0` does. This is deliberate -- > the v0 KDF ignored the algorithm, so one secret served both curves -- and diff --git a/sdk/go/dstack/client_v1.go b/sdk/go/dstack/client_v1.go index 10412fb0d..fd0f98ace 100644 --- a/sdk/go/dstack/client_v1.go +++ b/sdk/go/dstack/client_v1.go @@ -14,6 +14,7 @@ package dstack import ( + "bytes" "context" "encoding/hex" "encoding/json" @@ -163,12 +164,124 @@ func decodeHexField(name string, value string) ([]byte, error) { return decoded, nil } +// requireHexField decodes a `bytes` field the response is meaningless without. +// +// The pointer is what makes absence visible. Decoding into a `string` turns a +// JSON null, and a key the response never had, into "" -- which hex-decodes to +// empty bytes and returns a nil error, so a caller reads an empty private key +// or an empty app_id as a valid answer. The agent emits every field, so +// absence means the response did not come from a working agent, and the other +// three SDKs all refuse it. +func requireHexField(name string, value *string) ([]byte, error) { + if value == nil { + return nil, fmt.Errorf("no %s in response: absent or null", name) + } + return decodeHexField(name, *value) +} + +// optionalHexField decodes a `bytes` field whose absence is the empty default. +// +// os_image_hash and mr_aggregated are the two: reading a missing key as empty +// keeps a degraded Info readable rather than unparseable, and costs nothing +// because neither field means anything unattested. An explicit null is still a +// malformed value, not an omission -- which is why this takes the raw JSON: a +// *string is nil for both, and the two do not mean the same thing. +func optionalHexField(name string, raw json.RawMessage) ([]byte, error) { + if len(raw) == 0 { + return []byte{}, nil + } + if bytes.Equal(bytes.TrimSpace(raw), []byte("null")) { + return nil, fmt.Errorf("malformed %s in response: expected a hex string, got null", name) + } + var value string + if err := json.Unmarshal(raw, &value); err != nil { + return nil, fmt.Errorf("malformed %s in response: %w", name, err) + } + return decodeHexField(name, value) +} + +// optionalString reads a scalar `string` field, defaulting an absent one. +// +// proto3 has no presence for a scalar string, so absence is the empty default +// -- but a null is a value the agent did not send, and Rust, Python and +// JavaScript all refuse it. Raw rather than *string for the same reason +// optionalHexField is: a pointer is nil for both, and the two differ. +func optionalString(name string, raw json.RawMessage) (string, error) { + if len(raw) == 0 { + return "", nil + } + if bytes.Equal(bytes.TrimSpace(raw), []byte("null")) { + return "", fmt.Errorf("malformed %s in response: expected a string, got null", name) + } + var value string + if err := json.Unmarshal(raw, &value); err != nil { + return "", fmt.Errorf("malformed %s in response: %w", name, err) + } + return value, nil +} + +// requireString reads a string field the response is meaningless without. +// +// A bundle's vendor and format are what a caller switches on to pick a +// verifier, so a missing one does not degrade the answer -- it routes the +// evidence to no verifier at all. +func requireString(name string, value *string) (string, error) { + if value == nil { + return "", fmt.Errorf("missing %s in response", name) + } + return *value, nil +} + +// decodeBundleList reads a repeated GpuEvidenceBundle field from its raw JSON. +// +// emptyWhenAbsent follows the proto: a missing boottime_gpu_evidence is the +// empty list, a missing bundles is a malformed response. A null is malformed +// either way -- absence is an omission, null is a value, and a caller that +// reads null as "no GPUs" has been told something the agent did not say. +func decodeBundleList(name string, raw json.RawMessage, emptyWhenAbsent bool) ([]gpuEvidenceBundleJSON, error) { + if len(raw) == 0 { + if emptyWhenAbsent { + return []gpuEvidenceBundleJSON{}, nil + } + return nil, fmt.Errorf("missing %s in response", name) + } + if bytes.Equal(bytes.TrimSpace(raw), []byte("null")) { + return nil, fmt.Errorf("malformed %s in response: expected a list, got null", name) + } + var wire []gpuEvidenceBundleJSON + if err := json.Unmarshal(raw, &wire); err != nil { + return nil, fmt.Errorf("malformed %s in response: %w", name, err) + } + return wire, nil +} + +// rpcError reports the agent's own error message when it arrives with a 200. +// +// The transport already rejects a non-2xx status. This covers the other shape: +// a body that carries an error where the answer should be. Without it the +// fields simply come back absent, and before requireHexField that was +// indistinguishable from success. +func rpcError(data []byte) error { + var probe struct { + Error *string `json:"error"` + } + // A body that is not an object at all is the caller's decode to complain + // about, with the field names to say what was missing. + if err := json.Unmarshal(data, &probe); err != nil { + return nil + } + if probe.Error != nil { + return fmt.Errorf("%s", *probe.Error) + } + return nil +} + // Wire form of GpuEvidenceBundle, identical under `bundles` and under // `boottime_gpu_evidence`. type gpuEvidenceBundleJSON struct { - Vendor string `json:"vendor"` - Format string `json:"format"` - Evidence string `json:"evidence"` + Vendor *string `json:"vendor"` + Format *string `json:"format"` + Evidence *string `json:"evidence"` } // decodeGpuEvidenceBundles decodes one repeated GpuEvidenceBundle field, naming @@ -177,11 +290,19 @@ type gpuEvidenceBundleJSON struct { func decodeGpuEvidenceBundles(name string, wire []gpuEvidenceBundleJSON) ([]GpuEvidenceBundle, error) { bundles := make([]GpuEvidenceBundle, len(wire)) for i, bundle := range wire { - evidence, err := decodeHexField(fmt.Sprintf("evidence of %s element %d", name, i), bundle.Evidence) + vendor, err := requireString(fmt.Sprintf("vendor of %s element %d", name, i), bundle.Vendor) + if err != nil { + return nil, err + } + format, err := requireString(fmt.Sprintf("format of %s element %d", name, i), bundle.Format) if err != nil { return nil, err } - bundles[i] = GpuEvidenceBundle{Vendor: bundle.Vendor, Format: bundle.Format, Evidence: evidence} + evidence, err := requireHexField(fmt.Sprintf("evidence of %s element %d", name, i), bundle.Evidence) + if err != nil { + return nil, err + } + bundles[i] = GpuEvidenceBundle{Vendor: vendor, Format: format, Evidence: evidence} } return bundles, nil } @@ -300,6 +421,9 @@ func (c *DstackClientV1) IssueCert(ctx context.Context, options ...IssueCertV1Op if err != nil { return nil, err } + if err := rpcError(data); err != nil { + return nil, err + } var response IssueCertV1Response if err := json.Unmarshal(data, &response); err != nil { @@ -332,27 +456,34 @@ func (c *DstackClientV1) GetKey(ctx context.Context, domain string, algorithm st return nil, err } + if err := rpcError(data); err != nil { + return nil, err + } + var response struct { - Key string `json:"key"` - PublicKey string `json:"public_key"` - SignatureChain []string `json:"signature_chain"` + Key *string `json:"key"` + PublicKey *string `json:"public_key"` + SignatureChain *[]*string `json:"signature_chain"` } if err := json.Unmarshal(data, &response); err != nil { return nil, err } - key, err := decodeHexField("key", response.Key) + key, err := requireHexField("key", response.Key) if err != nil { return nil, err } - publicKey, err := decodeHexField("public_key", response.PublicKey) + publicKey, err := requireHexField("public_key", response.PublicKey) if err != nil { return nil, err } + if response.SignatureChain == nil { + return nil, fmt.Errorf("no signature_chain in response: absent or null") + } - chain := make([][]byte, len(response.SignatureChain)) - for i, link := range response.SignatureChain { - chain[i], err = decodeHexField(fmt.Sprintf("signature chain element %d", i), link) + chain := make([][]byte, len(*response.SignatureChain)) + for i, link := range *response.SignatureChain { + chain[i], err = requireHexField(fmt.Sprintf("signature chain element %d", i), link) if err != nil { return nil, err } @@ -385,20 +516,33 @@ func (c *DstackClientV1) Attest(ctx context.Context, reportData []byte, includeB return nil, err } + if err := rpcError(data); err != nil { + return nil, err + } + var response struct { - Attestation string `json:"attestation"` - BoottimeGpuEvidence []gpuEvidenceBundleJSON `json:"boottime_gpu_evidence"` + Attestation *string `json:"attestation"` + // Raw, because a pointer cannot tell an absent field from an explicit + // null and the two differ here: absence is the empty list, since the + // field is only populated when the request asked for it, while a null + // is a malformed value. Rust draws the same line with + // `#[serde(default)]`, which fills a missing key and rejects a null. + BoottimeGpuEvidence json.RawMessage `json:"boottime_gpu_evidence"` } if err := json.Unmarshal(data, &response); err != nil { return nil, err } - attestation, err := decodeHexField("attestation", response.Attestation) + attestation, err := requireHexField("attestation", response.Attestation) if err != nil { return nil, err } - boottimeGpuEvidence, err := decodeGpuEvidenceBundles("boottime_gpu_evidence", response.BoottimeGpuEvidence) + wire, err := decodeBundleList("boottime_gpu_evidence", response.BoottimeGpuEvidence, true) + if err != nil { + return nil, err + } + boottimeGpuEvidence, err := decodeGpuEvidenceBundles("boottime_gpu_evidence", wire) if err != nil { return nil, err } @@ -429,14 +573,25 @@ func (c *DstackClientV1) AttestGpu(ctx context.Context, nonce []byte) (*AttestGp return nil, err } + if err := rpcError(data); err != nil { + return nil, err + } + var response struct { - Bundles []gpuEvidenceBundleJSON `json:"bundles"` + Bundles json.RawMessage `json:"bundles"` } if err := json.Unmarshal(data, &response); err != nil { return nil, err } - bundles, err := decodeGpuEvidenceBundles("bundles", response.Bundles) + // Required, unlike boottime_gpu_evidence: bundles is the whole answer of + // this call, so an absent one is a malformed response rather than a host + // with no GPUs. + wire, err := decodeBundleList("bundles", response.Bundles, false) + if err != nil { + return nil, err + } + bundles, err := decodeGpuEvidenceBundles("bundles", wire) if err != nil { return nil, err } @@ -455,46 +610,78 @@ func (c *DstackClientV1) Info(ctx context.Context) (*InfoV1Response, error) { return nil, err } + if err := rpcError(data); err != nil { + return nil, err + } + var response struct { - AppID string `json:"app_id"` - AppName string `json:"app_name"` - ComposeHash string `json:"compose_hash"` - AppCompose string `json:"app_compose"` - InstanceID string `json:"instance_id"` - DeviceID string `json:"device_id"` - OsImageHash string `json:"os_image_hash"` - MrAggregated string `json:"mr_aggregated"` - VmConfig string `json:"vm_config"` - KeyProviderInfo string `json:"key_provider_info"` - CloudVendor string `json:"cloud_vendor"` - CloudProduct string `json:"cloud_product"` + AppID *string `json:"app_id"` + AppName json.RawMessage `json:"app_name"` + ComposeHash *string `json:"compose_hash"` + AppCompose json.RawMessage `json:"app_compose"` + InstanceID *string `json:"instance_id"` + DeviceID *string `json:"device_id"` + OsImageHash json.RawMessage `json:"os_image_hash"` + MrAggregated json.RawMessage `json:"mr_aggregated"` + VmConfig json.RawMessage `json:"vm_config"` + KeyProviderInfo json.RawMessage `json:"key_provider_info"` + CloudVendor json.RawMessage `json:"cloud_vendor"` + CloudProduct json.RawMessage `json:"cloud_product"` } if err := json.Unmarshal(data, &response); err != nil { return nil, err } - info := &InfoV1Response{ - AppName: response.AppName, - AppCompose: response.AppCompose, - VmConfig: response.VmConfig, - KeyProviderInfo: response.KeyProviderInfo, - CloudVendor: response.CloudVendor, - CloudProduct: response.CloudProduct, + info := &InfoV1Response{} + for _, field := range []struct { + name string + raw json.RawMessage + into *string + }{ + {"app_name", response.AppName, &info.AppName}, + {"app_compose", response.AppCompose, &info.AppCompose}, + {"vm_config", response.VmConfig, &info.VmConfig}, + {"key_provider_info", response.KeyProviderInfo, &info.KeyProviderInfo}, + {"cloud_vendor", response.CloudVendor, &info.CloudVendor}, + {"cloud_product", response.CloudProduct, &info.CloudProduct}, + } { + value, err := optionalString(field.name, field.raw) + if err != nil { + return nil, err + } + *field.into = value } + // app_id, compose_hash, instance_id and device_id are what Info exists to + // answer, so absence is a malformed response. os_image_hash and + // mr_aggregated are read as empty when absent, which keeps a degraded Info + // readable and costs nothing, because neither means anything unattested. for _, field := range []struct { name string - value string + value *string into *[]byte }{ {"app_id", response.AppID, &info.AppID}, {"compose_hash", response.ComposeHash, &info.ComposeHash}, {"instance_id", response.InstanceID, &info.InstanceID}, {"device_id", response.DeviceID, &info.DeviceID}, + } { + decoded, err := requireHexField(field.name, field.value) + if err != nil { + return nil, err + } + *field.into = decoded + } + + for _, field := range []struct { + name string + raw json.RawMessage + into *[]byte + }{ {"os_image_hash", response.OsImageHash, &info.OsImageHash}, {"mr_aggregated", response.MrAggregated, &info.MrAggregated}, } { - decoded, err := decodeHexField(field.name, field.value) + decoded, err := optionalHexField(field.name, field.raw) if err != nil { return nil, err } @@ -514,6 +701,9 @@ func (c *DstackClientV1) Version(ctx context.Context) (*VersionV1Response, error if err != nil { return nil, err } + if err := rpcError(data); err != nil { + return nil, err + } var response VersionV1Response if err := json.Unmarshal(data, &response); err != nil { diff --git a/sdk/go/dstack/client_v1_test.go b/sdk/go/dstack/client_v1_test.go index a3e4d1281..49b78a0ce 100644 --- a/sdk/go/dstack/client_v1_test.go +++ b/sdk/go/dstack/client_v1_test.go @@ -466,10 +466,26 @@ func TestV0GetTlsKeyKeepsTheReleasedServerAuthDefault(t *testing.T) { // Version selection is by URL path alone: every v1 method must post under /v1. func TestV1MethodsPostUnderTheV1Prefix(t *testing.T) { paths := make(chan string, 1) + // One minimal valid body per method. A single body for all of them used to + // work because every required field decoded to empty without complaint, + // which is exactly the leniency this surface no longer has. + bodies := map[string]string{ + "/v1/IssueCert": `{"key":"-----BEGIN----","certificate_chain":[]}`, + "/v1/GetKey": `{"key":"aa","public_key":"bb","signature_chain":[]}`, + "/v1/Attest": `{"attestation":"cc"}`, + "/v1/AttestGpu": `{"bundles":[]}`, + "/v1/Info": `{"app_id":"11","compose_hash":"22","instance_id":"33",` + + `"device_id":"44"}`, + "/v1/Version": `{"version":"0.6.0","rev":"deadbeef"}`, + } server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { paths <- r.URL.Path w.Header().Set("Content-Type", "application/json") - _, _ = w.Write([]byte(`{"bundles":[]}`)) + body, ok := bodies[r.URL.Path] + if !ok { + body = `{}` + } + _, _ = w.Write([]byte(body)) })) defer server.Close() @@ -541,3 +557,167 @@ func TestV1AgainstAnAgentWithoutV1(t *testing.T) { t.Errorf("expected the status to be reported, got: %v", err) } } + +// A response that omits a required `bytes` field, or nulls it, is malformed -- +// not a valid answer whose value happens to be empty. +// +// Decoding into a string made these indistinguishable: absent and null both +// became "", "" hex-decodes to empty bytes, and the error was nil. A caller +// then read an empty private key, or an empty app_id, as the agent's answer. +// Rust, Python and JavaScript all refuse these bodies. +func TestV1RejectsAbsentAndNullFields(t *testing.T) { + cases := []struct { + name string + body string + call func(ctx context.Context, c *dstack.DstackClientV1) error + want string + }{ + { + "key absent", + `{"public_key":"bb","signature_chain":[]}`, + func(ctx context.Context, c *dstack.DstackClientV1) error { + _, err := c.GetKey(ctx, "d", "ed25519") + return err + }, + "key", + }, + { + "key null", + `{"key":null,"public_key":"bb","signature_chain":[]}`, + func(ctx context.Context, c *dstack.DstackClientV1) error { + _, err := c.GetKey(ctx, "d", "ed25519") + return err + }, + "key", + }, + { + "signature_chain absent", + `{"key":"aa","public_key":"bb"}`, + func(ctx context.Context, c *dstack.DstackClientV1) error { + _, err := c.GetKey(ctx, "d", "ed25519") + return err + }, + "signature_chain", + }, + { + "signature_chain null element", + `{"key":"aa","public_key":"bb","signature_chain":["aabb",null]}`, + func(ctx context.Context, c *dstack.DstackClientV1) error { + _, err := c.GetKey(ctx, "d", "ed25519") + return err + }, + "signature chain element 1", + }, + { + "attestation absent", + `{}`, + func(ctx context.Context, c *dstack.DstackClientV1) error { + _, err := c.Attest(ctx, []byte("x"), false) + return err + }, + "attestation", + }, + { + // The whole answer of the call, so absence is malformed rather + // than a host with no GPUs. + "bundles absent", + `{}`, + func(ctx context.Context, c *dstack.DstackClientV1) error { + _, err := c.AttestGpu(ctx, bytes.Repeat([]byte{0}, 32)) + return err + }, + "bundles", + }, + { + "app_id absent", + `{"compose_hash":"22","instance_id":"33","device_id":"44"}`, + func(ctx context.Context, c *dstack.DstackClientV1) error { + _, err := c.Info(ctx) + return err + }, + "app_id", + }, + { + // Absent is the empty default for this one; an explicit null is a + // value the agent did not send. + "os_image_hash null", + `{"app_id":"11","compose_hash":"22","instance_id":"33","device_id":"44","os_image_hash":null}`, + func(ctx context.Context, c *dstack.DstackClientV1) error { + _, err := c.Info(ctx) + return err + }, + "os_image_hash", + }, + { + // A bundle's vendor is what a caller switches on to pick a + // verifier, so an absent one routes evidence nowhere. + "bundle vendor absent", + `{"bundles":[{"format":"f","evidence":"aa"}]}`, + func(ctx context.Context, c *dstack.DstackClientV1) error { + _, err := c.AttestGpu(ctx, bytes.Repeat([]byte{0}, 32)) + return err + }, + "vendor of bundles element 0", + }, + } + + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + w.Header().Set("Content-Type", "application/json") + _, _ = w.Write([]byte(tc.body)) + })) + defer server.Close() + + client := dstack.NewDstackClientV1(dstack.WithEndpoint(server.URL)) + err := tc.call(context.Background(), client) + if err == nil { + t.Fatalf("expected an error naming %s, got none", tc.want) + } + if !strings.Contains(err.Error(), tc.want) { + t.Errorf("expected the error to name %s, got: %v", tc.want, err) + } + }) + } +} + +// An absent optional field is the empty default, not an error: a degraded Info +// stays readable, and neither field means anything unattested anyway. +func TestV1ReadsAbsentOptionalBytesAsEmpty(t *testing.T) { + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + w.Header().Set("Content-Type", "application/json") + _, _ = w.Write([]byte(`{"app_id":"11","compose_hash":"22",` + + `"instance_id":"33","device_id":"44"}`)) + })) + defer server.Close() + + info, err := dstack.NewDstackClientV1(dstack.WithEndpoint(server.URL)).Info(context.Background()) + if err != nil { + t.Fatalf("an older agent's Info should still parse: %v", err) + } + if len(info.OsImageHash) != 0 || len(info.MrAggregated) != 0 { + t.Errorf("expected empty bytes, got %x and %x", info.OsImageHash, info.MrAggregated) + } +} + +// An error body arriving with a 200 is an error, not an empty answer. +// +// The transport rejects a non-2xx status, which left this shape: the agent's +// own message where the answer should be. Every required field is then absent, +// and before this the call returned a zero-length private key and a nil error. +func TestV1SurfacesAnErrorBodySentWith200(t *testing.T) { + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + w.Header().Set("Content-Type", "application/json") + _, _ = w.Write([]byte(`{"error":"boom"}`)) + })) + defer server.Close() + + _, err := dstack.NewDstackClientV1(dstack.WithEndpoint(server.URL)).GetKey( + context.Background(), "d", "ed25519") + if err == nil { + t.Fatal("expected the agent's error to surface, got none") + } + if !strings.Contains(err.Error(), "boom") { + t.Errorf("expected the agent's own words, got: %v", err) + } +} diff --git a/sdk/js/README.md b/sdk/js/README.md index 7706f2cd6..61295d8e4 100644 --- a/sdk/js/README.md +++ b/sdk/js/README.md @@ -29,7 +29,7 @@ const client = new DstackClient() const key = await client.getKey('storage-encryption', 'secp256k1') console.log(Buffer.from(key.key).toString('hex')) -const { attestation } = await client.attest('app-state-snapshot') +const { attestation } = await client.attest(Buffer.from('app-state-snapshot')) console.log(attestation) ``` @@ -107,21 +107,23 @@ Both arguments are required. `algorithm` is exactly `'secp256k1'` or `'ed25519'` The only CVM attestation entry point. The versioned attestation already carries the TDX quote and the event log, so there is no separate `getQuote`. ```typescript -const { attestation } = await client.attest('app-state-snapshot') +const { attestation } = await client.attest(Buffer.from('app-state-snapshot')) // Uint8Array ``` -`reportData` is 1 to 64 bytes (string, Buffer, or Uint8Array), zero-padded on the right by the agent. Pass `true` as the second argument to also return the boot-time GPU evidence in `boottime_gpu_evidence`, so a verifier gets both in one round trip: +Every field the proto declares `bytes` is a `Uint8Array` here, hex only on the wire. + +`reportData` is 1 to 64 bytes — a `Buffer` or `Uint8Array`, never a string — zero-padded on the right by the agent. v0 accepted a string and UTF-8 encoded it, so `attest('deadbeef')` committed to eight ASCII characters rather than the four bytes they spell; v1 makes you say which you meant. `attestGpu`'s nonce is bytes only for the same reason, where a 32-character string would have passed the length check. Pass `true` as the second argument to also return the boot-time GPU evidence in `boottime_gpu_evidence`, so a verifier gets both in one round trip: ```typescript -const { attestation, boottime_gpu_evidence } = await client.attest('snapshot', true) +const { attestation, boottime_gpu_evidence } = await client.attest(Buffer.from('snapshot'), true) for (const bundle of boottime_gpu_evidence) { - console.log(bundle.vendor, bundle.format, bundle.decodeEvidence()) + console.log(bundle.vendor, bundle.format, bundle.evidence) } ``` `boottime_gpu_evidence` is a list of the same `GpuEvidenceBundleV1` objects `attestGpu` returns, so one parser serves both; `format` is what tells them apart (`nvidia-nvattest-boottime-json-v1` here, `nvidia-nvattest-collect-evidence-json-v1` there). Absence is the empty list, not a sentinel: it is empty unless the flag was set and the guest has boot-time output. -That evidence is not bound to `reportData` — nvattest ran at boot against its own nonce. Bind it by replaying the runtime event log and comparing sha256 of the bytes `decodeEvidence()` returns — exactly the bytes nvattest emitted, so do not parse and re-serialize the JSON — against `evidence_sha256` in the measured `gpu-attestation` event. +That evidence is not bound to `reportData` — nvattest ran at boot against its own nonce. Bind it by replaying the runtime event log and comparing sha256 of `bundle.evidence` — exactly the bytes nvattest emitted, so do not parse and re-serialize the JSON — against `evidence_sha256` in the measured `gpu-attestation` event. ### `attestGpu(nonce)` @@ -130,13 +132,13 @@ Collect GPU evidence now, against a 32-byte nonce you choose. This answers "is t ```typescript const { bundles } = await client.attestGpu(crypto.randomBytes(32)) for (const bundle of bundles) { - console.log(bundle.vendor, bundle.format, bundle.decodeEvidence()) + console.log(bundle.vendor, bundle.format, bundle.evidence) } ``` The nonce must be exactly 32 bytes — SPDM fixes the length, and dstack applies no transform, so you can compare these bytes directly against the `eat_nonce` claim. Hash a longer challenge yourself. -Select a verifier from each bundle's `vendor` and `format`, then check the signature, certificate chain, measurements and embedded nonce. `evidence` is opaque and hex-encoded on the wire; `decodeEvidence()` gives the vendor's bytes verbatim. It does not by itself bind the GPU to this CVM. +Select a verifier from each bundle's `vendor` and `format`, then check the signature, certificate chain, measurements and embedded nonce. `evidence` is opaque, hex-encoded on the wire and decoded here to the vendor's bytes verbatim. It does not by itself bind the GPU to this CVM. ### `info()` @@ -144,14 +146,14 @@ App identity and configuration. Not attestation. ```typescript const info = await client.info() -info.app_id // hex +info.app_id // Uint8Array info.app_name -info.compose_hash // hex — sha256 over exactly the app_compose bytes +info.compose_hash // Uint8Array — sha256 over exactly the app_compose bytes info.app_compose // the deployed document, verbatim -info.instance_id // hex -info.device_id // hex — identifies the host machine, not this instance -info.os_image_hash // hex -info.mr_aggregated // hex +info.instance_id // Uint8Array +info.device_id // Uint8Array — identifies the host machine, not this instance +info.os_image_hash // Uint8Array +info.mr_aggregated // Uint8Array info.vm_config // JSON owned by the VMM info.key_provider_info // JSON owned by dstack-util info.cloud_vendor // e.g. "Google" @@ -401,7 +403,7 @@ Given a `GetTlsKeyResponse` both helpers hash the PEM key with SHA-256 first, wh | `new DstackClientV0()` | `new DstackClient()` | | `client.getTlsKey({ subject })` | `client.issueCert({ subject })` | | `client.getKey(path, purpose)` | `client.getKey(domain, algorithm)` — **different key material** | -| `client.getQuote(data)` | `client.attest(data)` | +| `client.getQuote(data)` | `client.attest(bytes)` — **bytes only**, where v0 UTF-8 encoded a string | | `client.sign(...)` / `client.verify(...)` | sign and verify locally with the key from `getKey` | | `client.emitEvent(...)` | bind the data through `report_data` on `attest()` | | `info.tcb_info.*` | `attest()`, which returns measurements quote-backed | @@ -417,7 +419,7 @@ Migrate a surface at a time: both clients can talk to the same agent at once, so | --- | --- | | `new TappdClient()` | `new DstackClient()` — or `new DstackClientV0()` to keep the same key material | | `client.deriveKey(path, subject)` | `client.issueCert({ subject })` | -| `client.tdxQuote(data)` | `client.attest(data)` | +| `client.tdxQuote(data)` | `client.attest(bytes)` — **bytes only**, where v0 UTF-8 encoded a string | | `/var/run/tappd.sock` | `/var/run/dstack.sock` | ## License diff --git a/sdk/js/src/__tests__/index-v1.test.ts b/sdk/js/src/__tests__/index-v1.test.ts index 030a1fa0c..38005dc2d 100644 --- a/sdk/js/src/__tests__/index-v1.test.ts +++ b/sdk/js/src/__tests__/index-v1.test.ts @@ -158,15 +158,16 @@ describe('DstackClientV1', () => { describe('attest', () => { it('should attest over report data', async () => { const client = new DstackClientV1() - const result = await client.attest('test') - expect(result.attestation).not.toBe('') + const result = await client.attest(Buffer.from('test')) + expect(result.attestation).toBeInstanceOf(Uint8Array) + expect(result.attestation.length).toBeGreaterThan(0) expect(result.boottime_gpu_evidence).toEqual([]) }) it('should accept the boot-time GPU evidence flag', async () => { const client = new DstackClientV1() - const result = await client.attest('test', true) - expect(result.attestation).not.toBe('') + const result = await client.attest(Buffer.from('test'), true) + expect(result.attestation.length).toBeGreaterThan(0) // Absence is the empty list, not a sentinel; the simulator has no GPU // output, so this is empty here but must still be an array. expect(Array.isArray(result.boottime_gpu_evidence)).toBe(true) @@ -175,19 +176,39 @@ describe('DstackClientV1', () => { it('should type boot-time evidence as the bundle list attestGpu returns', async () => { const client = new DstackClientV1() - const result = await client.attest('test', true) + const result = await client.attest(Buffer.from('test'), true) // Assigning one to the other is the assertion: one parser, both methods. const bundles: GpuEvidenceBundleV1[] = result.boottime_gpu_evidence for (const bundle of bundles) { - expect(bundle.decodeEvidence()).toBeInstanceOf(Uint8Array) + expect(bundle.evidence).toBeInstanceOf(Uint8Array) } }) it('should reject report data outside 1..64 bytes', async () => { const client = new DstackClientV1() - await expect(() => client.attest('')).rejects.toThrow('must not be empty') + await expect(() => client.attest(new Uint8Array(0))).rejects.toThrow('must not be empty') await expect(() => client.attest(Buffer.alloc(65))).rejects.toThrow('at most 64 bytes') }) + + it('should reject a string rather than attest its characters', async () => { + const client = new DstackClientV1() + // v0 UTF-8 encoded this. `attest('deadbeef')` then committed to eight + // ASCII characters rather than the four bytes they spell, silently. + await expect( + () => (client as unknown as { + attest: (d: unknown) => Promise + }).attest('deadbeef') + ).rejects.toThrow(/report data must be bytes, not a string/) + }) + + it('should reject a 32-character string nonce that would pass the length check', async () => { + const client = new DstackClientV1() + await expect( + () => (client as unknown as { + attestGpu: (n: unknown) => Promise + }).attestGpu('a'.repeat(32)) + ).rejects.toThrow(/nonce must be bytes, not a string/) + }) }) describe('attestGpu', () => { @@ -246,25 +267,147 @@ describe('DstackClientV1', () => { it('should decode boot-time evidence to the nvattest bytes verbatim', async () => { await withStubAgent(async client => { - const result = await client.attest('test', true) + const result = await client.attest(Buffer.from('test'), true) const [bundle] = result.boottime_gpu_evidence expect(bundle.vendor).toBe('nvidia') expect(bundle.format).toBe('nvidia-nvattest-boottime-json-v1') // Byte-exact: sha256 over these bytes is what `evidence_sha256` in the // measured `gpu-attestation` event commits to. - expect(Buffer.from(bundle.decodeEvidence()).toString('utf8')).toBe(nvattest_output) + expect(Buffer.from(bundle.evidence).toString('utf8')).toBe(nvattest_output) }) }) it('should hand both methods the same bundle shape', async () => { await withStubAgent(async client => { - const attested = await client.attest('test', true) + const attested = await client.attest(Buffer.from('test'), true) const collected = await client.attestGpu(new Uint8Array(32)) const boottime: GpuEvidenceBundleV1 = attested.boottime_gpu_evidence[0] const on_demand: GpuEvidenceBundleV1 = collected.bundles[0] // Only `format` separates them, so one parser handles both. expect(on_demand.format).toBe('nvidia-nvattest-collect-evidence-json-v1') - expect(on_demand.decodeEvidence()).toEqual(boottime.decodeEvidence()) + expect(on_demand.evidence).toEqual(boottime.evidence) + }) + }) + }) + + // Every one of these decoded to a shorter-than-asked-for Uint8Array with no + // error before the hex decoding was made strict: Node's decoder stops at the + // first pair it cannot parse and returns the prefix. A truncated `key` or a + // signature chain quietly one link short is a verification failure nobody can + // trace back to its cause. + describe('malformed hex from the agent', () => { + async function withAgentAnswering(body: unknown, fn: (client: DstackClientV1) => Promise) { + const server = http.createServer((_req, res) => { + res.writeHead(200, { 'Content-Type': 'application/json' }) + res.end(JSON.stringify(body)) + }) + await new Promise(resolve => server.listen(0, '127.0.0.1', () => resolve())) + try { + const { port } = server.address() as AddressInfo + await fn(new DstackClientV1(`http://127.0.0.1:${port}`)) + } finally { + await new Promise(resolve => server.close(() => resolve())) + } + } + + const identity = { + app_id: 'aa'.repeat(20), + compose_hash: 'bb'.repeat(32), + instance_id: 'cc'.repeat(20), + device_id: 'dd'.repeat(32), + os_image_hash: 'ee'.repeat(32), + mr_aggregated: 'ff'.repeat(48), + } + + it('should reject a non-hex character rather than truncate', async () => { + await withAgentAnswering({ ...identity, app_id: 'aabbzz' + 'aa'.repeat(17) }, client => + expect(client.info()).rejects.toThrow(/malformed app_id/) + ) + }) + + it('should reject an odd-length string rather than drop a digit', async () => { + await withAgentAnswering({ ...identity, compose_hash: 'abc' }, client => + expect(client.info()).rejects.toThrow(/malformed compose_hash/) + ) + }) + + it('should name the chain link that is malformed', async () => { + await withAgentAnswering( + { key: 'aa'.repeat(32), public_key: 'bb'.repeat(33), signature_chain: ['aabb', 'qq'] }, + client => expect(client.getKey('x', 'secp256k1')).rejects.toThrow(/signature_chain\[1\]/) + ) + }) + + it('should reject an absent required field instead of returning empty bytes', async () => { + const { instance_id: _dropped, ...without_instance_id } = identity + await withAgentAnswering(without_instance_id, client => + expect(client.info()).rejects.toThrow(/no instance_id/) + ) + }) + + it('should accept an absent optional field as empty', async () => { + const { os_image_hash: _a, mr_aggregated: _b, ...older_agent } = identity + await withAgentAnswering(older_agent, async client => { + const info = await client.info() + expect(info.os_image_hash).toEqual(new Uint8Array(0)) + expect(info.mr_aggregated).toEqual(new Uint8Array(0)) + expect(info.app_id.length).toBe(20) + }) + }) + + // A hex check alone does not survive a value that is not a string: + // `RegExp.test` stringifies, so `['00112233']` passes it, and `Buffer.from` + // then coerces the element as an octet and yields one attacker-chosen byte + // without erroring. These pin the type check that stops it. + it('should reject a one-element array rather than coerce it to a byte', async () => { + await withAgentAnswering({ ...identity, app_id: ['00112233'] }, client => + expect(client.info()).rejects.toThrow(/malformed app_id.*got an array/) + ) + }) + + it('should reject a chain link that is an array rather than a string', async () => { + await withAgentAnswering( + { key: 'aa'.repeat(32), public_key: 'bb'.repeat(33), signature_chain: [['61']] }, + client => expect(client.getKey('x', 'secp256k1')).rejects.toThrow( + /malformed signature_chain\[0\].*got an array/) + ) + }) + + it('should reject a null field rather than read it as empty bytes', async () => { + await withAgentAnswering({ ...identity, os_image_hash: null }, client => + expect(client.info()).rejects.toThrow(/malformed os_image_hash.*got null/) + ) + }) + + it('should name the field when a repeated one is not a list', async () => { + await withAgentAnswering( + { key: 'aa'.repeat(32), public_key: 'bb'.repeat(33), signature_chain: null }, + client => expect(client.getKey('x', 'secp256k1')).rejects.toThrow( + /malformed signature_chain: expected a list/) + ) + }) + + it('should reject an absent bundles rather than report no GPUs', async () => { + await withAgentAnswering({}, client => + expect(client.attestGpu(new Uint8Array(32))).rejects.toThrow( + /malformed bundles: expected a list/) + ) + }) + + it('should reject a bundle without the vendor a caller dispatches on', async () => { + await withAgentAnswering( + { bundles: [{ format: 'nvidia-nvattest-collect-evidence-json-v1', evidence: 'aa' }] }, + client => expect(client.attestGpu(new Uint8Array(32))).rejects.toThrow( + /malformed bundles\[0\]\.vendor.*got nothing/) + ) + }) + + it('should read an absent string field as empty, not undefined', async () => { + await withAgentAnswering(identity, async client => { + const info = await client.info() + expect(info.app_compose).toBe('') + expect(info.cloud_vendor).toBe('') + expect(info.app_name).toBe('') }) }) }) @@ -280,8 +423,8 @@ describe('DstackClientV1', () => { ]) { expect(result).toHaveProperty(field) } - expect(result.app_id).not.toBe('') - expect(result.instance_id).not.toBe('') + expect(result.app_id.length).toBeGreaterThan(0) + expect(result.instance_id.length).toBeGreaterThan(0) }) it('should not nest measurements in a tcb_info blob or mint an app_cert', async () => { @@ -291,12 +434,18 @@ describe('DstackClientV1', () => { expect(result.app_cert).toBeUndefined() }) - it('should hex-encode the byte fields', async () => { + it('should decode the byte fields rather than hand back hex', async () => { const client = new DstackClientV1() const result = await client.info() - expect(result.app_id).toMatch(/^[0-9a-f]+$/) - expect(result.compose_hash).toMatch(/^[0-9a-f]{64}$/) - expect(result.mr_aggregated).toMatch(/^[0-9a-f]{64}$/) + // The proto says `bytes`, so the SDK says `Uint8Array`: a caller that + // hashes or compares an identity gets the 20 or 32 bytes it means, not + // the 40 or 64 ASCII characters the wire carries. + expect(result.app_id).toBeInstanceOf(Uint8Array) + expect(result.app_id.length).toBe(20) + expect(result.compose_hash.length).toBe(32) + expect(result.instance_id.length).toBe(20) + expect(result.device_id.length).toBe(32) + expect(result.mr_aggregated.length).toBe(32) }) it('should serve app_compose verbatim rather than nested in another JSON string', async () => { diff --git a/sdk/js/src/client-v1.ts b/sdk/js/src/client-v1.ts index 6e77d1bd1..2c8c2825d 100644 --- a/sdk/js/src/client-v1.ts +++ b/sdk/js/src/client-v1.ts @@ -6,7 +6,75 @@ // unsuffixed `DstackClient` names since 0.6.0. import { send_rpc_request } from './send-rpc-request' -import { to_hex, throwOnRpcError, resolveDstackEndpoint, type Hex } from './shared' +import { to_hex, throwOnRpcError, resolveDstackEndpoint } from './shared' + +/** An even number of hex digits, and nothing else. */ +const HEX_ONLY = /^(?:[0-9a-fA-F]{2})*$/ + +/** + * Decode a wire hex string, or say which field was malformed. + * + * Strict on purpose. Node's hex decoder stops at the first pair it cannot + * parse and returns the prefix it managed, without error: `Buffer.from( + * '0102zz', 'hex')` is two bytes, and an odd-length string loses its last + * digit. These fields are private keys, signature chain links and application + * identity -- handing back a silently truncated one is worse than throwing, + * and Rust, Python and Go all refuse the same input. + */ +function decode_hex(value: unknown, field: string): Uint8Array { + // The type check is not redundant with the regex, and dropping it is a + // silent-wrong-value bug rather than a style regression. `RegExp.test` + // stringifies its argument, so a one-element array passes -- `['00112233']` + // becomes `'00112233'` -- and `Buffer.from` then ignores the `'hex'` + // argument for a non-string input and coerces the elements as octets: + // `Number('00112233') & 0xff`, one attacker-chosen byte, no error. TypeScript + // cannot stop this because a JSON response is `any` at runtime. + if (typeof value !== 'string') { + throw new Error( + `the agent returned a malformed ${field}: expected a hex string, got ${ + value === null ? 'null' : Array.isArray(value) ? 'an array' : typeof value}` + ) + } + if (!HEX_ONLY.test(value)) { + throw new Error( + `the agent returned a malformed ${field}: expected an even-length hex string` + ) + } + return new Uint8Array(Buffer.from(value, 'hex')) +} + +/** + * Decode a `bytes` field the proto declares required. + * + * Absence is an error rather than the empty default: `app_id` and `key` are + * answers the agent always has, so a response without one is a response that + * did not come from a working agent. An empty *string* still decodes to zero + * bytes, which is what every other SDK does with it. + */ +function from_hex(value: unknown, field: string): Uint8Array { + if (value === undefined) { + throw new Error(`the agent returned no ${field}`) + } + return decode_hex(value, field) +} + +/** + * Decode a `bytes` field, treating an absent key as the empty default. + * + * `os_image_hash` and `mr_aggregated` are the two that get this. Both are + * plain `bytes` in the proto, so a current agent always sends them -- empty + * when it could not compute one. Reading a missing key as those same empty + * bytes rather than an error keeps a degraded `Info` readable instead of + * unparseable, and costs nothing, because neither field means anything + * unattested anyway. + * + * Absent only. An explicit `null` is a malformed value, not an omission, and + * is rejected -- which is also what `#[serde(default)]` does in Rust and what + * a defaulted pydantic field does in Python. + */ +function from_optional_hex(value: unknown, field: string): Uint8Array { + return value === undefined ? new Uint8Array(0) : decode_hex(value, field) +} export interface IssueCertOptionsV1 { subject?: string; @@ -52,7 +120,7 @@ export interface GetKeyResponseV1 { export interface AttestResponseV1 { __name__: Readonly<'AttestResponseV1'> - attestation: Hex + attestation: Uint8Array /** * The GPU evidence nvattest recorded at boot, in the same bundle shape @@ -61,8 +129,8 @@ export interface AttestResponseV1 { * absence is the empty array, not a sentinel. * * Not bound to `report_data`: nvattest ran at boot against its own nonce. - * Bind it by replaying the runtime event log and comparing sha256 of the - * bytes `decodeEvidence()` returns against `evidence_sha256` in the measured + * Bind it by replaying the runtime event log and comparing sha256 of each + * bundle's `evidence` against `evidence_sha256` in the measured * `gpu-attestation` event. */ boottime_gpu_evidence: GpuEvidenceBundleV1[] @@ -86,20 +154,21 @@ export interface GpuEvidenceBundleV1 { vendor: string /** Vendor-specific evidence format and version. */ format: string - /** Opaque vendor-native evidence bytes, hex-encoded by the JSON RPC. */ - evidence: Hex - /** - * The evidence as raw bytes, exactly as the vendor emitted it. + * Opaque vendor-native evidence bytes, exactly as the vendor emitted them + * (hex-encoded by the JSON RPC, decoded here). * * Byte-exact by design: for a boot-time bundle the binding rule is sha256 * over precisely these bytes, compared against `evidence_sha256` in the * measured `gpu-attestation` event, so parsing and re-serialising the JSON * breaks the comparison. */ - decodeEvidence: () => Uint8Array + evidence: Uint8Array } +/** A bundle as it arrives, before `evidence` is decoded. */ +type GpuEvidenceBundleV1Wire = Omit & { evidence: string } + export interface AttestGpuResponseV1 { __name__: Readonly<'AttestGpuResponseV1'> @@ -115,32 +184,52 @@ export interface AttestGpuResponseV1 { * attestation. * * `app_id`, `compose_hash`, `instance_id`, `device_id`, `os_image_hash` and - * `mr_aggregated` are lowercase hex; the rest are plain strings, with the three - * document fields carrying JSON owned by someone else (see `docs/guest-api-v1.md`). + * `mr_aggregated` are `bytes` in the proto and `Uint8Array` here, lowercase hex + * only on the wire; the rest are plain strings, with the three document fields + * carrying JSON owned by someone else (see `docs/guest-api-v1.md`). */ export interface InfoResponseV1 { __name__: Readonly<'InfoResponseV1'> - app_id: Hex + app_id: Uint8Array app_name: string - compose_hash: Hex + compose_hash: Uint8Array /** * The app-compose document, verbatim. `compose_hash` is sha256 over exactly * these bytes, so do not parse and re-serialize before hashing: key order, * whitespace and unknown fields all change the digest. */ app_compose: string - instance_id: Hex + instance_id: Uint8Array /** Identifies the host machine, not this instance. */ - device_id: Hex - os_image_hash: Hex - mr_aggregated: Hex + device_id: Uint8Array + os_image_hash: Uint8Array + mr_aggregated: Uint8Array vm_config: string key_provider_info: string cloud_vendor: string cloud_product: string } +/** + * `InfoResponseV1` as it arrives: identity fields hex, everything else final. + * + * `os_image_hash` and `mr_aggregated` are optional here rather than on the + * wire: the proto declares them plain `bytes` and the agent always sends them, + * but a response that omits one is read as empty rather than rejected. + */ +type InfoResponseV1Wire = + Omit + & { + app_id: string + compose_hash: string + instance_id: string + device_id: string + os_image_hash?: string + mr_aggregated?: string + } + export interface VersionResponseV1 { __name__: Readonly<'VersionResponseV1'> @@ -149,18 +238,109 @@ export interface VersionResponseV1 { } /** - * Attach the byte accessor to the bundles a v1 RPC returned. + * Reject anything but bytes on a request path. + * + * v0 took a `string` here and UTF-8 encoded it, which reads as a convenience + * until the string is a hex digest: `attest('deadbeef')` committed to the + * eight ASCII characters rather than the four bytes they spell, with nothing + * to say so. v1 refuses the ambiguity, and the runtime check is not redundant + * with the parameter type -- a JavaScript caller, or TypeScript with an `any` + * from `JSON.parse`, reaches this with a string and no compiler in the way. + */ +function require_bytes(value: unknown, param: string): Buffer | Uint8Array { + if (typeof value === 'string') { + throw new Error( + `${param} must be bytes, not a string: use Buffer.from(value, 'utf8') to ` + + `attest the text, or Buffer.from(value, 'hex') if it is a hex digest` + ) + } + if (!(value instanceof Uint8Array)) { + throw new Error(`${param} must be a Buffer or Uint8Array`) + } + return value +} + +/** + * Read a `string` field, or say which one was not a string. + * + * The interfaces in this file declare these `string`, and a spread of the raw + * JSON quietly hands back `undefined` for a key the response omitted -- so + * `info().app_compose.length` throws on a value the compiler called safe. + * proto3 has no presence for a scalar `string`, so absence is the empty + * default, matching Rust's `#[serde(default)]` and pydantic's field default. + * A present-but-not-a-string value is still an error. + */ +function to_string(value: unknown, field: string): string { + if (value === undefined) { + return '' + } + return require_string(value, field) +} + +/** + * Read a `string` field the response is meaningless without. + * + * A bundle's `vendor` and `format` are what a caller dispatches on to pick a + * verifier, so handing back `undefined` there does not degrade the answer, it + * routes the evidence to no verifier at all -- quietly, since `undefined` + * matches no `case`. Rust and Python both make these required. + */ +function require_string(value: unknown, field: string): string { + if (typeof value !== 'string') { + throw new Error( + `the agent returned a malformed ${field}: expected a string, got ${ + value === undefined ? 'nothing' + : value === null ? 'null' + : Array.isArray(value) ? 'an array' : typeof value}` + ) + } + return value +} + +/** + * Read a `repeated` field, or say which one was not a list. + * + * `Array.isArray` rather than a truthiness check: a bare `.map()` on a `null` + * or absent field throws `TypeError: Cannot read properties of null`, which + * names no field and reads like an SDK bug rather than a bad response. + * + * `whenAbsent` follows the proto. A missing `boottime_gpu_evidence` is the + * empty list, because the field is only populated when asked for; a missing + * `bundles` or `signature_chain` is a malformed response, because those are + * the whole answer of the call that returns them. + */ +function to_list( + value: unknown, field: string, whenAbsent: 'empty' | 'error', +): unknown[] { + if (value === undefined && whenAbsent === 'empty') { + return [] + } + if (!Array.isArray(value)) { + throw new Error(`the agent returned a malformed ${field}: expected a list`) + } + return value +} + +/** + * Decode the bundles a v1 RPC returned. * * Shared by `attest` and `attestGpu` so both hand back the same object shape, * which is the point of the wire message being shared. */ function to_gpu_evidence_bundles( - bundles: Array> | undefined, + bundles: unknown, field: string, whenAbsent: 'empty' | 'error', ): GpuEvidenceBundleV1[] { - return (bundles ?? []).map(bundle => Object.freeze({ - ...bundle, - decodeEvidence: () => new Uint8Array(Buffer.from(bundle.evidence, 'hex')), - })) + return to_list(bundles, field, whenAbsent).map((bundle, i) => { + if (bundle === null || typeof bundle !== 'object') { + throw new Error(`the agent returned a malformed ${field}[${i}]: expected an object`) + } + const wire = bundle as GpuEvidenceBundleV1Wire + return Object.freeze({ + vendor: require_string(wire.vendor, `${field}[${i}].vendor`), + format: require_string(wire.format, `${field}[${i}].format`), + evidence: from_hex(wire.evidence, `${field}[${i}].evidence`), + }) + }) } /** @@ -227,7 +407,9 @@ export class DstackClientV1 { this.endpoint, '/v1/IssueCert', JSON.stringify(raw)) throwOnRpcError(result) return Object.freeze({ - ...result, + key: to_string(result.key, 'key'), + certificate_chain: to_list(result.certificate_chain, 'certificate_chain', 'error') + .map((cert, i) => to_string(cert, `certificate_chain[${i}]`)), __name__: 'IssueCertResponseV1' as const, }) } @@ -253,9 +435,10 @@ export class DstackClientV1 { this.endpoint, '/v1/GetKey', payload) throwOnRpcError(result) return Object.freeze({ - key: new Uint8Array(Buffer.from(result.key, 'hex')), - public_key: new Uint8Array(Buffer.from(result.public_key, 'hex')), - signature_chain: result.signature_chain.map(sig => new Uint8Array(Buffer.from(sig, 'hex'))), + key: from_hex(result.key, 'key'), + public_key: from_hex(result.public_key, 'public_key'), + signature_chain: to_list(result.signature_chain, 'signature_chain', 'error') + .map((link, i) => from_hex(link, `signature_chain[${i}]`)), __name__: 'GetKeyResponseV1' as const, }) } @@ -266,32 +449,36 @@ export class DstackClientV1 { * The only CVM attestation entry point in v1: the attestation already carries * the TDX quote and the event log, so there is no separate `getQuote`. * - * @param report_data 1 to 64 bytes, zero-padded on the right to 64 by the agent. + * @param report_data 1 to 64 bytes, zero-padded on the right to 64 by the + * agent. Bytes only: v0 also took a string and UTF-8 encoded it, which + * silently attested the characters of a hex digest rather than the digest. * @param include_boottime_gpu_evidence Also return the boot-time GPU evidence, * as the same {@link GpuEvidenceBundleV1} list `attestGpu` returns, so a * verifier gets both in one round trip. It is not bound to `report_data`. */ async attest( - report_data: string | Buffer | Uint8Array, + report_data: Buffer | Uint8Array, include_boottime_gpu_evidence: boolean = false, ): Promise { - const hex = to_hex(report_data) - if (hex.length === 0) { + const bytes = require_bytes(report_data, 'report data') + if (bytes.length === 0) { throw new Error('report data must not be empty') } - if (hex.length > 128) { - throw new Error(`report data must be at most 64 bytes, but received ${hex.length / 2}`) + if (bytes.length > 64) { + throw new Error(`report data must be at most 64 bytes, but received ${bytes.length}`) } + const hex = to_hex(bytes) const payload = JSON.stringify({ report_data: hex, include_boottime_gpu_evidence }) const result = await send_rpc_request<{ attestation: string, - boottime_gpu_evidence?: Array>, + boottime_gpu_evidence?: GpuEvidenceBundleV1Wire[], }>(this.endpoint, '/v1/Attest', payload) throwOnRpcError(result) return Object.freeze({ __name__: 'AttestResponseV1' as const, - attestation: result.attestation as Hex, - boottime_gpu_evidence: to_gpu_evidence_bundles(result.boottime_gpu_evidence), + attestation: from_hex(result.attestation, 'attestation'), + boottime_gpu_evidence: to_gpu_evidence_bundles( + result.boottime_gpu_evidence, 'boottime_gpu_evidence', 'empty'), }) } @@ -304,29 +491,43 @@ export class DstackClientV1 { * bind the GPU to this CVM. * * @param nonce Exactly 32 bytes, passed to the GPU verbatim. SPDM fixes the - * length; hash a longer challenge yourself. + * length; hash a longer challenge yourself. Bytes only, for the same reason + * `attest` takes bytes only -- and a 32-character string would have passed + * the length check. */ async attestGpu(nonce: Buffer | Uint8Array): Promise { - if (nonce.length !== 32) { - throw new Error(`nonce must be exactly 32 bytes, but received ${nonce.length}`) + const bytes = require_bytes(nonce, 'nonce') + if (bytes.length !== 32) { + throw new Error(`nonce must be exactly 32 bytes, but received ${bytes.length}`) } - const payload = JSON.stringify({ nonce: to_hex(nonce) }) + const payload = JSON.stringify({ nonce: to_hex(bytes) }) const result = await send_rpc_request<{ - bundles?: Array>, + bundles?: GpuEvidenceBundleV1Wire[], }>(this.endpoint, '/v1/AttestGpu', payload) throwOnRpcError(result) return Object.freeze({ - bundles: to_gpu_evidence_bundles(result.bundles), + bundles: to_gpu_evidence_bundles(result.bundles, 'bundles', 'error'), __name__: 'AttestGpuResponseV1' as const, }) } /** Return this application's identity and configuration. */ async info(): Promise { - const result = await send_rpc_request>(this.endpoint, '/v1/Info', '{}') + const result = await send_rpc_request(this.endpoint, '/v1/Info', '{}') throwOnRpcError(result) return Object.freeze({ - ...result, + app_id: from_hex(result.app_id, 'app_id'), + app_name: to_string(result.app_name, 'app_name'), + compose_hash: from_hex(result.compose_hash, 'compose_hash'), + instance_id: from_hex(result.instance_id, 'instance_id'), + device_id: from_hex(result.device_id, 'device_id'), + os_image_hash: from_optional_hex(result.os_image_hash, 'os_image_hash'), + mr_aggregated: from_optional_hex(result.mr_aggregated, 'mr_aggregated'), + app_compose: to_string(result.app_compose, 'app_compose'), + vm_config: to_string(result.vm_config, 'vm_config'), + key_provider_info: to_string(result.key_provider_info, 'key_provider_info'), + cloud_vendor: to_string(result.cloud_vendor, 'cloud_vendor'), + cloud_product: to_string(result.cloud_product, 'cloud_product'), __name__: 'InfoResponseV1' as const, }) } @@ -336,7 +537,8 @@ export class DstackClientV1 { const result = await send_rpc_request<{ version: string, rev: string }>(this.endpoint, '/v1/Version', '{}') throwOnRpcError(result) return Object.freeze({ - ...result, + version: to_string(result.version, 'version'), + rev: to_string(result.rev, 'rev'), __name__: 'VersionResponseV1' as const, }) } diff --git a/sdk/python/README.md b/sdk/python/README.md index 12b94f57d..9a67b3d6c 100644 --- a/sdk/python/README.md +++ b/sdk/python/README.md @@ -90,17 +90,22 @@ different keys for the same domain. ```python key = client.get_key('storage-encryption', 'secp256k1') -print(key.decode_key()) # 32 raw bytes -print(key.decode_public_key()) # SEC1 compressed (33 B), or 32 B for ed25519 -print(key.decode_signature_chain()) # two links: app root, then KMS root +print(key.key) # 32 raw bytes +print(key.public_key) # SEC1 compressed (33 B), or 32 B for ed25519 +print(key.signature_chain) # two links: app root, then KMS root ``` +Every field the proto declares `bytes` is `bytes` here, hex only on the wire. +`public_key` in particular is what the v1 key claim commits to, and the claim is +built over raw bytes — passing a hex string would build it over 66 ASCII +characters and produce a chain that silently never verifies. + **Parameters:** - `domain`: Any string. This replaces v0's `path` plus `purpose`; in v0 only `path` reached the KDF and `purpose` was merely echoed into the chain claim. Derivation is flat — two domains give unrelated keys, and `a/b` is not a child of `a`. - `algorithm`: Exactly `'secp256k1'` or `'ed25519'`. **Required.** There is no default and no `k256` alias, because in v0 a typo silently produced a key of the wrong type under a name the caller thought meant something else. An empty value is rejected client-side. -**Returns:** `GetKeyResponseV1` with hex `key`, `public_key` and -`signature_chain`, plus the usual `decode_*` helpers. +**Returns:** `GetKeyResponseV1` with `key: bytes`, `public_key: bytes` and +`signature_chain: list[bytes]`. ### `attest()` @@ -108,13 +113,19 @@ The sole CVM attestation entry point in v1. The dstack attestation format already carries the quote and the event log, so v0's TDX-only `get_quote` has nothing left to add. +`report_data` is bytes, never `str`. v0 accepted a `str` and UTF-8 encoded it, +so `attest("deadbeef")` committed to the eight ASCII characters rather than the +four bytes they spell, with no error to say so; v1 makes you write +`value.encode()` or `bytes.fromhex(value)` at the call site, where it is visible +which one was meant. `attest_gpu`'s nonce is bytes only for the same reason. + ```python result = client.attest(b'user:alice:nonce123') -print(result.decode_attestation()) +print(result.attestation) # bytes with_gpu = client.attest(b'user:alice:nonce123', include_boottime_gpu_evidence=True) for bundle in with_gpu.boottime_gpu_evidence: - print(bundle.vendor, bundle.format, bundle.decode_evidence()) + print(bundle.vendor, bundle.format, bundle.evidence) ``` `report_data` is 1–64 bytes and is zero-padded on the right to 64. @@ -126,10 +137,10 @@ against `attest_gpu()`'s `'nvidia-nvattest-collect-evidence-json-v1'`. Boot-time evidence is *not* bound to `report_data` — nvattest ran at boot against its own nonce. Bind it by replaying the runtime event log and comparing -sha256 of `decode_evidence()` against `evidence_sha256` in the measured -`gpu-attestation` event. `decode_evidence()` returns the nvattest output byte -for byte; do not parse and re-serialize before hashing, since key order and -whitespace change the digest. +sha256 of `bundle.evidence` against `evidence_sha256` in the measured +`gpu-attestation` event. `evidence` is the nvattest output byte for byte; do not +parse and re-serialize before hashing, since key order and whitespace change the +digest. ### `attest_gpu()` @@ -140,7 +151,7 @@ Use it after anything that may have reinitialised the GPU. ```python result = client.attest_gpu(os.urandom(32)) # exactly 32 bytes for bundle in result.bundles: - print(bundle.vendor, bundle.format, bundle.decode_evidence()) + print(bundle.vendor, bundle.format, bundle.evidence) ``` Select a verifier using each bundle's `vendor` and `format`, then check the @@ -194,6 +205,9 @@ this is, and a relying party still confirms them against an attestation. `app_compose` is the verbatim deployed document and `compose_hash` is sha256 over exactly those bytes — do not parse and re-serialize before hashing. +`app_id`, `instance_id`, `compose_hash`, `device_id`, `os_image_hash` and +`mr_aggregated` are `bytes`, matching the proto. Call `.hex()` to print one. + ### `version()` ```python @@ -239,7 +253,7 @@ standard library. from cryptography.hazmat.primitives.asymmetric import ed25519 key = client.get_key('signing/messages', 'ed25519') -signing_key = ed25519.Ed25519PrivateKey.from_private_bytes(key.decode_key()) +signing_key = ed25519.Ed25519PrivateKey.from_private_bytes(key.key) signature = signing_key.sign(b'message to sign') ``` diff --git a/sdk/python/src/dstack_sdk/dstack_client_v1.py b/sdk/python/src/dstack_sdk/dstack_client_v1.py index 55ccdd0fb..e57c8bba5 100644 --- a/sdk/python/src/dstack_sdk/dstack_client_v1.py +++ b/sdk/python/src/dstack_sdk/dstack_client_v1.py @@ -17,17 +17,77 @@ """ import binascii +import re +from typing import Annotated from typing import Any from typing import Dict from typing import List from typing import Optional from pydantic import BaseModel +from pydantic import BeforeValidator +from pydantic import PlainSerializer from .dstack_client_v0 import AsyncBaseClient from .dstack_client_v0 import BaseClient from .dstack_client_v0 import call_async +#: An even number of hex digits, and nothing else. +_HEX_ONLY = re.compile(r"\A(?:[0-9a-fA-F]{2})*\Z") + + +def _require_bytes(value: Any, param: str) -> bytes: + """Reject anything but bytes on a request path. + + v0 accepted a ``str`` here and UTF-8 encoded it, which reads as a + convenience until the string is a hex digest: ``attest("deadbeef")`` + committed to the eight ASCII characters rather than the four bytes they + spell, with nothing to say so, and a 32-character nonce passed the length + check on its way to attesting the wrong value. v1 refuses the ambiguity and + names the two ways out, because only the caller knows which was meant. + """ + if isinstance(value, str): + raise TypeError( + f"{param} must be bytes, not str: use value.encode() to attest the " + f"text, or bytes.fromhex(value) if it is a hex digest" + ) + if not isinstance(value, (bytes, bytearray, memoryview)): + raise TypeError(f"{param} must be bytes, not {type(value).__name__}") + return bytes(value) + + +def _decode_hex(value: Any) -> Any: + r"""Turn a wire hex string into bytes, leaving anything else to pydantic. + + The regex is not redundant with ``bytes.fromhex``: that helper skips ASCII + whitespace, so ``"aa bb"`` and ``"aa\nbb"`` decode happily while the error + message here promises they do not. Rust and Go reject both, and a field + one SDK accepts and another refuses is a field verifiers cannot rely on. + """ + if isinstance(value, str): + if not _HEX_ONLY.match(value): + raise ValueError(f"expected an even-length hex string, got {value!r}") + return bytes.fromhex(value) + return value + + +#: A protobuf ``bytes`` field: lowercase hex on the wire, ``bytes`` in Python. +#: +#: v0 exposed these as ``str`` with a ``decode_*`` helper beside them, which +#: made the wrong call the easy one -- ``public_key`` passed straight to a +#: signature-chain claim builder is 66 ASCII characters, not a 33-byte key, and +#: nothing raises until the chain fails to verify. The annotation also replaces +#: pydantic's own ``str`` -> ``bytes`` coercion, which would UTF-8 encode the +#: hex string rather than decode it -- the same silent wrong answer. +#: +#: Serialization is hex only in JSON mode, so ``model_dump_json()`` round-trips +#: through the wire form while ``model_dump()`` keeps the bytes. +HexBytes = Annotated[ + bytes, + BeforeValidator(_decode_hex), + PlainSerializer(lambda value: value.hex(), return_type=str, when_used="json"), +] + class IssueCertResponseV1(BaseModel): # PEM-encoded private key, freshly generated for this call. It is not @@ -39,18 +99,14 @@ class IssueCertResponseV1(BaseModel): class GetKeyResponseV1(BaseModel): - key: str - public_key: str - signature_chain: List[str] - - def decode_key(self) -> bytes: - return bytes.fromhex(self.key) - - def decode_public_key(self) -> bytes: - return bytes.fromhex(self.public_key) - - def decode_signature_chain(self) -> List[bytes]: - return [bytes.fromhex(chain) for chain in self.signature_chain] + # 32 bytes for both algorithms. + key: HexBytes + # SEC1 compressed (33 bytes) for secp256k1, raw (32 bytes) for ed25519. + # This is the exact byte string the chain's first link commits to. + public_key: HexBytes + # Two links: the app root key's signature over the v1 key claim, then the + # KMS root key's signature over the app root public key. + signature_chain: List[HexBytes] class GpuEvidenceBundleV1(BaseModel): @@ -68,34 +124,26 @@ class GpuEvidenceBundleV1(BaseModel): vendor: str format: str - # Opaque vendor-native evidence, hex-encoded on the wire. Do not assume - # UTF-8 or JSON. - evidence: str - - def decode_evidence(self) -> bytes: - """Return the evidence as raw bytes, exactly as the vendor emitted it. - - The exactness matters for the boot-time format: the binding rule is - sha256 over precisely these bytes, compared against ``evidence_sha256`` - in the measured ``gpu-attestation`` event. Parsing and re-serializing - the JSON changes key order and whitespace, and so changes the digest. - """ - return bytes.fromhex(self.evidence) + # Opaque vendor-native evidence, hex-encoded on the wire and exactly as the + # vendor emitted it. Do not assume UTF-8 or JSON. + # + # The exactness matters for the boot-time format: the binding rule is + # sha256 over precisely these bytes, compared against ``evidence_sha256`` + # in the measured ``gpu-attestation`` event. Parsing and re-serializing the + # JSON changes key order and whitespace, and so changes the digest. + evidence: HexBytes class AttestResponseV1(BaseModel): - attestation: str + attestation: HexBytes # The GPU evidence nvattest recorded during guest boot, in the same bundle # shape attest_gpu returns. Empty unless the request set # include_boottime_gpu_evidence and the guest has boot-time output. Not # bound to report_data: verify each bundle by replaying the runtime event - # log and comparing sha256 of decode_evidence() against evidence_sha256 in - # the `gpu-attestation` event. + # log and comparing sha256 of its `evidence` against evidence_sha256 in the + # `gpu-attestation` event. boottime_gpu_evidence: List[GpuEvidenceBundleV1] = [] - def decode_attestation(self) -> bytes: - return bytes.fromhex(self.attestation) - class AttestGpuResponseV1(BaseModel): """Result of fresh, on-demand GPU evidence collection.""" @@ -118,22 +166,26 @@ class InfoResponseV1(BaseModel): they identify *which* application and image this is, and a relying party still confirms them against an attestation. - The identity fields are hex strings, matching how the v0 ``InfoResponse`` - exposes the same values. + The identity fields are ``bytes``, matching the proto, where the v0 + ``InfoResponse`` hands back hex strings. Call ``.hex()`` to print one. """ - app_id: str + app_id: HexBytes app_name: str = "" - compose_hash: str + compose_hash: HexBytes # Verbatim deployed bytes; compose_hash is sha256 over exactly these. Do # not parse and re-serialize before hashing -- key order, whitespace and # unknown fields all change the digest, and that digest is what gets # whitelisted on chain. app_compose: str = "" - instance_id: str - device_id: str - os_image_hash: str - mr_aggregated: str + instance_id: HexBytes + # Identifies the host machine, not this instance. + device_id: HexBytes + # Plain `bytes` in the proto, so the agent always sends these -- empty when + # it could not compute one. A response that omits one is read as those same + # empty bytes rather than rejected, as Rust's `#[serde(default)]` does. + os_image_hash: HexBytes = b"" + mr_aggregated: HexBytes = b"" vm_config: str = "" key_provider_info: str = "" cloud_vendor: str = "" @@ -202,7 +254,7 @@ async def get_key(self, domain: str, algorithm: str) -> GetKeyResponseV1: async def attest( self, - report_data: str | bytes, + report_data: bytes, include_boottime_gpu_evidence: bool = False, ) -> AttestResponseV1: """Produce a versioned attestation over the given report data. @@ -211,16 +263,21 @@ async def attest( format already carries the quote and the event log, so v0's TDX-only ``get_quote`` has nothing left to add. + ``report_data`` is bytes. v0 also accepted a ``str`` and UTF-8 encoded + it, which reads as a convenience until the string is a hex digest: + ``attest("deadbeef")`` committed to the eight ASCII characters rather + than the four bytes they spell, with no error to say so. v1 refuses the + ambiguity -- encode the text or decode the hex at the call site, where + it is visible which one was meant. + Set include_boottime_gpu_evidence to also return the boot-time GPU attestation evidence in ``AttestResponseV1.boottime_gpu_evidence``, as the same ``GpuEvidenceBundleV1`` list ``attest_gpu`` returns. A guest with no boot-time output returns an empty list. """ - if not report_data or not isinstance(report_data, (bytes, str)): + report_bytes = _require_bytes(report_data, "report_data") + if not report_bytes: raise ValueError("report_data can not be empty") - report_bytes: bytes = ( - report_data.encode() if isinstance(report_data, str) else report_data - ) if len(report_bytes) > 64: raise ValueError("report_data must be at most 64 bytes") data: Dict[str, Any] = { @@ -239,10 +296,11 @@ async def attest_gpu(self, nonce: bytes) -> AttestGpuResponseV1: compared directly against the ``eat_nonce`` claim; to bind a longer challenge, hash it yourself. """ - if not isinstance(nonce, (bytes, bytearray)) or len(nonce) != 32: + nonce_bytes = _require_bytes(nonce, "nonce") + if len(nonce_bytes) != 32: raise ValueError("nonce must be exactly 32 bytes") result = await self._send_rpc_request( - "AttestGpu", {"nonce": binascii.hexlify(bytes(nonce)).decode()} + "AttestGpu", {"nonce": binascii.hexlify(nonce_bytes).decode()} ) return AttestGpuResponseV1(**result) @@ -299,7 +357,7 @@ def get_key(self, domain: str, algorithm: str) -> GetKeyResponseV1: @call_async def attest( self, - report_data: str | bytes, + report_data: bytes, include_boottime_gpu_evidence: bool = False, ) -> AttestResponseV1: """Produce a versioned attestation over the given report data.""" diff --git a/sdk/python/src/dstack_sdk/ethereum.py b/sdk/python/src/dstack_sdk/ethereum.py index 609607e57..060bafb97 100644 --- a/sdk/python/src/dstack_sdk/ethereum.py +++ b/sdk/python/src/dstack_sdk/ethereum.py @@ -6,7 +6,7 @@ Use with ``dstack_sdk.DstackClientV0`` responses to create ``eth_account`` objects for signing and transacting. These helpers take the v0 response models; -for a v1 key, hand ``GetKeyResponseV1.decode_key()`` to ``Account.from_key`` +for a v1 key, hand ``GetKeyResponseV1.key`` to ``Account.from_key`` yourself. """ diff --git a/sdk/python/src/dstack_sdk/solana.py b/sdk/python/src/dstack_sdk/solana.py index aca1c5896..a33fdc953 100644 --- a/sdk/python/src/dstack_sdk/solana.py +++ b/sdk/python/src/dstack_sdk/solana.py @@ -6,7 +6,7 @@ Use with ``dstack_sdk.DstackClientV0`` responses to create ``solders.Keypair`` objects for signing transactions on Solana. These helpers take the v0 response -models; for a v1 key, hand ``GetKeyResponseV1.decode_key()`` to +models; for a v1 key, hand ``GetKeyResponseV1.key`` to ``Keypair.from_seed`` yourself. """ diff --git a/sdk/python/tests/test_client_v1.py b/sdk/python/tests/test_client_v1.py index ab405bd33..941e325c9 100644 --- a/sdk/python/tests/test_client_v1.py +++ b/sdk/python/tests/test_client_v1.py @@ -4,6 +4,7 @@ from typing import List +from pydantic import ValidationError import pytest from dstack_sdk import AsyncDstackClient @@ -100,12 +101,13 @@ async def test_async_v1_info(): def check_info_response(result: InfoResponseV1): assert isinstance(result, InfoResponseV1) - assert len(result.app_id) == 40 - assert len(result.compose_hash) == 64 - assert len(result.instance_id) == 40 - assert len(result.device_id) == 64 - assert len(result.os_image_hash) in (0, 64) - assert len(result.mr_aggregated) == 64 + # bytes, not hex: the lengths are the raw ones the proto declares. + assert len(result.app_id) == 20 + assert len(result.compose_hash) == 32 + assert len(result.instance_id) == 20 + assert len(result.device_id) == 32 + assert len(result.os_image_hash) in (0, 32) + assert len(result.mr_aggregated) == 32 assert len(result.app_compose) > 0 # The measurement registers and the event log belong to attest(), which # returns them quote-backed. They must not reappear here. @@ -117,17 +119,17 @@ def test_sync_v1_get_key(): client = DstackClientV1() result = client.get_key("storage-encryption", "secp256k1") assert isinstance(result, GetKeyResponseV1) - assert len(result.decode_key()) == 32 + assert len(result.key) == 32 # secp256k1 public keys are SEC1 compressed, and the chain's first link # commits to exactly these bytes. - assert len(result.decode_public_key()) == 33 - assert len(result.decode_signature_chain()) == 2 + assert len(result.public_key) == 33 + assert len(result.signature_chain) == 2 ed = client.get_key("storage-encryption", "ed25519") - assert len(ed.decode_key()) == 32 - assert len(ed.decode_public_key()) == 32 + assert len(ed.key) == 32 + assert len(ed.public_key) == 32 # The v1 KDF binds the algorithm, so one name no longer serves two curves. - assert ed.decode_key() != result.decode_key() + assert ed.key != result.key @pytest.mark.asyncio @@ -160,13 +162,13 @@ def test_v1_keys_differ_from_v0_keys(): """No compatibility mode: the same name yields different key material.""" v0 = DstackClientV0().get_key("storage-encryption", "") v1 = DstackClientV1().get_key("storage-encryption", "secp256k1") - assert v1.decode_key() != v0.decode_key() + assert v1.key != v0.decode_key() def test_sync_v1_attest(): result = DstackClientV1().attest(b"user:alice:nonce123") assert isinstance(result, AttestResponseV1) - assert len(result.decode_attestation()) > 0 + assert len(result.attestation) > 0 @pytest.mark.asyncio @@ -191,7 +193,7 @@ async def fake_send(self, method, payload): monkeypatch.setenv("DSTACK_SIMULATOR_ENDPOINT", "http://localhost:0") monkeypatch.setattr(AsyncDstackClientV1, "_send_rpc_request", fake_send) result = await AsyncDstackClientV1().attest( - "test", include_boottime_gpu_evidence=True + b"test", include_boottime_gpu_evidence=True ) assert isinstance(result, AttestResponseV1) bundle = result.boottime_gpu_evidence[0] @@ -199,7 +201,7 @@ async def fake_send(self, method, payload): assert bundle.format == "nvidia-nvattest-boottime-json-v1" # sha256 of exactly these bytes is what the `gpu-attestation` event commits # to, so the decode must be byte-for-byte, not a re-serialized parse. - assert bundle.decode_evidence() == evidence + assert bundle.evidence == evidence def test_sync_v1_attest_boottime_gpu_evidence_defaults_to_empty(): @@ -222,7 +224,30 @@ async def test_async_v1_attest_report_data_bounds(): with pytest.raises(ValueError, match="64 bytes"): await client.attest(b"0" * 65) # 64 bytes is the maximum, not one past it. - assert len((await client.attest(b"0" * 64)).decode_attestation()) > 0 + assert len((await client.attest(b"0" * 64)).attestation) > 0 + + +@pytest.mark.asyncio +async def test_async_v1_attest_refuses_str(): + """v1 takes bytes, so a hex digest cannot be attested as its characters. + + v0 UTF-8 encoded a str, which made ``attest("deadbeef")`` commit to eight + ASCII characters rather than the four bytes they spell -- no error, just a + quote over the wrong value. The v0 client keeps that behaviour; v1 does not. + """ + client = AsyncDstackClientV1() + with pytest.raises(TypeError, match="bytes.fromhex"): + await client.attest("deadbeef") # type: ignore[arg-type] + with pytest.raises(TypeError, match="must be bytes, not int"): + await client.attest(42) # type: ignore[arg-type] + + +@pytest.mark.asyncio +async def test_async_v1_attest_gpu_refuses_str(): + """A 32-character str would otherwise pass the length check unnoticed.""" + client = AsyncDstackClientV1() + with pytest.raises(TypeError, match="must be bytes"): + await client.attest_gpu("a" * 32) # type: ignore[arg-type] @pytest.mark.asyncio @@ -248,16 +273,20 @@ async def fake_send(self, method, payload): assert isinstance(result, AttestGpuResponseV1) assert len(result.bundles) == 1 assert result.bundles[0].vendor == "nvidia" - assert result.bundles[0].decode_evidence() == evidence + assert result.bundles[0].evidence == evidence @pytest.mark.asyncio async def test_async_v1_attest_gpu_rejects_wrong_nonce_length(): """SPDM fixes the evidence nonce at 32 bytes; catch it before the round trip.""" client = AsyncDstackClientV1() - for bad in [b"", bytes(31), bytes(33), "not-bytes"]: + for bad in [b"", bytes(31), bytes(33)]: with pytest.raises(ValueError, match="32 bytes"): await client.attest_gpu(bad) + # A non-bytes nonce is a TypeError, not a length complaint -- see + # test_async_v1_attest_gpu_refuses_str. + with pytest.raises(TypeError): + await client.attest_gpu(object()) # type: ignore[arg-type] def test_sync_v1_attest_gpu_reaches_the_agent(): @@ -336,3 +365,48 @@ def test_v1_unix_socket_file_not_exist(monkeypatch): monkeypatch.delenv("DSTACK_SIMULATOR_ENDPOINT", raising=False) with pytest.raises(FileNotFoundError): DstackClientV1("/non/existent/socket") + + +# The wire is hex; anything else is a response no verifier should act on. Go +# names the field it could not decode and Rust refuses the string outright, so +# Python doing the same is what keeps a malformed answer malformed in every +# SDK rather than in three of four. +@pytest.mark.parametrize( + "bad", + [ + "aabbzz", # a non-hex pair + "abc", # odd length: the last digit has nowhere to go + "aa bb", # bytes.fromhex() skips whitespace; the wire never has any + "aa\nbb", + ], +) +def test_v1_malformed_hex_is_rejected(bad: str): + with pytest.raises(ValidationError): + GetKeyResponseV1(key=bad, public_key="bb" * 33, signature_chain=[]) + + +def test_v1_malformed_chain_link_is_rejected(): + """One bad link must fail the chain, not silently shorten it.""" + with pytest.raises(ValidationError): + GetKeyResponseV1( + key="aa" * 32, public_key="bb" * 33, signature_chain=["aabb", "qq"] + ) + + +def test_v1_absent_identity_hashes_are_empty(): + """`os_image_hash` and `mr_aggregated` default rather than reject. + + Both are plain `bytes` in the proto, so a current agent always sends them -- + empty when it could not compute one. Reading a response that omits one as + those same empty bytes keeps a degraded `Info` parseable instead of + unparseable, which is what Rust's `#[serde(default)]` already did. + """ + info = InfoResponseV1( + app_id="aa" * 20, + compose_hash="bb" * 32, + instance_id="cc" * 20, + device_id="dd" * 32, + ) + assert info.os_image_hash == b"" + assert info.mr_aggregated == b"" + assert info.app_id == b"\xaa" * 20 diff --git a/sdk/run-tests.sh b/sdk/run-tests.sh index 867619149..ff25c3755 100755 --- a/sdk/run-tests.sh +++ b/sdk/run-tests.sh @@ -19,7 +19,7 @@ trap simulator_stop EXIT INT TERM simulator_start pushd "$ROOT_DIR/rust" -cargo test -- --show-output +cargo test --workspace -- --show-output cargo run --example tappd_client_usage cargo run --example dstack_client_usage cargo test -p dstack-sdk-types --test no_std_test --no-default-features diff --git a/sdk/rust/Cargo.lock b/sdk/rust/Cargo.lock index e088cfe5d..15344c982 100644 --- a/sdk/rust/Cargo.lock +++ b/sdk/rust/Cargo.lock @@ -1993,6 +1993,9 @@ name = "hex" version = "0.4.3" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "7f24254aa9a54b5c858eaee2f5bccdb46aaf0e486a595ed5fd8f86ba55232a70" +dependencies = [ + "serde", +] [[package]] name = "hex-conservative" diff --git a/sdk/rust/README.md b/sdk/rust/README.md index 0c34f65ac..be2090248 100644 --- a/sdk/rust/README.md +++ b/sdk/rust/README.md @@ -78,10 +78,15 @@ let client = DstackClient::new(None); ```rust let key = client.get_key("storage-encryption", "secp256k1").await?; -let private_key = key.decode_key()?; // 32 bytes -let public_key = key.decode_public_key()?; // SEC1 compressed, or 32 raw for ed25519 +let private_key: Vec = key.key; // 32 bytes +let public_key: Vec = key.public_key; // SEC1 compressed, or 32 raw for ed25519 ``` +Every field the proto declares `bytes` is a `Vec` here, hex only on the +wire. `public_key` in particular is what the v1 key claim commits to, and the +claim is built over raw bytes -- handing it a hex string would build a claim +over 66 ASCII characters and produce a chain that silently never verifies. + `domain` is a caller-chosen domain-separation string -- not a DNS name and not a path. Derivation is **flat**: `a/b` is unrelated to `a`, and no key derives another. `algorithm` is required and must be `secp256k1` or `ed25519`; there is @@ -99,14 +104,14 @@ supported platform. ```rust let result = client.attest(b"custom data".to_vec(), true).await?; -let attestation = result.decode_attestation()?; +let attestation: Vec = result.attestation; // Boot-time GPU evidence uses the same bundle shape `attest_gpu` returns, so // one parser handles both. Empty when the flag was not set or the guest has no // GPU output -- absence is the empty list, not a sentinel. for bundle in &result.boottime_gpu_evidence { assert_eq!(bundle.format, dstack_sdk::dstack_client_v1::FORMAT_BOOTTIME); - let nvattest_output = bundle.decode_evidence()?; // exact bytes from disk + let nvattest_output = &bundle.evidence; // exact bytes from disk } ``` @@ -116,7 +121,7 @@ against a nonce you choose. A verifier for one does not appraise the other. That evidence is **not** bound to `report_data` -- nvattest ran at boot against its own nonce. Bind it by replaying the runtime event log and comparing sha256 -of the bundle's **exact** decoded bytes against `evidence_sha256` in the +of the bundle's **exact** `evidence` bytes against `evidence_sha256` in the `gpu-attestation` event. Parsing and re-serializing the JSON first changes the digest and breaks the comparison. @@ -166,6 +171,10 @@ and verify. `app_compose` is served directly here, rather than nested inside a `tcb_info` JSON string as v0 did, and `compose_hash` is sha256 over its verbatim bytes. +`app_id`, `instance_id`, `compose_hash`, `device_id`, `os_image_hash` and +`mr_aggregated` are `Vec`, matching the proto. Use `hex::encode` when you +want them printable. + #### `version() -> VersionResponse` Also the cheapest probe for whether an agent serves v1 at all. diff --git a/sdk/rust/src/dstack_client_v1.rs b/sdk/rust/src/dstack_client_v1.rs index ba44f82b6..efcc78236 100644 --- a/sdk/rust/src/dstack_client_v1.rs +++ b/sdk/rust/src/dstack_client_v1.rs @@ -117,7 +117,7 @@ impl DstackClientV1 { anyhow::bail!("report data must be 1 to 64 bytes") } let config = AttestConfig::builder() - .report_data(hex_encode(&report_data)) + .report_data(report_data) .include_boottime_gpu_evidence(include_boottime_gpu_evidence) .build(); let response = self.send_rpc_request("Attest", &config).await?; diff --git a/sdk/rust/tests/test_client_v1.rs b/sdk/rust/tests/test_client_v1.rs index 75386e872..76834a177 100644 --- a/sdk/rust/tests/test_client_v1.rs +++ b/sdk/rust/tests/test_client_v1.rs @@ -25,12 +25,12 @@ async fn get_key_returns_a_key_public_key_and_two_link_chain() { .unwrap(); // 32 raw bytes for both algorithms, hex-encoded on the wire. - assert_eq!(response.decode_key().unwrap().len(), 32); + assert_eq!(response.key.len(), 32); // The chain is the key's chain and nothing else: the claim link and the // KMS link. v0's `Sign` prepended the payload signature to its list, so // the real chain there started at index 1. assert_eq!(response.signature_chain.len(), 2); - assert_eq!(response.decode_signature_chain().unwrap()[0].len(), 65); + assert_eq!(response.signature_chain[0].len(), 65); } } @@ -40,17 +40,13 @@ async fn get_key_public_key_lengths_are_the_specified_ones() { .get_key("storage-encryption", "secp256k1") .await .unwrap(); - assert_eq!( - secp.decode_public_key().unwrap().len(), - 33, - "SEC1 compressed" - ); + assert_eq!(secp.public_key.len(), 33, "SEC1 compressed"); let ed = client() .get_key("storage-encryption", "ed25519") .await .unwrap(); - assert_eq!(ed.decode_public_key().unwrap().len(), 32); + assert_eq!(ed.public_key.len(), 32); } /// v1 has no default algorithm and no `k256` alias, so a caller cannot ask for @@ -105,13 +101,13 @@ async fn v1_keys_differ_from_v0_keys_for_the_same_name() { .get_key(Some("test".to_string()), Some("signing".to_string())) .await .unwrap(); - assert_ne!(v1.key, v0.key); + assert_ne!(v1.key, hex::decode(&v0.key).unwrap()); } #[tokio::test] async fn attest_returns_an_attestation() { let response = client().attest(b"test".to_vec(), false).await.unwrap(); - assert!(!response.decode_attestation().unwrap().is_empty()); + assert!(!response.attestation.is_empty()); assert!(response.boottime_gpu_evidence.is_empty()); } @@ -124,7 +120,7 @@ async fn attest_can_ask_for_the_boot_time_gpu_evidence() { // under test is that the flag is accepted on this surface at all -- it is // reserved on v0 -- and that the field decodes as a bundle list. let response = client().attest(b"test".to_vec(), true).await.unwrap(); - assert!(!response.decode_attestation().unwrap().is_empty()); + assert!(!response.attestation.is_empty()); let bundles: &Vec = &response.boottime_gpu_evidence; @@ -132,7 +128,7 @@ async fn attest_can_ask_for_the_boot_time_gpu_evidence() { for bundle in bundles { assert_eq!(bundle.vendor, "nvidia"); assert_eq!(bundle.format, dstack_sdk::dstack_client_v1::FORMAT_BOOTTIME); - assert!(bundle.decode_evidence().is_ok()); + assert!(!bundle.evidence.is_empty()); } } @@ -176,9 +172,9 @@ async fn attest_gpu_validates_the_nonce_length() { async fn info_reports_identity_and_configuration() { let info = client().info().await.unwrap(); - assert!(!info.decode_app_id().unwrap().is_empty()); - assert!(!info.decode_instance_id().unwrap().is_empty()); - assert_eq!(info.decode_compose_hash().unwrap().len(), 32); + assert!(!info.app_id.is_empty()); + assert!(!info.instance_id.is_empty()); + assert_eq!(info.compose_hash.len(), 32); // The app-compose document is served directly rather than nested inside a // `tcb_info` JSON string, which is what v0 did. assert!(info.app_compose.starts_with('{')); @@ -237,11 +233,13 @@ async fn derives_the_committed_key_vectors() { for (domain, algorithm, expected_key, expected_public_key) in vectors { let response = client().get_key(domain, algorithm).await.unwrap(); assert_eq!( - response.key, expected_key, + hex::encode(&response.key), + expected_key, "v1 key vector changed for ({domain:?}, {algorithm})" ); assert_eq!( - response.public_key, expected_public_key, + hex::encode(&response.public_key), + expected_public_key, "v1 public key vector changed for ({domain:?}, {algorithm})" ); } diff --git a/sdk/rust/types/Cargo.toml b/sdk/rust/types/Cargo.toml index d10d9f3ed..59bb9bf3b 100644 --- a/sdk/rust/types/Cargo.toml +++ b/sdk/rust/types/Cargo.toml @@ -15,7 +15,7 @@ authors = ["Encifher "] anyhow.workspace = true bon.workspace = true borsh = { workspace = true, optional = true } -hex = { workspace = true, features = ["alloc"] } +hex = { workspace = true, features = ["alloc", "serde"] } pkcs8 = { workspace = true, features = ["pem"] } serde.workspace = true serde_json = { workspace = true, features = ["alloc"] } diff --git a/sdk/rust/types/src/dstack_v1.rs b/sdk/rust/types/src/dstack_v1.rs index 040d14f59..917db972e 100644 --- a/sdk/rust/types/src/dstack_v1.rs +++ b/sdk/rust/types/src/dstack_v1.rs @@ -11,12 +11,18 @@ //! of the two lying about what the agent sent. //! //! Wire encoding follows the v0 convention: every protobuf `bytes` field is a -//! lowercase hex string in JSON, with a `decode_*` helper next to it. Fields -//! carrying JSON documents (`app_compose`, `vm_config`, `key_provider_info`, -//! `boottime_gpu_evidence`) are plain strings and are passed through unparsed. +//! lowercase hex string in JSON. The Rust types do not: a `bytes` field is a +//! `Vec`, and the hex lives in the serde layer. v0 exposed the hex string +//! and a `decode_*` helper beside it, which made the wrong call the easy one -- +//! `public_key` handed straight to a claim builder is 66 ASCII characters, not +//! a 33-byte key, and nothing objects until the chain fails to verify. Typing +//! the field as bytes makes that mistake unspellable. +//! +//! Fields carrying JSON documents (`app_compose`, `vm_config`, +//! `key_provider_info`, `boottime_gpu_evidence`) are plain strings and are +//! passed through unparsed. use alloc::{string::String, vec::Vec}; -use hex::FromHexError; use serde::{Deserialize, Serialize}; #[cfg(feature = "borsh_schema")] @@ -24,6 +30,33 @@ use borsh::BorshSchema; #[cfg(feature = "borsh")] use borsh::{BorshDeserialize, BorshSerialize}; +/// Serde for protobuf `repeated bytes`: a JSON array of lowercase hex strings. +/// +/// [`hex::serde`] covers a single `bytes` field, but there is no serde +/// attribute that composes it element-wise over a `Vec`, so the repeated case +/// needs its own module. Deserialization is all-or-nothing: one malformed +/// element fails the whole field rather than silently yielding a short chain. +mod hex_vec { + use alloc::{string::String, vec::Vec}; + use serde::{de::Error as _, Deserialize, Deserializer, Serialize, Serializer}; + + pub fn serialize(items: &[Vec], serializer: S) -> Result { + let encoded: Vec = items.iter().map(hex::encode).collect(); + encoded.serialize(serializer) + } + + pub fn deserialize<'de, D: Deserializer<'de>>( + deserializer: D, + ) -> Result>, D::Error> { + let encoded = Vec::::deserialize(deserializer)?; + encoded + .iter() + .map(hex::decode) + .collect::>, _>>() + .map_err(D::Error::custom) + } +} + /// Configuration for a certificate issuance request. #[derive(Debug, bon::Builder, Serialize, Deserialize)] #[cfg_attr(feature = "borsh", derive(BorshSerialize, BorshDeserialize))] @@ -75,44 +108,103 @@ pub struct IssueCertResponse { #[cfg_attr(feature = "borsh", derive(BorshSerialize, BorshDeserialize))] #[cfg_attr(feature = "borsh_schema", derive(BorshSchema))] pub struct GetKeyResponse { - /// The derived private key, hex-encoded. 32 bytes for both algorithms. - pub key: String, - /// The corresponding public key, hex-encoded. SEC1 compressed (33 bytes) - /// for secp256k1, raw (32 bytes) for ed25519. + /// The derived private key. 32 bytes for both algorithms. + #[serde(with = "hex::serde")] + pub key: Vec, + /// The corresponding public key. SEC1 compressed (33 bytes) for secp256k1, + /// raw (32 bytes) for ed25519. /// /// This is the exact byte string the chain's first link commits to, so a /// relying party never has to re-derive it from `key`. - pub public_key: String, - /// Two links, hex-encoded: the app root key's signature over the v1 key - /// claim, then the KMS root key's signature over the app root public key. + #[serde(with = "hex::serde")] + pub public_key: Vec, + /// Two links: the app root key's signature over the v1 key claim, then the + /// KMS root key's signature over the app root public key. /// /// `docs/guest-api-v1.md` specifies the claim encoding and the verification /// steps. Verifying is the relying party's job; this SDK does not do it. - pub signature_chain: Vec, + #[serde(with = "hex_vec")] + pub signature_chain: Vec>, } -impl GetKeyResponse { - pub fn decode_key(&self) -> Result, FromHexError> { - hex::decode(&self.key) +/// The value an attestation binds to: bytes, and only bytes. +/// +/// A newtype rather than a bare `Vec` so [`AttestConfig`]'s builder can +/// keep `#[builder(into)]` -- a `Vec`, an array and a slice all convert -- +/// while `&str` and `String` do not. Both of those implement `Into>`, +/// so on the bare type `.report_data("00ff")` compiles and attests the four +/// ASCII bytes of that string rather than the two bytes it spells. That is the +/// mistake this surface types its `bytes` fields to make unspellable, and the +/// one field a caller sends rather than receives should not be the exception. +/// +/// Length is the agent's to enforce; this type only fixes what the value is. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[cfg_attr(feature = "borsh", derive(BorshSerialize, BorshDeserialize))] +#[cfg_attr(feature = "borsh_schema", derive(BorshSchema))] +#[serde(transparent)] +pub struct ReportData(#[serde(with = "hex::serde")] pub Vec); + +impl From> for ReportData { + fn from(bytes: Vec) -> Self { + Self(bytes) } +} - pub fn decode_public_key(&self) -> Result, FromHexError> { - hex::decode(&self.public_key) +impl From<&[u8]> for ReportData { + fn from(bytes: &[u8]) -> Self { + Self(bytes.to_vec()) } +} - pub fn decode_signature_chain(&self) -> Result>, FromHexError> { - self.signature_chain.iter().map(hex::decode).collect() +impl From<[u8; N]> for ReportData { + fn from(bytes: [u8; N]) -> Self { + Self(bytes.to_vec()) + } +} + +impl From<&[u8; N]> for ReportData { + fn from(bytes: &[u8; N]) -> Self { + Self(bytes.to_vec()) + } +} + +impl core::ops::Deref for ReportData { + type Target = [u8]; + + fn deref(&self) -> &Self::Target { + &self.0 + } +} + +impl AsRef<[u8]> for ReportData { + fn as_ref(&self) -> &[u8] { + &self.0 } } /// Configuration for a v1 attestation request. +/// +/// `report_data` takes bytes and only bytes. A hex string is a different value +/// with the same spelling, and the builder will not take one: +/// +/// ```compile_fail +/// use dstack_sdk_types::dstack_v1::AttestConfig; +/// // 8 ASCII bytes where 4 were meant -- rejected at compile time. +/// let _ = AttestConfig::builder().report_data("deadbeef").build(); +/// ``` +/// +/// ``` +/// use dstack_sdk_types::dstack_v1::AttestConfig; +/// let config = AttestConfig::builder().report_data([0xde, 0xad, 0xbe, 0xef]).build(); +/// assert_eq!(config.report_data.len(), 4); +/// ``` #[derive(Debug, bon::Builder, Serialize, Deserialize)] #[cfg_attr(feature = "borsh", derive(BorshSerialize, BorshDeserialize))] #[cfg_attr(feature = "borsh_schema", derive(BorshSchema))] pub struct AttestConfig { - /// The report data in hexadecimal format, at most 64 bytes once decoded + /// The report data, at most 64 bytes. Hex-encoded on the wire. #[builder(into)] - pub report_data: String, + pub report_data: ReportData, /// Also return the boot-time GPU attestation evidence #[builder(default = false)] pub include_boottime_gpu_evidence: bool, @@ -123,8 +215,9 @@ pub struct AttestConfig { #[cfg_attr(feature = "borsh", derive(BorshSerialize, BorshDeserialize))] #[cfg_attr(feature = "borsh_schema", derive(BorshSchema))] pub struct AttestResponse { - /// The attestation, hex-encoded - pub attestation: String, + /// The attestation + #[serde(with = "hex::serde")] + pub attestation: Vec, /// Boot-time GPU attestation evidence. Empty unless the request asked for /// it and boot-time output exists, so absence is just the empty list. /// @@ -133,9 +226,9 @@ pub struct AttestResponse { /// written at boot, [`FORMAT_ON_DEMAND`] is collected against a caller's /// nonce, and a verifier for one does not appraise the other. /// - /// Each bundle's `evidence` decodes to the nvattest output byte for byte. - /// That exactness is the contract: the only thing binding this evidence to - /// the boot is sha256 over precisely those bytes, compared against + /// Each bundle's `evidence` is the nvattest output byte for byte. That + /// exactness is the contract: the only thing binding this evidence to the + /// boot is sha256 over precisely those bytes, compared against /// `evidence_sha256` in the measured `gpu-attestation` event after /// replaying the runtime event log. Re-serializing the JSON first changes /// the digest. @@ -145,12 +238,6 @@ pub struct AttestResponse { pub boottime_gpu_evidence: Vec, } -impl AttestResponse { - pub fn decode_attestation(&self) -> Result, FromHexError> { - hex::decode(&self.attestation) - } -} - /// Vendor-native GPU evidence collected on demand. #[derive(Debug, Serialize, Deserialize)] #[cfg_attr(feature = "borsh", derive(BorshSerialize, BorshDeserialize))] @@ -179,14 +266,10 @@ pub struct GpuEvidenceBundle { pub vendor: String, /// Vendor-specific evidence format and version pub format: String, - /// Hex-encoded opaque vendor-native evidence bytes - pub evidence: String, -} - -impl GpuEvidenceBundle { - pub fn decode_evidence(&self) -> Result, FromHexError> { - hex::decode(&self.evidence) - } + /// Opaque vendor-native evidence bytes, verbatim as the vendor tool emitted + /// them. Hex-encoded on the wire. + #[serde(with = "hex::serde")] + pub evidence: Vec, } /// Application identity and configuration. @@ -203,27 +286,32 @@ impl GpuEvidenceBundle { #[cfg_attr(feature = "borsh", derive(BorshSerialize, BorshDeserialize))] #[cfg_attr(feature = "borsh_schema", derive(BorshSchema))] pub struct InfoResponse { - /// App ID, hex-encoded - pub app_id: String, + /// App ID + #[serde(with = "hex::serde")] + pub app_id: Vec, /// App name, from app-compose + #[serde(default)] pub app_name: String, - /// Compose hash, hex-encoded. sha256 over the verbatim bytes of - /// `app_compose`; do not re-serialize before hashing. - pub compose_hash: String, + /// Compose hash: sha256 over the verbatim bytes of `app_compose`; do not + /// re-serialize before hashing. + #[serde(with = "hex::serde")] + pub compose_hash: Vec, /// The app-compose document, exactly as deployed. Empty on the external /// surface unless the app set `public_tcbinfo`. #[serde(default)] pub app_compose: String, - /// App instance ID, hex-encoded - pub instance_id: String, - /// Device ID, hex-encoded. Identifies the host machine, not this instance. - pub device_id: String, - /// OS image hash, hex-encoded - #[serde(default)] - pub os_image_hash: String, - /// Aggregated measurement register value, hex-encoded - #[serde(default)] - pub mr_aggregated: String, + /// App instance ID + #[serde(with = "hex::serde")] + pub instance_id: Vec, + /// Device ID. Identifies the host machine, not this instance. + #[serde(with = "hex::serde")] + pub device_id: Vec, + /// OS image hash + #[serde(default, with = "hex::serde")] + pub os_image_hash: Vec, + /// Aggregated measurement register value + #[serde(default, with = "hex::serde")] + pub mr_aggregated: Vec, /// The VM's hardware configuration, as a JSON document produced by the VMM #[serde(default)] pub vm_config: String, @@ -238,20 +326,6 @@ pub struct InfoResponse { pub cloud_product: String, } -impl InfoResponse { - pub fn decode_app_id(&self) -> Result, FromHexError> { - hex::decode(&self.app_id) - } - - pub fn decode_instance_id(&self) -> Result, FromHexError> { - hex::decode(&self.instance_id) - } - - pub fn decode_compose_hash(&self) -> Result, FromHexError> { - hex::decode(&self.compose_hash) - } -} - /// The guest agent version. #[derive(Debug, Serialize, Deserialize)] #[cfg_attr(feature = "borsh", derive(BorshSerialize, BorshDeserialize))] @@ -262,3 +336,99 @@ pub struct VersionResponse { /// Git revision pub rev: String, } + +#[cfg(test)] +mod tests { + use super::*; + use alloc::{string::ToString, vec, vec::Vec}; + + #[test] + fn signature_chain_round_trips_through_hex() { + let response = GetKeyResponse { + key: vec![0x01; 32], + public_key: vec![0x02; 33], + signature_chain: vec![vec![0xaa, 0xbb], vec![0xcc]], + }; + let json = serde_json::to_string(&response).expect("serializes"); + assert!(json.contains(r#""signature_chain":["aabb","cc"]"#)); + + let decoded: GetKeyResponse = serde_json::from_str(&json).expect("deserializes"); + assert_eq!(decoded.key, response.key); + assert_eq!(decoded.public_key, response.public_key); + assert_eq!(decoded.signature_chain, response.signature_chain); + } + + #[test] + fn signature_chain_accepts_an_empty_list() { + let json = r#"{"key":"01","public_key":"02","signature_chain":[]}"#; + let decoded: GetKeyResponse = serde_json::from_str(json).expect("deserializes"); + assert!(decoded.signature_chain.is_empty()); + + let reencoded = serde_json::to_string(&decoded).expect("serializes"); + assert!(reencoded.contains(r#""signature_chain":[]"#)); + } + + #[test] + fn signature_chain_rejects_a_malformed_element() { + let json = r#"{"key":"01","public_key":"02","signature_chain":["aabb","zz"]}"#; + let err = serde_json::from_str::(json) + .expect_err("a non-hex element fails the whole field"); + assert!(err.to_string().contains("Invalid character"), "{err}"); + } + + #[test] + fn report_data_is_hex_on_the_wire() { + let config = AttestConfig::builder() + .report_data(vec![0xde, 0xad, 0xbe, 0xef]) + .build(); + let json = serde_json::to_string(&config).expect("serializes"); + assert!(json.contains(r#""report_data":"deadbeef""#), "{json}"); + } + + #[test] + fn report_data_accepts_every_byte_shape_a_caller_holds() { + let owned = AttestConfig::builder().report_data(vec![0xaa, 0xbb]).build(); + let array = AttestConfig::builder().report_data([0xaa, 0xbb]).build(); + let borrowed_array = AttestConfig::builder().report_data(b"\xaa\xbb").build(); + let slice = AttestConfig::builder() + .report_data(&[0xaa, 0xbb][..]) + .build(); + + for config in [owned, array, borrowed_array, slice] { + assert_eq!(&config.report_data[..], &[0xaa, 0xbb]); + } + } + + #[test] + fn report_data_round_trips_through_the_wire_form() { + let json = r#"{"report_data":"deadbeef","include_boottime_gpu_evidence":false}"#; + let config: AttestConfig = serde_json::from_str(json).expect("deserializes"); + assert_eq!(config.report_data, ReportData(vec![0xde, 0xad, 0xbe, 0xef])); + assert_eq!(serde_json::to_string(&config).expect("serializes"), json); + } + + #[test] + fn report_data_rejects_a_non_hex_wire_value() { + let json = r#"{"report_data":"zz","include_boottime_gpu_evidence":false}"#; + let err = serde_json::from_str::(json).expect_err("not hex"); + assert!(err.to_string().contains("Invalid character"), "{err}"); + } + + #[test] + fn info_identity_fields_decode_to_bytes() { + let json = r#"{ + "app_id": "0011", + "app_name": "demo", + "compose_hash": "2233", + "instance_id": "4455", + "device_id": "6677" + }"#; + let info: InfoResponse = serde_json::from_str(json).expect("deserializes"); + assert_eq!(info.app_id, vec![0x00, 0x11]); + assert_eq!(info.compose_hash, vec![0x22, 0x33]); + assert_eq!(info.instance_id, vec![0x44, 0x55]); + assert_eq!(info.device_id, vec![0x66, 0x77]); + assert_eq!(info.os_image_hash, Vec::::new()); + assert_eq!(info.mr_aggregated, Vec::::new()); + } +}