-
-
Notifications
You must be signed in to change notification settings - Fork 1.4k
feat(webapp): run the dashboard agent through AWS Bedrock behind an env switch #4609
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
kathiekiwi
wants to merge
19
commits into
main
Choose a base branch
from
feat/dashboard-agent-bedrock
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
19 commits
Select commit
Hold shift + click to select a range
f776554
feat(dashboard-agent): run model calls through Bedrock behind an env …
kathiekiwi 2e6e26f
feat(dashboard-agent): report cache token usage on both providers
kathiekiwi bfb3a59
fix(dashboard-agent): guard Bedrock model-id shape and drop the guess…
kathiekiwi de36f49
fix(rbac): deny non-member user-actor tokens at the ability layer
kathiekiwi b938d4d
fix(webapp): route dashboard agent head-start through the shared mode…
kathiekiwi 4f93ff1
docs: add server changes note for dashboard agent auth hardening
kathiekiwi 0a54744
fix(webapp): make dashboard agent head-start gate provider-aware
kathiekiwi 87851ad
fix(webapp): gate bedrock head-start on region and validate provider env
kathiekiwi 66cfdfb
chore: drop server-changes note
kathiekiwi f4ba8d3
fix(webapp): gate bedrock head-start on AWS_REGION only
kathiekiwi df2a1bb
fix(rbac): gate user-actor membership floor on a scoped context
kathiekiwi fde7011
fix(dashboard-agent): route the runtime eval judge through the provid…
kathiekiwi c094a76
fix(dashboard-agent): keep the Bedrock prefix cachePoint on short con…
kathiekiwi 85bd73e
test(rbac): prove unscoped user-actor path skips the user lookup
kathiekiwi cbfa5f8
fix(webapp): gate bedrock head-start on AWS_REGION or AWS_DEFAULT_REGION
kathiekiwi 642d8a3
fix(dashboard-agent): resolve Bedrock region from AWS_REGION or AWS_D…
kathiekiwi 89cd034
refactor(dashboard-agent): treat empty Bedrock region as unset, match…
kathiekiwi cbf7e3b
fix(dashboard-agent): stop leaking an undocumented ttl to AWS Bedrock
kathiekiwi b6a73d5
docs(dashboard-agent): trim the cache-breakpoint comments
kathiekiwi File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
99 changes: 99 additions & 0 deletions
99
internal-packages/dashboard-agent/src/model-provider.test.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,99 @@ | ||
| import { afterEach, describe, expect, it } from "vitest"; | ||
| import { PROMPT_CACHE_CONTROL } from "./prompt-prefix"; | ||
| import { | ||
| BEDROCK_MODEL_IDS, | ||
| isLongLivedCacheBreakpoint, | ||
| isStepCacheBreakpoint, | ||
| resolveDashboardAgentModel, | ||
| STEP_CACHE_CONTROL, | ||
| withCacheBreakpoint, | ||
| withoutCacheBreakpoint, | ||
| } from "./model-provider"; | ||
|
|
||
| function useBedrock() { | ||
| process.env.DASHBOARD_AGENT_MODEL_PROVIDER = "bedrock"; | ||
| } | ||
|
|
||
| afterEach(() => { | ||
| delete process.env.DASHBOARD_AGENT_MODEL_PROVIDER; | ||
| }); | ||
|
|
||
| describe("resolveDashboardAgentModel", () => { | ||
| it("resolves a canonical prompt string against Anthropic by default", () => { | ||
| expect(resolveDashboardAgentModel("anthropic:claude-sonnet-4-6").modelId).toBe( | ||
| "claude-sonnet-4-6" | ||
| ); | ||
| }); | ||
|
|
||
| it("maps the same canonical string to a Bedrock inference profile", () => { | ||
| useBedrock(); | ||
| expect(resolveDashboardAgentModel("anthropic:claude-sonnet-4-6").modelId).toBe( | ||
| "us.anthropic.claude-sonnet-4-6-v1" | ||
| ); | ||
| expect(resolveDashboardAgentModel("anthropic:claude-haiku-4-5").modelId).toBe( | ||
| "us.anthropic.claude-haiku-4-5-20251001-v1:0" | ||
| ); | ||
| }); | ||
|
|
||
| it("throws rather than guessing a profile for an unmapped id", () => { | ||
| useBedrock(); | ||
| expect(() => resolveDashboardAgentModel("anthropic:claude-made-up-9-9")).toThrow( | ||
| /No Bedrock model mapping/ | ||
| ); | ||
| }); | ||
|
|
||
| // Structural, not an echo of the table: an AWS us cross-region Anthropic profile | ||
| // is either dated with a `:N` suffix, or an undated `-vN` (the 4-6 generation). A | ||
| // dated id must never drop its `:N`, and every id must end in a version. | ||
| it("every Bedrock-mapped id has a well-formed AWS inference-profile shape", () => { | ||
| const dated = /^us\.anthropic\.claude-[a-z]+(?:-\d+)+-\d{8}-v\d+:\d+$/; | ||
| const undated = /^us\.anthropic\.claude-[a-z]+(?:-\d+)+-v\d+$/; | ||
| for (const id of Object.values(BEDROCK_MODEL_IDS)) { | ||
| expect(dated.test(id) || undated.test(id), id).toBe(true); | ||
| if (/-\d{8}-/.test(id)) expect(id, id).toMatch(/:\d+$/); | ||
| } | ||
| }); | ||
| }); | ||
|
|
||
| describe("cache breakpoints", () => { | ||
| it("keeps the Anthropic cacheControl ttls intact, tagged with the discriminator", () => { | ||
| expect(withCacheBreakpoint({ openai: { store: false } }, "prefix")).toEqual({ | ||
| __cacheBreakpoint: { kind: "prefix" }, | ||
| openai: { store: false }, | ||
| anthropic: { cacheControl: PROMPT_CACHE_CONTROL }, | ||
| }); | ||
| expect(withCacheBreakpoint(undefined, "step")).toEqual({ | ||
| __cacheBreakpoint: { kind: "step" }, | ||
| anthropic: { cacheControl: STEP_CACHE_CONTROL }, | ||
| }); | ||
| }); | ||
|
|
||
| it("emits a plain Bedrock cachePoint with no ttl for either marker", () => { | ||
| useBedrock(); | ||
| for (const breakpoint of ["prefix", "step"] as const) { | ||
| const options = withCacheBreakpoint(undefined, breakpoint); | ||
| // The only thing the SDK serialises to AWS is bedrock.cachePoint — it must be plain. | ||
| expect(options.bedrock.cachePoint).toEqual({ type: "default" }); | ||
| expect(options.bedrock.cachePoint).not.toHaveProperty("ttl"); | ||
| expect(options.__cacheBreakpoint).toEqual({ kind: breakpoint }); | ||
| } | ||
| }); | ||
|
|
||
| it("classifies and strips the active provider's breakpoint via the discriminator", () => { | ||
| const anthropicStep = withCacheBreakpoint({ anthropic: { keep: true } }, "step"); | ||
| expect(isStepCacheBreakpoint(anthropicStep)).toBe(true); | ||
| expect(isLongLivedCacheBreakpoint(withCacheBreakpoint(undefined, "prefix"))).toBe(true); | ||
| // The strip removes both the provider field and the top-level discriminator. | ||
| expect(withoutCacheBreakpoint(anthropicStep)).toEqual({ anthropic: { keep: true } }); | ||
|
|
||
| useBedrock(); | ||
| const bedrockStep = withCacheBreakpoint(undefined, "step"); | ||
| const bedrockPrefix = withCacheBreakpoint(undefined, "prefix"); | ||
| // The two Bedrock markers are byte-identical on the wire — only the tag tells them apart. | ||
| expect(bedrockStep.bedrock).toEqual(bedrockPrefix.bedrock); | ||
| expect(isStepCacheBreakpoint(bedrockStep)).toBe(true); | ||
| expect(isLongLivedCacheBreakpoint(bedrockStep)).toBe(false); | ||
| expect(isLongLivedCacheBreakpoint(bedrockPrefix)).toBe(true); | ||
| expect(withoutCacheBreakpoint(bedrockStep)).toEqual({}); | ||
| }); | ||
| }); |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.