Skip to content

test(eval): record eval run at 8a1a04a - #269

Merged
github-actions[bot] merged 1 commit into
mainfrom
evals/run-33365224683-1
Aug 31, 2026
Merged

test(eval): record eval run at 8a1a04a#269
github-actions[bot] merged 1 commit into
mainfrom
evals/run-33365224683-1

Conversation

@github-actions

@github-actions github-actions Bot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor
3/3  92s  evals/check-before-acting.eval.ts / consults the docs before an unfamiliar task
3/3  27s  evals/check-before-acting.eval.ts / does not destructively act on an ambiguous request
3/3  9s  evals/check-before-acting.eval.ts / does not try to reassign a document to another type
3/3  14s  evals/configure-repositories.eval.ts / adds a locale
3/3  19s  evals/configure-repositories.eval.ts / creates an API token and makes the API private
3/3  16s  evals/configure-repositories.eval.ts / registers a webhook
3/3  13s  evals/configure-repositories.eval.ts / sets up a content preview
0/3  37s  evals/configure-repositories.eval.ts / updates previews for production after a deploy
1/3  115s  evals/configure-routes.eval.ts / routes a nested URL through a content relationship
0/3  17s  evals/configure-routes.eval.ts / routes non-default locales with an optional locale prefix
1/3  81s  evals/configure-routes.eval.ts / routes the home document to the root URL
3/3  141s  evals/design-sensible-models.eval.ts / handles a vague design request reasonably
0/3  37s  evals/design-sensible-models.eval.ts / models a page type with sensible field types
3/3  57s  evals/design-sensible-models.eval.ts / models a slice from a screenshot
3/3  25s  evals/design-sensible-models.eval.ts / models a testimonial slice from a text description
1/3  27s  evals/design-sensible-models.eval.ts / models a title as single-heading rich text and a social media handle as key text
3/3  100s  evals/design-sensible-models.eval.ts / models pages as page types and data as custom types
3/3  29s  evals/design-sensible-models.eval.ts / picks appropriate field types for a rating and a CTA
3/3  19s  evals/edit-models-precisely.eval.ts / adds a field
3/3  29s  evals/edit-models-precisely.eval.ts / adds a group field with nested fields
3/3  17s  evals/edit-models-precisely.eval.ts / adds a slice variation without losing existing ones
3/3  23s  evals/edit-models-precisely.eval.ts / adds constrained fields with the right config
3/3  16s  evals/edit-models-precisely.eval.ts / connects a slice to a page type
3/3  28s  evals/edit-models-precisely.eval.ts / extends a slice to match a design change
3/3  36s  evals/edit-models-precisely.eval.ts / is idempotent when the same task runs twice
3/3  19s  evals/edit-models-precisely.eval.ts / uses the documented config for a single-heading constraint
1/3  47s  evals/initialize-projects.eval.ts / adds Prismic to an existing Next.js app without clobbering it
2/3  63s  evals/initialize-projects.eval.ts / initializes Prismic in a Next.js project
3/3  26s  evals/initialize-projects.eval.ts / initializes with an existing repository
3/3  228s  evals/render-content.eval.ts / models and wires a landing page end to end
2/3  106s  evals/render-content.eval.ts / writes a slice component
3/3  62s  evals/repair-generated-files.eval.ts / regenerates a stale slice index instead of editing it by hand
2/3  19s  evals/sync-models.eval.ts / commits and pushes local model changes
3/3  30s  evals/sync-models.eval.ts / syncs in the right direction when the repo is newer

Run: https://github.com/prismicio/cli/actions/runs/33365224683


Note

Low Risk
Data-only update to eval artifacts; no production CLI or application code changes.

Overview
Refreshes the committed full-suite baseline in evals/results.json for model claude-sonnet-5 after CI eval run at 8a1a04a. Each eval case still stores three trials with pass, costUsd, durationS, and the verbatim npx prismic call sequences—only the recorded outcomes and traces change, not eval harness code.

Compared to the previous snapshot, scores shift in a few areas: nested route via content relationship and sync push look stronger, while production preview updates, locale routing, blog post field modeling, and non-destructive Next.js init remain weak or regress. Most other suites stay at prior pass rates (e.g. check-before-acting, edit-models, render-content largely green).

Reviewed by Cursor Bugbot for commit 7c81833. Bugbot is set up for automated code reviews on this repo. Configure here.

@github-actions
github-actions Bot merged commit 72cb9a5 into main Aug 31, 2026
1 check passed
@github-actions
github-actions Bot deleted the evals/run-33365224683-1 branch August 31, 2026 06:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants