Skip to content

test(eval): record eval run at d5ab9bd - #260

Merged
github-actions[bot] merged 1 commit into
mainfrom
evals/run-32001185558-1
Aug 17, 2026
Merged

test(eval): record eval run at d5ab9bd#260
github-actions[bot] merged 1 commit into
mainfrom
evals/run-32001185558-1

Conversation

@github-actions

@github-actionsgithub-actionsBot commented Aug 17, 2026

Copy link
Copy Markdown
Contributor
3/3 104s evals/check-before-acting.eval.ts / consults the docs before an unfamiliar task
3/3 22s evals/check-before-acting.eval.ts / does not destructively act on an ambiguous request
3/3 9s evals/check-before-acting.eval.ts / does not try to reassign a document to another type
3/3 15s evals/configure-repositories.eval.ts / adds a locale
3/3 19s evals/configure-repositories.eval.ts / creates an API token and makes the API private
3/3 19s evals/configure-repositories.eval.ts / registers a webhook
3/3 15s evals/configure-repositories.eval.ts / sets up a content preview
1/3 53s evals/configure-repositories.eval.ts / updates previews for production after a deploy
1/3 100s evals/configure-routes.eval.ts / routes a nested URL through a content relationship
0/3 5s evals/configure-routes.eval.ts / routes non-default locales with an optional locale prefix
0/3 84s evals/configure-routes.eval.ts / routes the home document to the root URL
1/3 81s evals/design-sensible-models.eval.ts / handles a vague design request reasonably
2/3 46s evals/design-sensible-models.eval.ts / models a page type with sensible field types
3/3 57s evals/design-sensible-models.eval.ts / models a slice from a screenshot
3/3 29s evals/design-sensible-models.eval.ts / models a testimonial slice from a text description
1/3 27s evals/design-sensible-models.eval.ts / models a title as single-heading rich text and a social media handle as key text
3/3 117s evals/design-sensible-models.eval.ts / models pages as page types and data as custom types
3/3 27s evals/design-sensible-models.eval.ts / picks appropriate field types for a rating and a CTA
3/3 23s evals/edit-models-precisely.eval.ts / adds a field
3/3 27s evals/edit-models-precisely.eval.ts / adds a group field with nested fields
3/3 18s evals/edit-models-precisely.eval.ts / adds a slice variation without losing existing ones
3/3 28s evals/edit-models-precisely.eval.ts / adds constrained fields with the right config
3/3 14s evals/edit-models-precisely.eval.ts / connects a slice to a page type
3/3 35s evals/edit-models-precisely.eval.ts / extends a slice to match a design change
3/3 40s evals/edit-models-precisely.eval.ts / is idempotent when the same task runs twice
3/3 20s evals/edit-models-precisely.eval.ts / uses the documented config for a single-heading constraint
1/3 33s evals/initialize-projects.eval.ts / adds Prismic to an existing Next.js app without clobbering it
3/3 98s evals/initialize-projects.eval.ts / initializes Prismic in a Next.js project
3/3 25s evals/initialize-projects.eval.ts / initializes with an existing repository
3/3 374s evals/render-content.eval.ts / models and wires a landing page end to end
3/3 130s evals/render-content.eval.ts / writes a slice component
3/3 54s evals/repair-generated-files.eval.ts / regenerates a stale slice index instead of editing it by hand
2/3 33s evals/sync-models.eval.ts / commits and pushes local model changes
3/3 25s evals/sync-models.eval.ts / syncs in the right direction when the repo is newer

Run: https://github.com/prismicio/cli/actions/runs/32001185558


Note

Low Risk
Single-file update to eval benchmark output; no runtime or CLI behavior is modified in this diff.

Overview
Refreshes evals/results.json with the latest benchmark run (model claude-sonnet-5), replacing per-scenario pass/fail, cost, duration, and CLI call traces for each of three trials.

The snapshot adds results for repair-generated-files.eval.ts (regenerate stale slice index via gen slice-index) at 3/3. Compared to the prior snapshot in the PR description, notable shifts include Next.js init (3/3 vs 0/3), sync-models pull-when-remote-is-newer (3/3 vs 2/3), and still-weak areas such route configuration (home root, locale prefix) and mixed scores on production preview updates and blog field-type modeling.

No CLI or application source files change—only the committed eval artifact for CI tracking.

Reviewed by Cursor Bugbot for commit bd125f5. Bugbot is set up for automated code reviews on this repo. Configure here.

@github-actions
github-actionsBot merged commit 1c8fd5f into mainAug 17, 2026
@github-actions
github-actionsBot deleted the evals/run-32001185558-1 branch August 17, 2026 06:31
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants