Skip to content

feat: publish benchmark answer browser - #2

Merged
farhan-helmy merged 1 commit into
mainfrom
feat/benchmark-answer-browser
Aug 19, 2026
Merged

feat: publish benchmark answer browser#2
farhan-helmy merged 1 commit into
mainfrom
feat/benchmark-answer-browser

Conversation

@farhan-helmy

Copy link
Copy Markdown
Contributor

Summary

Publishes the complete BFCL Malay answer set so readers can compare Malay and Indonesian phrasing without opening local run files. The dashboard now presents 1,040 prompts and 4,160 verbatim model responses side by side, with category, result, and text filters plus per-category JSON downloads.

Safety and behavior

  • Public JSON uses an allowlist for prompts, translated function schemas, expected calls, scores, latency, answer text, and normalized tool calls.
  • Credentials, provider metadata, local paths, private harness details, and raw error messages are excluded.
  • The Pages build rejects schema drift or unexpected answer fields before deployment.
  • Category data loads lazily, ignores stale request failures, and exits cleanly on timeouts or missing files.

Validation

  • 51 benchmark tests passed.
  • 12 site and real-Chrome browser tests passed.
  • JavaScript syntax, Python compilation, production site build, and diff checks passed.
  • Desktop and mobile layouts were inspected with the full answer data.
  • A full generated-site scan found no credential-shaped values or private runtime details.
  • Formal code review completed with verdict Ready with fixes; all four validated findings were fixed and verified.

Known residuals

None.

Post-deploy validation

  • Watch the GitHub Pages build and deploy jobs after merge.
  • Verify the dashboard answer browser, data/answers/manifest.json, and data/answers/chatable.json return HTTP 200.
  • Healthy state: 1,040 manifest cases, all four models, rendered answers, and no loading or console error state.
  • Failure trigger: any missing answer file, non-200 response, stuck loading state, or model/count mismatch. Roll back by reverting this PR.
  • Validation window: immediately after deployment and again within 15 minutes. Owner: PixelSpaceAI.

Compound Engineering

@farhan-helmy
farhan-helmy merged commit db5fbb9 into mainAug 19, 2026
4 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

@farhan-helmy