Home

EvalCI Wiki

EvalCI is a zero-infra GitHub Action that runs @eval_case quality suites on every PR and blocks merge if scores drop below your threshold.

→ Full documentation:https://synapsekit.github.io/synapsekit-docs/docs/evalci/overview

---

Quick links

Page	Description
Overview	What EvalCI does and how it works
Quickstart	Get running in under 5 minutes
Writing Eval Cases	@eval_case decorator reference
Action Reference	All inputs, outputs, and exit codes
Examples	RAG, agents, multi-provider, Slack notifications
Troubleshooting	Common errors and fixes

---

30-second setup

# .github/workflows/eval.yml
- uses: SynapseKit/evalci@v1with:
path: tests/evalsthreshold: "0.80"env:
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}

# tests/evals/eval_rag.pyfromsynapsekitimporteval_case@eval_case(min_score=0.80)asyncdefeval_my_pipeline():
result=awaitpipeline.ask("What is SynapseKit?")
return {"score": result.relevancy_score}

---

Get help

• Open an issue
• Start a discussion
• Full docs