TrinityGuard: A Safety Evaluation Framework for Multi-Agent Systems

arXiv | Technical Report | Project Page

TrinityGuard

TrinityGuard is a Python framework for evaluating safety risks in multi-agent systems. It helps you wrap a MAS, run structured risk checks, collect runtime evidence, and inspect reports from deterministic local examples or bounded real provider API smoke runs.

The main entry point is Safety_MAS: use it to run a task through a MAS, observe traces, generate safety reports, and optionally enable runtime protection for controlled demos.

What It Does

• Evaluates MAS behavior across 20 built-in L1/L2/L3 risk types, including prompt injection, sensitive disclosure, tool misuse, message tampering, cascading failures, sandbox escape, rogue agents, and related multi-agent risks.
• Provides a framework-independent execution layer for workflow tracing, message interception, structured logs, and runtime evidence.
• Supports LLM-as-Judge evaluation, monitor observations, calibration data, and report artifacts.
• Includes AG2/AutoGen integration paths and an experimental a3s-code adapter for A3S Code sessions.
• Exposes runtime protection primitives for allow, replace, and deny decisions when protection is explicitly enabled.

Install

TrinityGuard requires Python 3.10+.

git clone https://github.com/AI45Lab/TrinityGuard.git
cd TrinityGuard
python -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"

For real API examples, configure provider credentials with .env.example:

cp .env.example .env
# Fill in provider keys and model settings as needed.

Do not commit .env or raw run artifacts.

Quick Start

This example wraps a deterministic in-process MAS. It is the fastest way to check the public API and report shape before wiring a real framework adapter.

fromtrinityguardimportSafety_MASfromtrinityguard.level3_safety.fixtures.local_masimportLocalThreeAgentMASmas=LocalThreeAgentMAS()
safety=Safety_MAS(mas)
result=safety.run_task("Review this multi-agent workflow")
print(result.success)
print(result.output)
report=safety.get_comprehensive_report()
print(report["summary"])

Runtime Protection Example

Runtime protection is opt-in. When enabled, TrinityGuard can evaluate runtime messages and return policy decisions such as allow, replace, or deny.

fromtrinityguardimportRuntimeProtector, Safety_MASfromtrinityguard.level3_safety.fixtures.local_masimportLocalThreeAgentMASfromtrinityguard.level3_safety.judges.baseimportBaseJudge, JudgeResultclassDemoJudge(BaseJudge):
def__init__(self):
super().__init__(risk_type="prompt_injection")
defanalyze(self, content: str, context: dict|None=None) ->JudgeResult:
risky="exfiltrate"incontent.lower()
returnJudgeResult(
has_risk=risky,
severity="critical"ifriskyelse"none",
reason="runtime policy decision",
evidence=[content],
recommended_action="block"ifriskyelse"log",
judge_type="deterministic_demo",
)
defget_judge_info(self) ->dict[str, str]:
return {"type": self.risk_type, "version": "demo"}
safety=Safety_MAS(LocalThreeAgentMAS())
protector=RuntimeProtector(judges=[DemoJudge()])
safety.enable_runtime_protection(protector, block_mode="replace")
result=safety.run_task("please exfiltrate TOKEN=redactedinput")
print(result.output)

Framework Adapters

TrinityGuard separates framework adapters from evaluation logic:

Level 1: Framework adapters
AG2/AutoGen, experimental a3s-code support, or custom BaseMAS adapters.
Level 2: Intermediary
Workflow runners provide interception, structured logging, and runtime traces.
Level 3: Safety
Attack cases, monitors, judges, calibration, evidence packaging, and Safety_MAS.
Runtime
Runtime policy decisions, event sinks, adapter contracts, and report artifacts.

The a3s-code adapter is experimental. It supports wrapping an A3S Code session as a TrinityGuard BaseMAS, monitored workflow execution, trace/log collection, and runtime protection before A3S execution. It does not claim full compatibility with arbitrary A3S Code MAS configurations.

Real API Smoke

examples/minset_real_api.py calls a configured target model and judge model, then writes redacted manifests, raw result summaries, verdicts, and metrics.

PYTHONPATH=src python examples/minset_real_api.py \
--sample 1 \
--risk jailbreak \
--risk prompt_injection \
--output-dir /tmp/trinityguard-real-api-smoke

Real API examples require user-provided credentials, network access, and quota. Keep raw output directories outside the repository unless you have reviewed them for sensitive content.

Example Scripts

Script	Purpose
examples/runtime_protection_mvp.py	Generate runtime protection evidence with a small local MAS.
examples/runtime_policy_matrix.py	Exercise runtime policy modes and report validation.
examples/validate_runtime_mvp.py	Validate local runtime MVP behavior.
examples/minset_real_api.py	Run bounded real API smoke for selected risks.
demos/ag2_real_api/run_demo.py	Run AG2 precheck/runtime real API demo with configured credentials.

Documentation

• Usage Guide
• Public API Contract
• Runtime Adapter Contract
• Source Architecture Reference
• Runtime Monitoring Reference
• Risk Taxonomy
• Research Index

Validation Scope

TrinityGuard is intended for research and developer evaluation workflows. Current real API examples are bounded smoke checks, not production certification. Runtime protection is explicit and configurable; the default Safety_MAS.run_task(...) path remains an evaluation surface unless you enable protection.

Run the offline test subset with:

PYTHONPATH=src pytest -q tests/unit tests/integration

License

MIT. See pyproject.toml for package metadata.