Agent Simulation Engine SDK

Official Python SDK for the Lyzr Agent Simulation Engine (A-Sim) Platform

Installation • Quick Start • Features • API Reference • Examples

---

Overview

The Agent Simulation Engine (A-Sim) SDK enables you to programmatically test, evaluate, and improve your AI agents through automated simulation and reinforcement learning loops.

What is A-Sim?

A-Sim is a comprehensive platform for:

Feature	Description
Automated Testing	Generate realistic test cases from persona × scenario combinations
AI Evaluation	Evaluate agent responses on accuracy, helpfulness, safety & more
RL Hardening	Continuously improve agents through reinforcement learning loops
Ground Truth Validation	Validate responses against knowledge base facts

---

How to Run

Step 1: Install the SDK

pip install git+https://github.com/LyzrCore/agent-simulation-engine.git#subdirectory=sdk

Step 2: Get Your API Key

Get your studio-api-key from Lyzr Studio

Step 3: Run Complete Workflow

fromagent_simulation_engineimportASIMEngineimporttime# ============================================# STEP 1: Initialize Engine# ============================================engine=ASIMEngine(api_key="your-studio-api-key")
# ============================================# STEP 2: Create Environment# ============================================env=engine.environments.create(
agent_id="your-agent-id", # From Lyzr Studioname="My Agent Tests"
)
print(f"✓ Environment created: {env.environment_id}")
# ============================================# STEP 3: Generate Personas & Scenarios# ============================================personas=engine.personas.generate(env.environment_id)
print(f"✓ Generated {personas.count} personas")
scenarios=engine.scenarios.generate(env.environment_id)
print(f"✓ Generated {scenarios.count} scenarios")
# ============================================# STEP 4: Generate Simulations (Test Cases)# ============================================job=engine.simulations.generate(env.environment_id)
print(f"✓ Simulation job started: {job.job_id}")
# Wait for completionwhileTrue:
status=engine.jobs.get_status(env.environment_id, job.job_id)
print(f" Progress: {status.progress}")
ifstatus.summary.completed+status.summary.failed==status.summary.total:
breaktime.sleep(3)
print(f"✓ Simulations generated!")
# ============================================# STEP 5: Run Evaluations# ============================================eval_run=engine.evaluations.create(
environment_id=env.environment_id,
evaluation_run_name="Round 1"
)
print(f"✓ Evaluation started: {eval_run.evaluation_run_id}")
# Wait for completionwhileTrue:
status=engine.jobs.get_evaluation_status(env.environment_id, eval_run.job_id)
print(f" Progress: {status.progress}")
ifstatus.summary.completed+status.summary.failed==status.summary.total:
breaktime.sleep(3)
# ============================================# STEP 6: View Results# ============================================results=engine.evaluations.list(env.environment_id)
pass_count=sum(1foreinresults.evaluationsife.judgment=="PASS")
fail_count=sum(1foreinresults.evaluationsife.judgment=="FAIL")
print(f"\n📊 Results: {pass_count} PASS | {fail_count} FAIL")
# ============================================# STEP 7: Harden Agent (if needed)# ============================================iffail_count>0:
hardening=engine.hardening.harden_agent(
environment_id=env.environment_id,
run_id=eval_run.evaluation_run_id,
round_number=1
)
print(f"\n🔧 Agent Hardened!")
print(f" Original: {hardening.original_config.agent_instructions[:100]}...")
print(f" Improved: {hardening.improved_config.agent_instructions[:100]}...")

Step 4: Run It!

python your_script.py

---

Installation

From GitHub (Recommended)

pip install git+https://github.com/LyzrCore/agent-simulation-engine.git#subdirectory=sdk

From Source

git clone https://github.com/LyzrCore/agent-simulation-engine.git
cd agent-simulation-engine/sdk
pip install -e .

Requirements

• Python 3.8+
• requests >= 2.28.0
• pydantic >= 2.0.0

---

Quick Start

fromagent_simulation_engineimportASIMEngineimporttime# Initialize the engineengine=ASIMEngine(api_key="studio-api-key")
# Create an environment for your agentenv=engine.environments.create(
agent_id="studio-agent-key",
name="Customer Support Tests"
)
# Generate personas and scenarios using AIpersonas=engine.personas.generate(env.environment_id)
scenarios=engine.scenarios.generate(env.environment_id)
# Generate test simulationsjob=engine.simulations.generate(env.environment_id)
# Poll until completewhileTrue:
status=engine.jobs.get_status(env.environment_id, job.job_id)
ifstatus.summary.completed+status.summary.failed==status.summary.total:
breaktime.sleep(2)
# Run evaluationseval_run=engine.evaluations.create(
environment_id=env.environment_id,
evaluation_run_name="Round 1"
)
print(f"Evaluation started: {eval_run.evaluation_run_id}")

---

Features

World Model Architecture

Reinforcement Hardening Loop

# 1. Run initial evaluationseval_run=engine.evaluations.create(env_id, "Round 1")
# 2. Wait for completion, then hardenhardening=engine.hardening.harden_agent(
environment_id=env_id,
run_id=eval_run.evaluation_run_id,
round_number=1
)
# 3. View improvementsprint("Original:", hardening.original_config.agent_instructions)
print("Improved:", hardening.improved_config.agent_instructions)
# 4. Continue with improved agentnew_round=engine.hardening.continue_run(
environment_id=env_id,
run_id=eval_run.evaluation_run_id,
round_number=1,
agent_config=hardening.improved_config.model_dump()
)
# 5. Repeat until all tests pass!

---

API Reference

Initialization

fromagent_simulation_engineimportASIMEngineengine=ASIMEngine(
api_key="studio-api-key", # Requiredbase_url="https://agent.api.lyzr.ai", # Optionaltimeout=30# Optional (seconds)
)

Resources

Resource	Description	Methods
engine.environments	Manage test environments	create(), get(), list_by_agent(), delete()
engine.personas	User archetypes	create(), list(), generate(), delete()
engine.scenarios	Task types	create(), list(), generate(), delete()
engine.simulations	Test cases	create(), list(), get(), update(), delete(), generate()
engine.evaluations	Run evaluations	create(), list(), get()
engine.jobs	Track async jobs	get_status(), list(), cancel(), get_evaluation_status()
engine.evaluation_runs	RL training rounds	get(), list(), get_round(), sync_round()
engine.hardening	Improve agents	harden_agent(), continue_run()

---

Examples

Create Environment & Generate Test Data

# Create environmentenv=engine.environments.create(
agent_id="studio-agent-key",
name="Product Support Tests"
)
# Add personas manuallypersonas=engine.personas.create(env.environment_id, personas=[
{"name": "New Customer", "description": "First-time user, unfamiliar with product"},
{"name": "Power User", "description": "Experienced user with technical knowledge"},
{"name": "Frustrated Customer", "description": "User experiencing issues, potentially angry"},
])
# Or generate with AIpersonas=engine.personas.generate(env.environment_id)
scenarios=engine.scenarios.generate(env.environment_id)

Run Evaluations

# Start evaluation runeval_run=engine.evaluations.create(
environment_id=env.environment_id,
evaluation_run_name="Initial Assessment",
metrics=["task_completion", "hallucinations", "answer_relevancy"]
)
# Poll for completionwhileTrue:
status=engine.jobs.get_evaluation_status(env.environment_id, eval_run.job_id)
print(f"Progress: {status.progress}")
ifstatus.summary.completed+status.summary.failed==status.summary.total:
breaktime.sleep(2)
# Get resultsresults=engine.evaluations.list(env.environment_id)
forevalinresults.evaluations:
print(f"{eval.id}: {eval.judgment} - {eval.scores}")

---

Error Handling

The SDK provides specific exception types for different error scenarios:

fromagent_simulation_engineimport (
ASIMEngine,
ASIMError,
AuthenticationError,
NotFoundError,
ValidationError,
RateLimitError,
ServerError,
)
engine=ASIMEngine(api_key="studio-api-key")
try:
env=engine.environments.get("invalid-id")
exceptAuthenticationError:
print("Invalid API key")
exceptNotFoundError:
print("Environment not found")
exceptValidationErrorase:
print(f"Invalid request: {e.message}")
exceptRateLimitError:
print("Rate limit exceeded, please retry later")
exceptServerError:
print("Server error, please try again")
exceptASIMErrorase:
print(f"API error: {e.message} (status: {e.status_code})")

---

Support

Resource	Link
GitHub Issues	Report a Bug

---

Built with by Lyzr AI

Website • Twitter • LinkedIn