Where LLM Agents Fail and How They Can Learn From Failures

AgentDebug

AgentDebug is a framework for understanding, detecting, and recovering from LLM agent failures. It provides:

1. AgentErrorTaxonomy: A classification system covering 17 error types across 5 modules (memory, reflection, planning, action, system).
2. AgentErrorBench: Annotated failure trajectories from ALFWorld, GAIA, and WebShop environments.
3. AgentDebug Framework: A two-stage debugging pipeline that isolates root-cause failures and provides corrective feedback.

Installation

git clone https://github.com/ulab-uiuc/AgentDebug.git
cd AgentDebug
pip install -e .

Environment Setup

AgentDebug includes vendored environments (ALFWorld, WebShop, GAIA) with modular agent prompts. The rollout system requires these environments to be functional.

API Keys: Set the following environment variables (or put them in a .env file):

export OPENAI_API_KEY="..."export ANTHROPIC_API_KEY="..."# optionalexport GEMINI_API_KEY="..."# optionalexport TOGETHER_API_KEY="..."# optional

Repository Structure

AgentDebug/
├── detector/ # Core error detection framework
│ ├── fine_grained_analysis.py # Phase 1: step-level per-module error detection
│ ├── critical_error_detection.py # Phase 2: critical error identification
│ └── error_definitions.py # Error taxonomy (5 modules, 17 types)
├── agentdebug/
│ ├── engines/ # Multi-provider LLM abstraction
│ │ ├── openai.py # OpenAI (GPT-4o, GPT-4.1, etc.)
│ │ ├── anthropic.py # Anthropic (Claude)
│ │ ├── gemini.py # Google (Gemini)
│ │ └── together.py # Together AI (Llama, Qwen, etc.)
│ ├── environments/ # Environment wrappers + modular prompts
│ │ ├── alfworld/ # ALFWorld embodied tasks
│ │ ├── webshop/ # WebShop e-commerce tasks
│ │ └── gaia/ # GAIA general AI assistant tasks
│ └── rollout/ # Trajectory collection
│ ├── rollout.py # Unified rollout across all environments
│ └── step_to_episode.py # Step-level → episode-level conversion
├── examples/ # Sample data and demo scripts
└── docs/ # Documentation

Quick Start

Run the Detector on a Trajectory

fromdetector.fine_grained_analysisimportErrorTypeDetectorfromdetector.critical_error_detectionimportCriticalErrorAnalyzer# Configure with your API keyapi_config= {
"base_url": "https://api.openai.com/v1/chat/completions",
"api_key": "your-api-key",
"model": "gpt-4o-mini",
"temperature": 0.0,
"max_retries": 3,
"timeout": 60,
}
# Phase 1: Step-level error detectiondetector=ErrorTypeDetector(api_config)
trajectory_data=detector.parse_trajectory("path/to/trajectory.json")
phase1_results=awaitdetector.analyze_trajectory(trajectory_data)
# Phase 2: Critical error identificationanalyzer=CriticalErrorAnalyzer(api_config)
critical_error=awaitanalyzer.identify_critical_error(phase1_results, trajectory_data)

Collect Rollout Trajectories

# AlfWorld rollout with Together AI (cheap, fast)
python -m agentdebug.rollout.rollout \
--env alfworld \
--provider together \
--model meta-llama/Llama-3.3-70B-Instruct-Turbo \
--unique_envs \
--total_envs 100 \
--concurrency 4 \
--dump_path output/alfworld_steps.jsonl
# Convert steps to episodes
python -m agentdebug.rollout.step_to_episode \
--input_jsonl output/alfworld_steps.jsonl \
--output_jsonl output/alfworld_episodes.jsonl

Multi-Provider LLM Engine

fromagentdebug.enginesimportcreate_chat_model# Automatically routes to the correct provider based on model namemodel=create_chat_model("gpt-4o-mini") # → OpenAImodel=create_chat_model("claude-sonnet-4-6") # → Anthropicmodel=create_chat_model("gemini-2.5-flash") # → Geminimodel=create_chat_model("meta-llama/Llama-3.3-70B-Instruct-Turbo") # → Togetherresponse=model("What is 2+2?")

Error Taxonomy

Module	Error Types	Description
Memory	hallucination, memory_retrieval_failure, over_simplification	Agent misremembers or fails to recall information
Reflection	progress_misjudge, outcome_misinterpretation, causal_misattribution, hallucination	Agent incorrectly evaluates its own progress
Planning	constraint_ignorance, impossible_action, inefficient_plan	Agent creates flawed plans
Action	misalignment, invalid_action, format_error, parameter_error	Agent executes wrong actions
System	step_limit, tool_execution_error, llm_limit, environment_error	External system failures

AgentErrorBench Dataset

Download annotated failure trajectories: AgentErrorBench on Google Drive

• ALFWorld: 100 trajectories from embodied agent tasks
• GAIA: 50 trajectories from general AI assistant tasks
• WebShop: 50 trajectories from web navigation tasks

Key Results

Metric	Improvement
All-Correct Accuracy	+24%
Step Accuracy	+17%
Task Success Rate	Up to +26%

Citation

@article{agentdebug2025,
title={Where LLM Agents Fail and How They Can Learn From Failures},
author={Zhu, Kunlun and Liu, Zijia and Li, Bingxuan and Tian, Muxin and Yang Yingxuan and Zhang, Jiaxun and others},
journal={arXiv preprint arXiv:2509.25370},
year={2025}
}

License

This project is licensed under the MIT License - see the LICENSE file for details.

Contributing

We welcome contributions! Please feel free to submit issues, create pull requests, or reach out for collaborations.