Agent IO

A Rust SDK for building AI agents with multi-provider LLM support.

Features

• Multi-provider LLM support: OpenAI, Anthropic, Google Gemini, and OpenAI-compatible providers
• Tool/Function calling: Built-in tool system — define tools with a simple #[tool] macro
• Streaming responses: Event-based real-time response handling
• Context compaction: Automatic management of long conversation context
• Token tracking: Usage tracking and cost calculation across providers
• Retry mechanism: Built-in exponential backoff retry for rate limit handling
• Memory system: In-memory memory by default, with optional LanceDB persistence via feature flag

Installation

[dependencies]
agent-io = { version = "0.3", features = ["openai"] }
tokio = { version = "1", features = ["full"] }

Enable memory-lancedb only if you need persistent vector-backed memory:

agent-io = { version = "0.3", features = ["openai", "memory-lancedb"] }

Quick Start

Basic Usage

use std::sync::Arc;use agent_io::{Agent, llm::ChatOpenAI};#[tokio::main]asyncfnmain() -> Result<(),Box<dyn std::error::Error>>{let llm = ChatOpenAI::new("gpt-4o")?;let agent = Agent::builder().with_llm(Arc::new(llm)).build()?;let response = agent.query("Hello!").await?;println!("{}", response);Ok(())}

Defining Tools with #[tool]

The #[tool] macro eliminates boilerplate — just write a plain async fn:

use std::sync::Arc;use agent_io::{Agent, llm::ChatOpenAI, tool};/// Get the current weather for a city#[tool(location = "The city name to fetch weather for")]asyncfnget_weather(location:String) -> agent_io::Result<String>{Ok(format!("Weather in {location}: Sunny, 25°C"))}/// Evaluate a simple arithmetic expression#[tool(expression = "The expression to evaluate, e.g. '15 * 7'")]asyncfncalculator(expression:String) -> agent_io::Result<String>{// ... implementationOk("Result: 105".to_string())}#[tokio::main]asyncfnmain() -> Result<(),Box<dyn std::error::Error>>{let llm = ChatOpenAI::new("gpt-5.4")?;let agent = Agent::builder().with_llm(Arc::new(llm)).tool(get_weather())// macro generates Arc<dyn Tool> constructor.tool(calculator()).system_prompt("You are a helpful assistant.").build()?;let response = agent.query("What's the weather in Tokyo and 15 * 7?").await?;println!("{}", response);Ok(())}

The macro automatically:

• Uses the function doc comment as the tool description
• Maps Rust types to JSON Schema types (String → "string", f64 → "number", etc.)
• Uses the attribute key=value pairs as parameter descriptions
• Generates a zero-arg constructor returning Arc<dyn Tool>

Manual Tool Definition

For more control, use FunctionTool or ToolBuilder directly:

use std::sync::Arc;use agent_io::tools::{ToolBuilder,Tool};use serde::Deserialize;#[derive(Deserialize)]structWeatherArgs{location:String}let tool:Arc<dynTool> = ToolBuilder::new("get_weather").description("Get weather for a location").string_param("location","The city name").build(|args:WeatherArgs| Box::pin(asyncmove{Ok(format!("Sunny in {}", args.location))}));

Supported Providers

Provider	Type	Environment Variable
OpenAI	ChatOpenAI	OPENAI_API_KEY
Anthropic	ChatAnthropic	ANTHROPIC_API_KEY
Google Gemini	ChatGoogle	GOOGLE_API_KEY
DeepSeek	ChatDeepSeek	DEEPSEEK_API_KEY
Groq	ChatGroq	GROQ_API_KEY
Mistral	ChatMistral	MISTRAL_API_KEY
Ollama	ChatOllama	— (local)
OpenRouter	ChatOpenRouter	OPENROUTER_API_KEY
OpenAI-compatible	ChatOpenAICompatible	configurable

Feature Flags

[dependencies.agent-io]
version = "0.3"features = ["openai", "anthropic", "google"]
# Optional persistent memory backend# features = ["openai", "memory-lancedb"]# Or enable the bundled provider set:# features = ["full"]

Examples

# Basic example (manual tool definition)
cargo run --example basic
# Macro-based tools (zero boilerplate)
cargo run --example macro_tools
# Multi-provider
cargo run --example multi_provider --features full

Workspace

This repository is a Cargo workspace:

agent-io/ # main SDK crate
agent-io-macros/ # proc-macro crate (#[tool])

License

Licensed under the Apache License 2.0.