Currently natively supports: OpenAI, Anthropic, Gemini, xAI, Ollama, Groq, DeepSeek, Cohere, Together, Fireworks, Nebius, Mimo, Zai (Zhipu AI), BigModel, Kimi (Moonshot AI), Bedrock (AWS), Cerebras, OpenRouter.
Also supports a custom URL with ServiceTargetResolver (see examples/c06-target-resolver.rs).
Provides a single, ergonomic API for many generative AI providers, such as Anthropic, OpenAI, Gemini, xAI, Ollama, Groq, and more.
NOTE: This is the Terraphim fork synced to upstream v0.6.0-beta.8, adding Kimi (Moonshot AI), AWS Bedrock, Cerebras, and OpenRouter adapters, plus SSE streaming improvements.
Docs for LLMs | CHANGELOG | BIG THANKS
- What's new:
- Upstream sync: Merged upstream v0.6.0-beta.8 breaking changes (Adapter trait, AuthData::None, StopReason, ChatMessage::tool_response, ReasoningContent).
- New Adapter: Kimi (Moonshot AI): Full chat and streaming support via
kimi::namespace. - New Adapter: Bedrock (AWS): SigV4-signed requests for AWS Bedrock models.
- New Adapter: Cerebras: Fast inference provider support.
- New Adapter: OpenRouter: Multi-provider routing via
openrouter::namespace. - Zai URL fix: Resolved trailing slash issues in Zai adapter endpoint construction.
- SSE streaming improvements: Adapter-specific SSE parsing for Kimi and Zai streaming responses.
- Bearer token auth: Added BearerToken variant to AuthData for providers requiring it.
- What's still awesome:
- Normalized and ergonomic Chat API across all major providers.
- Native protocol support for Gemini and Anthropic protocols (Reasoning/Thinking controls).
- PDF, image, and embedding support.
- Custom auth, endpoint, and header overrides.
See CHANGELOG
- Check out AIPACK, which wraps this genai library into an agentic runtime to run, build, and share AI Agent Packs. See
pro@coderfor a simple example of how I use AI PACK/genai for production coding.
Note: Feel free to send me a short description and a link to your application or library that uses genai.
- Native Multi-AI Provider/Model: OpenAI, Anthropic, Gemini, Ollama, Groq, xAI, DeepSeek (direct chat and streaming) (see examples/c00-readme.rs)
- DeepSeekR1 support, with
reasoning_content(and stream support), plus DeepSeek Groq and Ollama support (andreasoning_contentnormalization) - Image Analysis (for OpenAI, Gemini flash-2, Anthropic) (see examples/c07-image.rs)
- Custom Auth/API Key (see examples/c02-auth.rs)
- Model aliases (see examples/c05-model-names.rs)
- Custom endpoint, auth, and model identifier (see examples/c06-target-resolver.rs)
Examples | Thanks | Library Focus | Changelog | Provider Mapping: ChatOptions | Usage
//! Base examples demonstrating the core capabilities of genaiuse genai::chat::printer::{print_chat_stream,PrintChatStreamOptions};use genai::chat::{ChatMessage,ChatRequest};use genai::Client;constMODEL_OPENAI:&str = "gpt-4o-mini";// o1-mini, gpt-4o-miniconstMODEL_ANTHROPIC:&str = "claude-3-haiku-20240307";// or namespaced with simple name "fireworks::qwen3-30b-a3b", or "fireworks::accounts/fireworks/models/qwen3-30b-a3b"constMODEL_FIREWORKS:&str = "accounts/fireworks/models/qwen3-30b-a3b";constMODEL_TOGETHER:&str = "together::openai/gpt-oss-20b";constMODEL_GEMINI:&str = "gemini-2.0-flash";constMODEL_GROQ:&str = "llama-3.1-8b-instant";constMODEL_OLLAMA:&str = "gemma:2b";// sh: `ollama pull gemma:2b`constMODEL_XAI:&str = "grok-3-mini";constMODEL_DEEPSEEK:&str = "deepseek-chat";constMODEL_ZAI:&str = "glm-4-plus";constMODEL_COHERE:&str = "command-r7b-12-2024";// NOTE: These are the default environment keys for each AI Adapter Type.// They can be customized; see `examples/c02-auth.rs`constMODEL_AND_KEY_ENV_NAME_LIST:&[(&str,&str)] = &[// -- De/activate models/providers(MODEL_OPENAI,"OPENAI_API_KEY"),(MODEL_ANTHROPIC,"ANTHROPIC_API_KEY"),(MODEL_GEMINI,"GEMINI_API_KEY"),(MODEL_FIREWORKS,"FIREWORKS_API_KEY"),(MODEL_TOGETHER,"TOGETHER_API_KEY"),(MODEL_GROQ,"GROQ_API_KEY"),(MODEL_XAI,"XAI_API_KEY"),(MODEL_DEEPSEEK,"DEEPSEEK_API_KEY"),(MODEL_OLLAMA,""),(MODEL_ZAI,"ZAI_API_KEY"),(MODEL_COHERE,"COHERE_API_KEY"),];// NOTE: Model to AdapterKind (AI Provider) type mapping rule// - starts_with "gpt" -> OpenAI// - starts_with "claude" -> Anthropic// - starts_with "command" -> Cohere// - starts_with "gemini" -> Gemini// - model in Groq models -> Groq// - starts_with "glm" -> ZAI// - For anything else -> Ollama//// This can be customized; see `examples/c03-mapper.rs`#[tokio::main]asyncfnmain() -> Result<(),Box<dyn std::error::Error>>{let question = "Why is the sky red?";let chat_req = ChatRequest::new(vec![// -- Messages (de/activate to see the differences)ChatMessage::system("Answer in one sentence"),ChatMessage::user(question),]);let client = Client::default();let print_options = PrintChatStreamOptions::from_print_events(false);for(model, env_name)inMODEL_AND_KEY_ENV_NAME_LIST{// Skip if the environment name is not setif !env_name.is_empty() && std::env::var(env_name).is_err(){println!("===== Skipping model: {model} (env var not set: {env_name})");continue;}let adapter_kind = client.resolve_service_target(model).await?.model.adapter_kind;println!("\n===== MODEL: {model} ({adapter_kind}) =====");println!("\n--- Question:\n{question}");println!("\n--- Answer:");let chat_res = client.exec_chat(model, chat_req.clone(),None).await?;println!("{}", chat_res.first_text().unwrap_or("NO ANSWER"));println!("\n--- Answer: (streaming)");let chat_res = client.exec_chat_stream(model, chat_req.clone(),None).await?;print_chat_stream(chat_res,Some(&print_options)).await?;println!();}Ok(())}- examples/c00-readme.rs - Quick overview code with multiple providers and streaming.
- examples/c01-conv.rs - Shows how to build a conversation flow.
- examples/c02-auth.rs - Demonstrates how to provide a custom
AuthResolverto provide auth data (i.e., for api_key) per adapter kind. - examples/c03-mapper.rs - Demonstrates how to provide a custom
AdapterKindResolverto customize the "model name" to "adapter kind" mapping. - examples/c04-chat-options.rs - Demonstrates how to set chat generation options such as
temperatureandmax_tokensat the client level (for all requests) and at the per-request level. - examples/c05-model-names.rs - Shows how to get model names per AdapterKind.
- examples/c06-target-resolver.rs - For custom auth, endpoint, and model.
- examples/c07-image.rs - Image analysis support
genai live coding, code design, & best practices
- Adding Gemini Structured Output (vid-0060)
- Adding OpenAI Structured Output (vid-0059)
- Splitting the json value extension trait to its own public crate value-extvalue-ext
- (part 1/3) Module, Error, constructors/builders
- (part 2/3) Extension Traits, Project Files, Versioning
- (part 3/3) When to Async? Project Files, Versioning strategy
Focuses on standardizing chat completion APIs across major AI services.
Native implementation, meaning no per-service SDKs.
- Reason: While there are some variations across the various APIs, they all follow the same pattern and high-level flow and constructs. Managing the differences at a lower layer is actually simpler and more cumulative across services than doing SDK gymnastics.
Prioritizes ergonomics and commonality, with depth being secondary. (If you require a complete client API, consider using async-openai and ollama-rs; they are both excellent and easy to use.)
Initially, this library will mostly focus on text chat APIs, with images and function calling coming later.
- (1) - OpenAI-compatible notes
- Models: OpenAI, DeepSeek, Groq, Ollama, xAI, Mimo, Together, Fireworks, Nebius, Zai, Together, Fireworks, Nebius, Zai
| Property | OpenAI Compatibles (*1) | Anthropic | Gemini generationConfig. | Cohere |
|---|---|---|---|---|
temperature | temperature | temperature | temperature | temperature |
max_tokens | max_tokens | max_tokens (default 1024) | maxOutputTokens | max_tokens |
top_p | top_p | top_p | topP | p |
| Property | OpenAI Compatibles (1) | Anthropic usage. | Gemini usageMetadata. | Cohere meta.tokens. |
|---|---|---|---|---|
prompt_tokens | prompt_tokens | input_tokens (added) | promptTokenCount (2) | input_tokens |
completion_tokens | completion_tokens | output_tokens (added) | candidatesTokenCount (2) | output_tokens |
total_tokens | total_tokens | (computed) | totalTokenCount (2) | (computed) |
prompt_tokens_details | prompt_tokens_details | cached/cache_creation | N/A for now | N/A for now |
completion_tokens_details | completion_tokens_details | N/A for now | N/A for now | N/A for now |
(1) - OpenAI-compatible notes
- Models: OpenAI, DeepSeek, Groq, Ollama, xAI, Mimo
- For Groq, the property
x_groq.usage. - At this point, Ollama does not emit input/output tokens when streaming due to a limitation in the Ollama OpenAI compatibility layer. (see ollama #4448 - Streaming Chat Completion via OpenAI API should support stream option to include Usage)
prompt_tokens_detailsandcompletion_tokens_detailswill have the value sent by the compatible provider (or None)
(2): Gemini tokens
- Right now, with the Gemini Stream API, it's not clear whether usage for each event is cumulative or must be summed. It appears to be cumulative, meaning the last message shows the total number of input, output, and total tokens, so that is the current assumption. See possible tweet answer for more info.
- Will add more data to ChatResponse and ChatStream, especially usage metadata.
- Add vision/image support to chat messages and responses.
- Add function calling support to chat messages and responses.
- Add
embedandembed_batch. - Add the AWS Bedrock variants (e.g., Mistral and Anthropic). Most of the work will be on the "interesting" token signature scheme. To avoid bringing in large SDKs, this might be a lower-priority feature.
- Add the Google Vertex AI variants.
- May add the Azure OpenAI variant (not sure yet).
- crates.io: crates.io/crates/genai
- GitHub: github.com/jeremychone/rust-genai
- Sponsored by BriteSnow (Jeremy Chone's consulting company)