Skip to content
BranchesTags

Repository files navigation

FastAI 0.1.13 — Unified AI client for Java

StatusLicense: MITJavaPlatformJitPack

💡 One interface for all Local, Gateway and Cloud AI models — No JSON, No HTTP, No Boilerplate.

FastAI is a minimalist, hyper-fast Java AI library that unifies 20+ major LLM providers (OmniRoute, Groq, Cerebras, SambaNova, GitHub Models, NVIDIA NIM, SiliconFlow, Cloudflare, Together AI, Fireworks, LarpRouter, Ollama, LM Studio, OpenAI, OpenRouter.ai, Claude, Mistral, DeepSeek, Gemini) behind a single, elegant interface. Built for Java developers who hate JSON parsing, HTTP clients, and bloated frameworks.

If you need a drop-in AI module, multi-provider interchangeability, or clean FastJava-style code, FastAI is your solution.

FastAI Showcase

Quick Start

importfastai.AI;
importfastai.FastAI;
publicclassQuickStartDemo {
publicstaticvoidmain(String[] args) {
// 1. Direct Local GGUF Engine with full Vulkan/Metal GPU Offloading (ON)AIgpuAI = FastAI.connect("llama:models/qwen2.5-coder-1.5b.gguf")
.withGpu(true) // Full GPU acceleration ON (99 layers)
.withContextSize(2048);
// 2. Direct Local GGUF Engine on CPU Only (GPU OFF)AIcpuAI = FastAI.connect("llama:models/qwen2.5-coder-1.5b.gguf")
.withGpu(false); // Force CPU execution (0 GPU layers)// 3. Fluid streaming with full sampling controlgpuAI.withSystemPrompt("You are an expert Java performance engineer.")
.withTemperature(0.7f)
.withTopP(0.9f)
.withTopK(40)
.withMaxTokens(256)
.stream("Write a quicksort in Java:", token -> {
System.out.print(token);
System.out.flush();
});
// 4. Cloud provider instance with identical sampling & streaming interfaceAIcloudAI = FastAI.connect("openai:gpt-4o", System.getenv("OPENAI_API_KEY"));
cloudAI.withTemperature(0.2f)
.withMaxTokens(500)
.stream("Summarize the latest 2026 tech trends:", token -> {
System.out.print(token);
System.out.flush();
});
}
}

Table of Contents

Why FastAI?

Current AI libraries in Java (LangChain4j, Spring AI) are huge, framework-heavy, and come with dependency hell. Direct SDKs lock you into one provider.

FastAI solves this by providing:

  • Zero JSON handling — everything is native Java Strings and Files.
  • Provider Interchangeability — switch between ollama, openrouter, groq, cerebras, sambanova and openai by changing one string.
  • Zero Dependencies — pure Java 17+, no Jackson, no Spring.
  • True Unified Interface — AI is all you need to know.

Key Features

  • 🌐 Local + Cloud Support — Use local models, Groq, Cerebras, Gemini, GitHub Models, or cloud giants with the exact same code.
  • ⚡ In-Process Local GPU Engine — Direct zero-IPC local LLM inference via FastAIModel with Vulkan (Intel/AMD/NVIDIA) and Metal (Apple Silicon) GPU offloading.
  • 🔄 Auto-Fallback Free Router — FastAI.auto() seamlessly cascades across free-tier providers on rate limits or outages with circuit-breaking health caches.
  • 📎 Simple Attachments — Pass a java.io.File and let FastAI handle the Base64/Multipart encoding.
  • 🎭 System Prompts — Native support for System vs User prompts.
  • ⚡ Ultra-Lightweight — Just drop the JAR into your project.
  • 🌊 Streaming First — Every provider supports unified streaming callbacks.

Installation

Option 1: Maven (Recommended)

Add the JitPack repository and the dependencies to your pom.xml:

<repositories>
<repository>
<id>jitpack.io</id>
<url>https://jitpack.io</url>
</repository>
</repositories>
<dependencies>
<!-- FastAI Library -->
<dependency>
<groupId>com.github.andrestubbe</groupId>
<artifactId>fastai</artifactId>
<version>0.1.13</version>
</dependency>
<!-- FastJSON (Required Dependency) -->
<dependency>
<groupId>com.github.andrestubbe</groupId>
<artifactId>FastJSON</artifactId>
<version>0.1.3</version>
</dependency>
<!-- FastAIModel (Local In-Process GPU Engine) -->
<dependency>
<groupId>com.github.andrestubbe.FastAIModel</groupId>
<artifactId>fastaimodel-llama</artifactId>
<version>0.1.4</version>
</dependency>
</dependencies>

API Reference

Connect

// Auto-Fallback Free Router (Cycles through free providers on rate-limit/errors)AIai = FastAI.auto();
AIai = FastAI.connect("auto:free");
// Local Providers & In-Process GPU EngineAIai = FastAI.connect("llama:models/qwen2.5-coder-1.5b.gguf"); // Native Vulkan/Metal In-Process GPU EngineAIai = FastAI.connect("ollama:llama3.1");
AIai = FastAI.connect("lmstudio:phi3");
// GatewaysAIai = FastAI.connect("omniroute:claude-3-5-sonnet"); AIai = FastAI.connect("omniroute:deepseek-r1", "apiKey", "http://localhost:8000/v1");
AIai = FastAI.connect("larprouter:gpt-5.6-sol", "sk-larp-...");
// Free-Tier / High-Speed Cloud ProvidersAIai = FastAI.connect("groq:llama-3.3-70b-versatile", "gsk_...");
AIai = FastAI.connect("cerebras:llama3.1-70b", "csk-...");
AIai = FastAI.connect("sambanova:Meta-Llama-3.1-70B-Instruct", "key...");
// OpenRouter Unified Gateway (200+ models)AIai = FastAI.connect("openrouter:anthropic/claude-3.5-sonnet", "sk-or-...");
AIai = FastAI.connect("openrouter:deepseek/deepseek-r1", "sk-or-...");
// Cloud Providers (requires API Key as second argument)AIai = FastAI.connect("openai:gpt-4o", "sk-...");
AIai = FastAI.connect("claude:opus", "sk-ant-...");
AIai = FastAI.connect("mistral:large", "key...");
AIai = FastAI.connect("deepseek:chat", "key...");
AIai = FastAI.connect("gemini:gemini-1.5-flash", "AIzaSy...");

Generation & Prompting

// Simple promptStringanswer = ai.ask("Hello!");
// System + User promptStringanswer = ai.ask("You are a math expert.", "Explain integrals.");
// Multimodal (Vision/Files)Stringanswer = ai.ask("What is in this image?", newFile("diagram.png"));

Streaming

ai.stream("Write a poem", System.out::print);

Real-World Production Patterns

1. Auto-Healing Resilient Cloud Agent

// Automatically switches to next free provider if rate-limitedAIai = FastAI.auto();
Stringresult = ai.ask("Analyze this stack trace and suggest a patch:\n" + stackTrace);

2. Local-First Air-Gapped Code Completion (0 IPC, GPU Accelerated)

try (AIlocalEngine = FastAI.connect("llama:models/qwen2.5-coder-1.5b.gguf")) {
localEngine.stream("public static <T> List<T> reverse(List<T> list) {", System.out::print);
}

3. Real-Time Multimodal Vision Inspector

AIvisionAI = FastAI.connect("gemini:gemini-2.0-flash", apiKey);
Stringcaption = visionAI.ask("Describe the UI anomalies in this rendering:", newFile("render_output.png"));

Providers Supported

ProviderTypeTierFeatures
OllamaLocalUNLIMITED LOCALChat, Streaming, List Models
llama.cppLocalUNLIMITED LOCALGGUF Local Inference (CPU/GPU)
LM StudioLocalUNLIMITED LOCALChat, Streaming via Local API
GroqCloudPERMANENT_FREEUltra-Fast Inference, Rate-limited Free
CerebrasCloudPERMANENT_FREEUltra-Fast Inference, Free Tier
SambaNovaCloudPERMANENT_FREEFast Llama/Qwen Inference, Free Tier
GeminiCloudPERMANENT_FREEChat, Streaming, List Models
GitHub ModelsCloudPERMANENT_FREEGPT-4o-mini, Llama, Free Rate-Limits
Cloudflare AICloudPERMANENT_FREEWorkers AI Llama, Daily Free Tokens
MistralCloudPERMANENT_FREEChat, Streaming, Free Experiment Tier
OpenRouterGatewayFREE / PAIDChat, Streaming, 200+ Models (:free)
OmniRouteGatewayTRIAL / PRO340+ Providers, Auto-Fallback, Compress
LarpRouterGatewayTRIAL ($0.10)Multi-Model Routing, OpenAI compatible
SiliconFlowCloudTRIAL / FREEDeepSeek V3/R1, Qwen Fast Endpoints
NVIDIA NIMCloudTRIAL (1k req)Enterprise Model APIs, Free Credits
Together AICloudTRIAL / PAIDBroad Open-Source Models, Fast Speed
Fireworks AICloudTRIAL / PAIDOptimized Inference, Function Calling
DeepSeekCloudTRIAL / PAIDChat, Streaming, Low Cost
OpenAICloudPAIDChat, Streaming
Anthropic ClaudeCloudPAIDChat, Streaming

Tier Legend:

  • UNLIMITED LOCAL: Runs 100% locally on your machine without external costs or rate limits.
  • PERMANENT_FREE: Ongoing free request quota/rate limits (ideal for agent loops & tests without subscription).
  • TRIAL: Starts with free introductory balance/tokens before requiring a top-up.
  • PAID: Requires paid account/credit from the first request.

Performance Benchmarks

FastAI is rigorously profiled using JMH to guarantee zero overhead on network parsing and request transformation.

Metric / Hot-Path OperationScore (ops/ms)Ops per Second
Usage Parser~12,748 ops/ms> 12.7 Million
SSE Byte-Level Stream Decoder~63.98 ops/ms> 63,900 ops/sec

Measured on Windows 11, Intel Core i5-1135G7 (Surface Pro 8), JDK 21.0.12. Evaluates continuous multi-chunk SSE decoding and inline usage metrics extraction without intermediate String line allocations.

Framework Comparison

FastAI is zero-dependency and zero-allocation for the core connection layer:

MetricLangChain4jSpring AIFastAI
Dependencies15+20+0
JAR Size~5MB~10MB~50KB
Startup Time2-3s5-10s<100ms
Memory OverheadHighHighMinimal
Learning CurveHoursHours5 minutes

Technical Examples & Demos

CaseJava ExampleLauncherDescription
Unified AI DemoDemo.javarun-demo.batUniversal CLI demo supporting all 20+ local and cloud providers with real-time streaming.
JMH MicrobenchmarksFastAIBenchmark.javarun-benchmark.batJMH throughput benchmark for Byte-Level SSE Streaming and Usage parsing.

API Quick Reference

Method / FactoryReturn TypeDescription
FastAI.connect(spec, args...)AIConnects to a provider (e.g. "gemini", "ollama") with optional API key.
ai.ask(prompt)StringSends a simple user prompt to the model and returns the response.
ai.ask(systemPrompt, userPrompt)StringSends a prompt with a configured system instruction.
ai.ask(prompt, file)StringSends a prompt along with a file attachment (images/vision).
ai.stream(prompt, handler)voidStreams tokens back dynamically as they are generated.
ai.getModels()List<String>Lists all available models from the provider.

Documentation

  • COMPILE.md: Full compilation and build guide.
  • REFERENCE.md: API reference for factory methods, sampling controls, and streaming contracts.
  • PHILOSOPHY.md: Unified fluent design and zero-dependency architecture.
  • ROADMAP.md: Planned milestone features and gateway extensions.
  • CHANGELOG.md: Version history and release notes.

Platform Support

PlatformStatus
Windows 10/11✅ Fully Supported
Linux🚧 Planned
macOS🚧 Planned

License

MIT License See LICENSE file for details.

Related Projects

  • FastAI — Unified AI client interface for Java
  • FastAIAgent — Autonomous agent loop, intent-graphs, and tool execution
  • FastAIBot — Zero-bloat bot harnesses and persona runtime
  • FastAIGraph — In-memory knowledge graph and multi-hop relationship engine
  • FastAIHybrid — Dense-sparse hybrid search fusion (BM25 + Vectors)
  • FastAIMCP — Model Context Protocol (MCP) server & tool integration
  • FastAIMemory — Conversation history, sliding windows, and rolling summaries
  • FastAIModel — Native local inference runtime (GGUF/ONNX)
  • FastAIRag — Ultra-fast document chunking and vector retrieval
  • FastAIReasoner — Deterministic planning, chain-of-thought, and self-correction
  • FastAIRerank — Cross-encoder relevance filtering and Top-N prompt pruner
  • FastAIRuntime — Sandboxed process runner and tool-calling execution pipeline
  • FastAIVectorDB — High-throughput SIMD/AVX2 vector database
  • FastCore — Unified JNI loader and platform abstraction

Part of the FastJava Ecosystem — Making the JVM faster. Small package. Maximum speed. Zero bloat. 🚀📋

About

🤖 Unified AI client for Java. One interface for all Local and Cloud AI models. No JSON, No HTTP boilerplate.

github.com/andrestubbe

Topics

aianthropiccloud-aijavajitpackllmlocal-aiollamaopenaiperformanceunified-api

Resources

Readme
MIT license
Activity

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages