FastEmbed-rs 🦀

Rust implementation of @qdrant/fastembed

🍕 Features

Supports synchronous usage. No dependency on Tokio.
Uses @pykeio/ort for performant ONNX inference.
Uses @huggingface/tokenizers for fast encodings.
Supports batch embeddings generation with parallelism using @rayon-rs/rayon.

The default model is Flag Embedding, which is top of the MTEB leaderboard.

🔍 Not looking for Rust?

Python 🐍: fastembed
Go 🐳: fastembed-go
JavaScript 🌐: fastembed-js

🤖 Models

Text Embedding

Reranking

BAAI/bge-reranker-base

🚀 Installation

Run the following command in your project directory:

cargo add fastembed

Or add the following line to your Cargo.toml:

[dependencies]
fastembed = "3"

📖 Usage

Generating Text Embeddings

use fastembed::{TextEmbedding,InitOptions,EmbeddingModel};// With default InitOptionslet model = TextEmbedding::try_new(Default::default())?;// With custom InitOptionslet model = TextEmbedding::try_new(InitOptions{model_name:EmbeddingModel::AllMiniLML6V2,show_download_progress:true,
..Default::default()})?;let documents = vec!["passage: Hello, World!","query: Hello, World!","passage: This is an example passage.",// You can leave out the prefix but it's recommended"fastembed-rs is licensed under Apache 2.0"];// Generate embeddings with the default batch size, 256let embeddings = model.embed(documents,None)?;println!("Embeddings length: {}", embeddings.len());// -> Embeddings length: 4println!("Embedding dimension: {}", embeddings[0].len());// -> Embedding dimension: 384

Candidates Reranking

use fastembed::{TextRerank,RerankInitOptions,RerankerModel};let model = TextRerank::try_new(RerankInitOptions{model_name:RerankerModel::BGERerankerBase,show_download_progress:true,
..Default::default()}).unwrap();let documents = vec!["hi","The giant panda (Ailuropoda melanoleuca), sometimes called a panda bear, is a bear species endemic to China.","panda is animal","i dont know","kind of mammal",];// Rerank with the default batch sizelet results = model.rerank("what is panda?", documents,true,None);println!("Rerank result: {:?}", results);

Alternatively, raw .onnx files can be loaded through the UserDefinedEmbeddingModel struct (for "bring your own" text embedding models) using TextEmbedding::try_new_from_user_defined(...).

🚒 Under the hood

Why fast?

It's important we justify the "fast" in FastEmbed. FastEmbed is fast because:

Quantized model weights
ONNX Runtime which allows for inference on CPU, GPU, and other dedicated runtimes

Why light?

No hidden dependencies via Huggingface Transformers

Why accurate?

Better than OpenAI Ada-002
Top of the Embedding leaderboards e.g. MTEB

📄 LICENSE

FastEmbed-rs 🦀

Rust implementation of @qdrant/fastembed

🍕 Features

Supports synchronous usage. No dependency on Tokio.
Uses @pykeio/ort for performant ONNX inference.
Uses @huggingface/tokenizers for fast encodings.
Supports batch embeddings generation with parallelism using @rayon-rs/rayon.

The default model is Flag Embedding, which is top of the MTEB leaderboard.

🔍 Not looking for Rust?

Python 🐍: fastembed
Go 🐳: fastembed-go
JavaScript 🌐: fastembed-js

🤖 Models

Text Embedding

Reranking

BAAI/bge-reranker-base

🚀 Installation

Run the following command in your project directory:

cargo add fastembed

Or add the following line to your Cargo.toml:

[dependencies]
fastembed = "3"

📖 Usage

Generating Text Embeddings

use fastembed::{TextEmbedding,InitOptions,EmbeddingModel};// With default InitOptionslet model = TextEmbedding::try_new(Default::default())?;// With custom InitOptionslet model = TextEmbedding::try_new(InitOptions{model_name:EmbeddingModel::AllMiniLML6V2,show_download_progress:true,
..Default::default()})?;let documents = vec!["passage: Hello, World!","query: Hello, World!","passage: This is an example passage.",// You can leave out the prefix but it's recommended"fastembed-rs is licensed under Apache 2.0"];// Generate embeddings with the default batch size, 256let embeddings = model.embed(documents,None)?;println!("Embeddings length: {}", embeddings.len());// -> Embeddings length: 4println!("Embedding dimension: {}", embeddings[0].len());// -> Embedding dimension: 384

Candidates Reranking

use fastembed::{TextRerank,RerankInitOptions,RerankerModel};let model = TextRerank::try_new(RerankInitOptions{model_name:RerankerModel::BGERerankerBase,show_download_progress:true,
..Default::default()}).unwrap();let documents = vec!["hi","The giant panda (Ailuropoda melanoleuca), sometimes called a panda bear, is a bear species endemic to China.","panda is animal","i dont know","kind of mammal",];// Rerank with the default batch sizelet results = model.rerank("what is panda?", documents,true,None);println!("Rerank result: {:?}", results);

Alternatively, raw .onnx files can be loaded through the UserDefinedEmbeddingModel struct (for "bring your own" text embedding models) using TextEmbedding::try_new_from_user_defined(...).

🚒 Under the hood

Why fast?

It's important we justify the "fast" in FastEmbed. FastEmbed is fast because:

Quantized model weights
ONNX Runtime which allows for inference on CPU, GPU, and other dedicated runtimes

Why light?

No hidden dependencies via Huggingface Transformers

Why accurate?

Better than OpenAI Ada-002
Top of the Embedding leaderboards e.g. MTEB

📄 LICENSE

FastEmbed-rs 🦀

Rust implementation of @qdrant/fastembed

🍕 Features

Supports synchronous usage. No dependency on Tokio.
Uses @pykeio/ort for performant ONNX inference.
Uses @huggingface/tokenizers for fast encodings.
Supports batch embeddings generation with parallelism using @rayon-rs/rayon.

The default model is Flag Embedding, which is top of the MTEB leaderboard.

🔍 Not looking for Rust?

Python 🐍: fastembed
Go 🐳: fastembed-go
JavaScript 🌐: fastembed-js

🤖 Models

Text Embedding

Reranking

BAAI/bge-reranker-base

🚀 Installation

Run the following command in your project directory:

cargo add fastembed

Or add the following line to your Cargo.toml:

[dependencies]
fastembed = "3"

📖 Usage

Generating Text Embeddings

use fastembed::{TextEmbedding,InitOptions,EmbeddingModel};// With default InitOptionslet model = TextEmbedding::try_new(Default::default())?;// With custom InitOptionslet model = TextEmbedding::try_new(InitOptions{model_name:EmbeddingModel::AllMiniLML6V2,show_download_progress:true,
..Default::default()})?;let documents = vec!["passage: Hello, World!","query: Hello, World!","passage: This is an example passage.",// You can leave out the prefix but it's recommended"fastembed-rs is licensed under Apache 2.0"];// Generate embeddings with the default batch size, 256let embeddings = model.embed(documents,None)?;println!("Embeddings length: {}", embeddings.len());// -> Embeddings length: 4println!("Embedding dimension: {}", embeddings[0].len());// -> Embedding dimension: 384

Candidates Reranking

use fastembed::{TextRerank,RerankInitOptions,RerankerModel};let model = TextRerank::try_new(RerankInitOptions{model_name:RerankerModel::BGERerankerBase,show_download_progress:true,
..Default::default()}).unwrap();let documents = vec!["hi","The giant panda (Ailuropoda melanoleuca), sometimes called a panda bear, is a bear species endemic to China.","panda is animal","i dont know","kind of mammal",];// Rerank with the default batch sizelet results = model.rerank("what is panda?", documents,true,None);println!("Rerank result: {:?}", results);

Alternatively, raw .onnx files can be loaded through the UserDefinedEmbeddingModel struct (for "bring your own" text embedding models) using TextEmbedding::try_new_from_user_defined(...).

🚒 Under the hood

Why fast?

It's important we justify the "fast" in FastEmbed. FastEmbed is fast because:

Quantized model weights
ONNX Runtime which allows for inference on CPU, GPU, and other dedicated runtimes

Why light?

No hidden dependencies via Huggingface Transformers

Why accurate?

Better than OpenAI Ada-002
Top of the Embedding leaderboards e.g. MTEB

📄 LICENSE

FastEmbed-rs 🦀

Rust implementation of @qdrant/fastembed

🍕 Features

Supports synchronous usage. No dependency on Tokio.
Uses @pykeio/ort for performant ONNX inference.
Uses @huggingface/tokenizers for fast encodings.
Supports batch embeddings generation with parallelism using @rayon-rs/rayon.

The default model is Flag Embedding, which is top of the MTEB leaderboard.

🔍 Not looking for Rust?

Python 🐍: fastembed
Go 🐳: fastembed-go
JavaScript 🌐: fastembed-js

🤖 Models

Text Embedding

Reranking

BAAI/bge-reranker-base

🚀 Installation

Run the following command in your project directory:

cargo add fastembed

Or add the following line to your Cargo.toml:

[dependencies]
fastembed = "3"

📖 Usage

Generating Text Embeddings

use fastembed::{TextEmbedding,InitOptions,EmbeddingModel};// With default InitOptionslet model = TextEmbedding::try_new(Default::default())?;// With custom InitOptionslet model = TextEmbedding::try_new(InitOptions{model_name:EmbeddingModel::AllMiniLML6V2,show_download_progress:true,
..Default::default()})?;let documents = vec!["passage: Hello, World!","query: Hello, World!","passage: This is an example passage.",// You can leave out the prefix but it's recommended"fastembed-rs is licensed under Apache 2.0"];// Generate embeddings with the default batch size, 256let embeddings = model.embed(documents,None)?;println!("Embeddings length: {}", embeddings.len());// -> Embeddings length: 4println!("Embedding dimension: {}", embeddings[0].len());// -> Embedding dimension: 384

Candidates Reranking

use fastembed::{TextRerank,RerankInitOptions,RerankerModel};let model = TextRerank::try_new(RerankInitOptions{model_name:RerankerModel::BGERerankerBase,show_download_progress:true,
..Default::default()}).unwrap();let documents = vec!["hi","The giant panda (Ailuropoda melanoleuca), sometimes called a panda bear, is a bear species endemic to China.","panda is animal","i dont know","kind of mammal",];// Rerank with the default batch sizelet results = model.rerank("what is panda?", documents,true,None);println!("Rerank result: {:?}", results);

Alternatively, raw .onnx files can be loaded through the UserDefinedEmbeddingModel struct (for "bring your own" text embedding models) using TextEmbedding::try_new_from_user_defined(...).

🚒 Under the hood

Why fast?

It's important we justify the "fast" in FastEmbed. FastEmbed is fast because:

Quantized model weights
ONNX Runtime which allows for inference on CPU, GPU, and other dedicated runtimes

Why light?

No hidden dependencies via Huggingface Transformers

Why accurate?

Better than OpenAI Ada-002
Top of the Embedding leaderboards e.g. MTEB

📄 LICENSE

FastEmbed-rs 🦀

Rust implementation of @qdrant/fastembed

🍕 Features

Supports synchronous usage. No dependency on Tokio.
Uses @pykeio/ort for performant ONNX inference.
Uses @huggingface/tokenizers for fast encodings.
Supports batch embeddings generation with parallelism using @rayon-rs/rayon.

The default model is Flag Embedding, which is top of the MTEB leaderboard.

🔍 Not looking for Rust?

Python 🐍: fastembed
Go 🐳: fastembed-go
JavaScript 🌐: fastembed-js

🤖 Models

Text Embedding

Reranking

BAAI/bge-reranker-base

🚀 Installation

Run the following command in your project directory:

cargo add fastembed

Or add the following line to your Cargo.toml:

[dependencies]
fastembed = "3"

📖 Usage

Generating Text Embeddings

use fastembed::{TextEmbedding,InitOptions,EmbeddingModel};// With default InitOptionslet model = TextEmbedding::try_new(Default::default())?;// With custom InitOptionslet model = TextEmbedding::try_new(InitOptions{model_name:EmbeddingModel::AllMiniLML6V2,show_download_progress:true,
..Default::default()})?;let documents = vec!["passage: Hello, World!","query: Hello, World!","passage: This is an example passage.",// You can leave out the prefix but it's recommended"fastembed-rs is licensed under Apache 2.0"];// Generate embeddings with the default batch size, 256let embeddings = model.embed(documents,None)?;println!("Embeddings length: {}", embeddings.len());// -> Embeddings length: 4println!("Embedding dimension: {}", embeddings[0].len());// -> Embedding dimension: 384

Candidates Reranking

use fastembed::{TextRerank,RerankInitOptions,RerankerModel};let model = TextRerank::try_new(RerankInitOptions{model_name:RerankerModel::BGERerankerBase,show_download_progress:true,
..Default::default()}).unwrap();let documents = vec!["hi","The giant panda (Ailuropoda melanoleuca), sometimes called a panda bear, is a bear species endemic to China.","panda is animal","i dont know","kind of mammal",];// Rerank with the default batch sizelet results = model.rerank("what is panda?", documents,true,None);println!("Rerank result: {:?}", results);

Alternatively, raw .onnx files can be loaded through the UserDefinedEmbeddingModel struct (for "bring your own" text embedding models) using TextEmbedding::try_new_from_user_defined(...).

🚒 Under the hood

Why fast?

It's important we justify the "fast" in FastEmbed. FastEmbed is fast because:

Quantized model weights
ONNX Runtime which allows for inference on CPU, GPU, and other dedicated runtimes

Why light?

No hidden dependencies via Huggingface Transformers

Why accurate?

Better than OpenAI Ada-002
Top of the Embedding leaderboards e.g. MTEB

📄 LICENSE

FastEmbed-rs 🦀

Rust implementation of @qdrant/fastembed

🍕 Features

Supports synchronous usage. No dependency on Tokio.
Uses @pykeio/ort for performant ONNX inference.
Uses @huggingface/tokenizers for fast encodings.
Supports batch embeddings generation with parallelism using @rayon-rs/rayon.

The default model is Flag Embedding, which is top of the MTEB leaderboard.

🔍 Not looking for Rust?

Python 🐍: fastembed
Go 🐳: fastembed-go
JavaScript 🌐: fastembed-js

🤖 Models

Text Embedding

Reranking

BAAI/bge-reranker-base

🚀 Installation

Run the following command in your project directory:

cargo add fastembed

Or add the following line to your Cargo.toml:

[dependencies]
fastembed = "3"

📖 Usage

Generating Text Embeddings

use fastembed::{TextEmbedding,InitOptions,EmbeddingModel};// With default InitOptionslet model = TextEmbedding::try_new(Default::default())?;// With custom InitOptionslet model = TextEmbedding::try_new(InitOptions{model_name:EmbeddingModel::AllMiniLML6V2,show_download_progress:true,
..Default::default()})?;let documents = vec!["passage: Hello, World!","query: Hello, World!","passage: This is an example passage.",// You can leave out the prefix but it's recommended"fastembed-rs is licensed under Apache 2.0"];// Generate embeddings with the default batch size, 256let embeddings = model.embed(documents,None)?;println!("Embeddings length: {}", embeddings.len());// -> Embeddings length: 4println!("Embedding dimension: {}", embeddings[0].len());// -> Embedding dimension: 384

Candidates Reranking

use fastembed::{TextRerank,RerankInitOptions,RerankerModel};let model = TextRerank::try_new(RerankInitOptions{model_name:RerankerModel::BGERerankerBase,show_download_progress:true,
..Default::default()}).unwrap();let documents = vec!["hi","The giant panda (Ailuropoda melanoleuca), sometimes called a panda bear, is a bear species endemic to China.","panda is animal","i dont know","kind of mammal",];// Rerank with the default batch sizelet results = model.rerank("what is panda?", documents,true,None);println!("Rerank result: {:?}", results);

Alternatively, raw .onnx files can be loaded through the UserDefinedEmbeddingModel struct (for "bring your own" text embedding models) using TextEmbedding::try_new_from_user_defined(...).

🚒 Under the hood

Why fast?

It's important we justify the "fast" in FastEmbed. FastEmbed is fast because:

Quantized model weights
ONNX Runtime which allows for inference on CPU, GPU, and other dedicated runtimes

Why light?

No hidden dependencies via Huggingface Transformers

Why accurate?

Better than OpenAI Ada-002
Top of the Embedding leaderboards e.g. MTEB

📄 LICENSE

FastEmbed-rs 🦀

Rust implementation of @qdrant/fastembed

🍕 Features

Supports synchronous usage. No dependency on Tokio.
Uses @pykeio/ort for performant ONNX inference.
Uses @huggingface/tokenizers for fast encodings.
Supports batch embeddings generation with parallelism using @rayon-rs/rayon.

The default model is Flag Embedding, which is top of the MTEB leaderboard.

🔍 Not looking for Rust?

Python 🐍: fastembed
Go 🐳: fastembed-go
JavaScript 🌐: fastembed-js

🤖 Models

Text Embedding

Reranking

BAAI/bge-reranker-base

🚀 Installation

Run the following command in your project directory:

cargo add fastembed

Or add the following line to your Cargo.toml:

[dependencies]
fastembed = "3"

📖 Usage

Generating Text Embeddings

use fastembed::{TextEmbedding,InitOptions,EmbeddingModel};// With default InitOptionslet model = TextEmbedding::try_new(Default::default())?;// With custom InitOptionslet model = TextEmbedding::try_new(InitOptions{model_name:EmbeddingModel::AllMiniLML6V2,show_download_progress:true,
..Default::default()})?;let documents = vec!["passage: Hello, World!","query: Hello, World!","passage: This is an example passage.",// You can leave out the prefix but it's recommended"fastembed-rs is licensed under Apache 2.0"];// Generate embeddings with the default batch size, 256let embeddings = model.embed(documents,None)?;println!("Embeddings length: {}", embeddings.len());// -> Embeddings length: 4println!("Embedding dimension: {}", embeddings[0].len());// -> Embedding dimension: 384

Candidates Reranking

use fastembed::{TextRerank,RerankInitOptions,RerankerModel};let model = TextRerank::try_new(RerankInitOptions{model_name:RerankerModel::BGERerankerBase,show_download_progress:true,
..Default::default()}).unwrap();let documents = vec!["hi","The giant panda (Ailuropoda melanoleuca), sometimes called a panda bear, is a bear species endemic to China.","panda is animal","i dont know","kind of mammal",];// Rerank with the default batch sizelet results = model.rerank("what is panda?", documents,true,None);println!("Rerank result: {:?}", results);

Alternatively, raw .onnx files can be loaded through the UserDefinedEmbeddingModel struct (for "bring your own" text embedding models) using TextEmbedding::try_new_from_user_defined(...).

🚒 Under the hood

Why fast?

It's important we justify the "fast" in FastEmbed. FastEmbed is fast because:

Quantized model weights
ONNX Runtime which allows for inference on CPU, GPU, and other dedicated runtimes

Why light?

No hidden dependencies via Huggingface Transformers

Why accurate?

Better than OpenAI Ada-002
Top of the Embedding leaderboards e.g. MTEB

📄 LICENSE

FastEmbed-rs 🦀

Rust implementation of @qdrant/fastembed

🍕 Features

Supports synchronous usage. No dependency on Tokio.
Uses @pykeio/ort for performant ONNX inference.
Uses @huggingface/tokenizers for fast encodings.
Supports batch embeddings generation with parallelism using @rayon-rs/rayon.

The default model is Flag Embedding, which is top of the MTEB leaderboard.

🔍 Not looking for Rust?

Python 🐍: fastembed
Go 🐳: fastembed-go
JavaScript 🌐: fastembed-js

🤖 Models

Text Embedding

Reranking

BAAI/bge-reranker-base

🚀 Installation

Run the following command in your project directory:

cargo add fastembed

Or add the following line to your Cargo.toml:

[dependencies]
fastembed = "3"

📖 Usage

Generating Text Embeddings

use fastembed::{TextEmbedding,InitOptions,EmbeddingModel};// With default InitOptionslet model = TextEmbedding::try_new(Default::default())?;// With custom InitOptionslet model = TextEmbedding::try_new(InitOptions{model_name:EmbeddingModel::AllMiniLML6V2,show_download_progress:true,
..Default::default()})?;let documents = vec!["passage: Hello, World!","query: Hello, World!","passage: This is an example passage.",// You can leave out the prefix but it's recommended"fastembed-rs is licensed under Apache 2.0"];// Generate embeddings with the default batch size, 256let embeddings = model.embed(documents,None)?;println!("Embeddings length: {}", embeddings.len());// -> Embeddings length: 4println!("Embedding dimension: {}", embeddings[0].len());// -> Embedding dimension: 384

Candidates Reranking

use fastembed::{TextRerank,RerankInitOptions,RerankerModel};let model = TextRerank::try_new(RerankInitOptions{model_name:RerankerModel::BGERerankerBase,show_download_progress:true,
..Default::default()}).unwrap();let documents = vec!["hi","The giant panda (Ailuropoda melanoleuca), sometimes called a panda bear, is a bear species endemic to China.","panda is animal","i dont know","kind of mammal",];// Rerank with the default batch sizelet results = model.rerank("what is panda?", documents,true,None);println!("Rerank result: {:?}", results);

Alternatively, raw .onnx files can be loaded through the UserDefinedEmbeddingModel struct (for "bring your own" text embedding models) using TextEmbedding::try_new_from_user_defined(...).

🚒 Under the hood

Why fast?

It's important we justify the "fast" in FastEmbed. FastEmbed is fast because:

Quantized model weights
ONNX Runtime which allows for inference on CPU, GPU, and other dedicated runtimes

Name		Name	Last commit message	Last commit date
.github/workflows		.github/workflows
benches		benches
src		src
.gitignore		.gitignore
.releaserc		.releaserc
Cargo.toml		Cargo.toml
LICENSE		LICENSE
README.md		README.md

Name		Name	Last commit message	Last commit date
.github/workflows		.github/workflows
benches		benches
src		src
.gitignore		.gitignore
.releaserc		.releaserc
Cargo.toml		Cargo.toml
LICENSE		LICENSE
README.md		README.md

Name		Name	Last commit message	Last commit date
.github/workflows		.github/workflows
benches		benches
src		src
.gitignore		.gitignore
.releaserc		.releaserc
Cargo.toml		Cargo.toml
LICENSE		LICENSE
README.md		README.md

Name		Name	Last commit message	Last commit date
.github/workflows		.github/workflows
benches		benches
src		src
.gitignore		.gitignore
.releaserc		.releaserc
Cargo.toml		Cargo.toml
LICENSE		LICENSE
README.md		README.md

Name		Name	Last commit message	Last commit date
.github/workflows		.github/workflows
benches		benches
src		src
.gitignore		.gitignore
.releaserc		.releaserc
Cargo.toml		Cargo.toml
LICENSE		LICENSE
README.md		README.md

Uh oh!

Latest commit

History

Folders and files

Repository files navigation

FastEmbed-rs 🦀

Rust implementation of @qdrant/fastembed

🍕 Features

🔍 Not looking for Rust?

🤖 Models

Text Embedding

Reranking

🚀 Installation

📖 Usage

Generating Text Embeddings

Candidates Reranking

🚒 Under the hood

Why fast?

Why light?

Why accurate?

📄 LICENSE

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages

Uh oh!

Latest commit

History

Folders and files

Repository files navigation

FastEmbed-rs 🦀

Rust implementation of @qdrant/fastembed

🍕 Features

🔍 Not looking for Rust?

🤖 Models

Text Embedding

Reranking

🚀 Installation

📖 Usage

Generating Text Embeddings

Candidates Reranking

🚒 Under the hood

Why fast?

Why light?

Why accurate?

📄 LICENSE

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages

Uh oh!

Latest commit

History

Folders and files

Repository files navigation

FastEmbed-rs 🦀

Rust implementation of @qdrant/fastembed

🍕 Features

🔍 Not looking for Rust?

🤖 Models

Text Embedding

Reranking

🚀 Installation

📖 Usage

Generating Text Embeddings

Candidates Reranking

🚒 Under the hood

Why fast?

Why light?

Why accurate?