Skip to content
BranchesTags

Repository files navigation

SeekDBClient

SeekDBClient is a unified Python client that wraps three database connection modes—embedded SeekDB, remote SeekDB servers, and OceanBase—behind a single, concise API.

Table of Contents

  1. Installation
  2. Client Connection
  3. AdminClient Connection and Database Management
  4. Collection (Table) Management
  5. DML Operations
  6. DQL Operations
  7. Embedding Functions
  8. Testing

Installation

pip install -U pyseekdb

1. Client Connection

The Client class provides a unified interface for connecting to SeekDB in different modes. It automatically selects the appropriate connection mode based on the parameters provided.

1.1 Embedded SeekDB Client

Connect to a local embedded SeekDB instance:

importpyseekdb# Create embedded client with explicit pathclient=pyseekdb.Client(
path="./seekdb", # Path to SeekDB data directorydatabase="demo"# Database name
)
# Create embedded client with default path (current working directory)# If path is not provided, uses seekdb.db in the current process working directoryclient=pyseekdb.Client(
database="demo"# Database name (path defaults to current working directory/seekdb.db)
)
# Execute SQL queriesrows=client.execute("SELECT 1")
print(rows)

1.2 Remote Server Client

Connect to a remote server (supports both SeekDB Server and OceanBase Server):

importpyseekdb# Create remote server client (SeekDB Server)client=pyseekdb.Client(
host="127.0.0.1", # Server hostport=2881, # Server port (default: 2881)tenant="sys", # Tenant name (default: "sys" for SeekDB Server)database="demo", # Database nameuser="root", # Username (default: "root")password=""# Password (can be retrieved from SEEKDB_PASSWORD environment variable)
)
# Create remote server client (OceanBase Server)client=pyseekdb.Client(
host="127.0.0.1", # Server hostport=2881, # Server port (default: 2881)tenant="test", # Tenant namedatabase="demo", # Database nameuser="root", # Username (default: "root")password=""# Password (can be retrieved from SEEKDB_PASSWORD environment variable)
)

Note: If the password parameter is not provided (empty string), the client will automatically retrieve it from the SEEKDB_PASSWORD environment variable. This is useful for keeping passwords out of your code:

export SEEKDB_PASSWORD="your_password"
# Password will be automatically retrieved from SEEKDB_PASSWORD environment variableclient=pyseekdb.Client(
host="127.0.0.1",
port=2881,
tenant="sys", # or "test" for OceanBasedatabase="demo",
user="root"# password parameter omitted - will use SEEKDB_PASSWORD from environment
)

1.4 Client Methods and Properties

Method / PropertyDescription
create_collection()Create a new collection (see Collection Management)
get_collection()Get an existing collection object
delete_collection()Delete a collection
list_collections()List all collections in the current database
has_collection()Check if a collection exists
get_or_create_collection()Get an existing collection or create it if it doesn't exist
count_collection()Count the number of collections in the current database

Note: The Client factory function returns a proxy that only exposes collection operations. For database management operations, use AdminClient (see section 2).

2. AdminClient Connection and Database Management

The AdminClient class provides database management operations. It uses the same connection modes as Client but only exposes database management methods.

2.1 Embedded/Server AdminClient

importpyseekdb# Embedded mode - Database managementadmin=pyseekdb.AdminClient(path="./seekdb")
# Remote server mode - Database management (SeekDB Server)admin=pyseekdb.AdminClient(
host="127.0.0.1",
port=2881,
tenant="sys", # Default tenant for SeekDB Serveruser="root",
password=""# Can be retrieved from SEEKDB_PASSWORD environment variable
)
# Remote server mode - Database management (OceanBase Server)admin=pyseekdb.AdminClient(
host="127.0.0.1",
port=2881,
tenant="test", # Default tenant for OceanBaseuser="root",
password=""# Can be retrieved from SEEKDB_PASSWORD environment variable
)
# Use context managerwithpyseekdb.AdminClient(host="127.0.0.1", port=2881, tenant="sys", user="root") asadmin:
# Create databaseadmin.create_database("my_database")
# List all databasesdatabases=admin.list_databases()
fordbindatabases:
print(f"Database: {db.name}")
# Get database informationdb=admin.get_database("my_database")
print(f"Database: {db.name}, Charset: {db.charset}")
# Delete databaseadmin.delete_database("my_database")

2.2 AdminClient Methods

MethodDescription
create_database(name, tenant=DEFAULT_TENANT)Create a new database (uses client's tenant for remote server mode)
get_database(name, tenant=DEFAULT_TENANT)Get database object with metadata (uses client's tenant for remote server mode)
delete_database(name, tenant=DEFAULT_TENANT)Delete a database (uses client's tenant for remote server mode)
list_databases(limit=None, offset=None, tenant=DEFAULT_TENANT)List all databases with optional pagination (uses client's tenant for remote server mode)

Parameters:

  • name (str): Database name
  • tenant (str, optional): Tenant name (uses client's tenant if different, ignored for embedded mode)
  • limit (int, optional): Maximum number of results to return
  • offset (int, optional): Number of results to skip for pagination

2.4 Database Object

The get_database() and list_databases() methods return Database objects with the following properties:

  • name (str): Database name
  • tenant (str, optional): Tenant name (None for embedded/server mode)
  • charset (str, optional): Character set
  • collation (str, optional): Collation
  • metadata (dict): Additional metadata

3. Collection (Table) Management

Collections are the primary data structures in SeekDBClient, similar to tables in traditional databases. Each collection stores documents with vector embeddings, metadata, and full-text search capabilities.

3.1 Creating a Collection

importpyseekdbfrompyseekdbimportDefaultEmbeddingFunction, HNSWConfiguration# Create a clientclient=pyseekdb.Client(host="127.0.0.1", port=2881, database="test")
# Create a collection with vector dimension (traditional way)collection=client.create_collection(
name="my_collection",
configuration=HNSWConfiguration(dimension=128, distance='cosine')
)
# Create a collection with default embedding function (auto-calculates dimension)collection=client.create_collection(
name="my_collection",
embedding_function=DefaultEmbeddingFunction() # Uses default model (384 dimensions)
)
# Create a collection with custom embedding functionef=DefaultEmbeddingFunction(model_name='all-MiniLM-L6-v2')
config=HNSWConfiguration(dimension=384, distance='cosine') # Must match EF dimensioncollection=client.create_collection(
name="my_collection",
configuration=config,
embedding_function=ef
)
# Create a collection without embedding function (embeddings must be provided manually)collection=client.create_collection(
name="my_collection",
configuration=HNSWConfiguration(dimension=128, distance='cosine'),
embedding_function=None# Explicitly disable embedding function
)
# Get or create collection (creates if doesn't exist)collection=client.get_or_create_collection(
name="my_collection",
configuration=HNSWConfiguration(dimension=128, distance='cosine'),
embedding_function=DefaultEmbeddingFunction()
)

Parameters:

  • name (str): Collection name (required)
  • configuration (HNSWConfiguration, optional): Index configuration with dimension and distance metric
    • If not provided, uses default (dimension=384, distance='cosine')
    • If set to None, dimension will be calculated from embedding_function
  • embedding_function (EmbeddingFunction, optional): Function to convert documents to embeddings
    • If not provided, uses DefaultEmbeddingFunction() (384 dimensions)
    • If set to None, collection will not have an embedding function
    • If provided, the dimension will be automatically calculated and validated against configuration.dimension

Note: When embedding_function is provided, the system will automatically calculate the vector dimension by calling the function. If configuration.dimension is also provided, it must match the embedding function's dimension, otherwise a ValueError will be raised.

3.2 Getting a Collection

# Get an existing collection (uses default embedding function if collection doesn't have one)collection=client.get_collection("my_collection")
# Get collection with specific embedding functionef=DefaultEmbeddingFunction(model_name='all-MiniLM-L6-v2')
collection=client.get_collection("my_collection", embedding_function=ef)
# Get collection without embedding functioncollection=client.get_collection("my_collection", embedding_function=None)
# Check if collection existsifclient.has_collection("my_collection"):
collection=client.get_collection("my_collection")

Parameters:

  • name (str): Collection name (required)
  • embedding_function (EmbeddingFunction, optional): Embedding function to use for this collection
    • If not provided, uses DefaultEmbeddingFunction() by default
    • If set to None, collection will not have an embedding function
    • Important: The embedding function set here will be used for all operations on this collection (add, upsert, update, query, hybrid_search) when documents/texts are provided without embeddings

3.3 Listing Collections

# List all collectionscollections=client.list_collections()
forcollincollections:
print(f"Collection: {coll.name}, Dimension: {coll.dimension}")
# Count collections in databasecollection_count=client.count_collection()
print(f"Database has {collection_count} collections")

3.4 Deleting a Collection

# Delete a collectionclient.delete_collection("my_collection")

3.5 Collection Properties

Each Collection object has the following properties:

  • name (str): Collection name
  • id (str, optional): Collection unique identifier
  • dimension (int, optional): Vector dimension
  • embedding_function (EmbeddingFunction, optional): Embedding function associated with this collection
  • metadata (dict): Collection metadata
  • client: Reference to the client that created it

Accessing Embedding Function:

collection=client.get_collection("my_collection")
ifcollection.embedding_functionisnotNone:
print(f"Collection uses embedding function: {collection.embedding_function}")
print(f"Embedding dimension: {collection.embedding_function.dimension}")

4. DML Operations

DML (Data Manipulation Language) operations allow you to insert, update, and delete data in collections.

4.1 Add Data

The add() method inserts new records into a collection. If a record with the same ID already exists, an error will be raised.

Behavior with Embedding Function:

  1. If embeddings are provided: Embeddings are used directly, embedding_function is NOT called (even if provided)
  2. If embeddings are NOT provided but documents are provided:
    • If collection has an embedding_function (set during creation or retrieval), it will automatically generate embeddings from documents
    • If collection does NOT have an embedding_function, a ValueError will be raised
  3. If neither embeddings nor documents are provided: A ValueError will be raised
# Add single item with embeddings (embedding_function not used)collection.add(
ids="item1",
embeddings=[0.1, 0.2, 0.3],
documents="This is a document",
metadatas={"category": "AI", "score": 95}
)
# Add multiple items with embeddings (embedding_function not used)collection.add(
ids=["item1", "item2", "item3"],
embeddings=[
[0.1, 0.2, 0.3],
[0.4, 0.5, 0.6],
[0.7, 0.8, 0.9]
],
documents=[
"Document 1",
"Document 2",
"Document 3"
],
metadatas=[
{"category": "AI", "score": 95},
{"category": "ML", "score": 88},
{"category": "DL", "score": 92}
]
)
# Add with only embeddings (no documents)collection.add(
ids=["vec1", "vec2"],
embeddings=[[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]]
)
# Add with only documents - embeddings auto-generated by embedding_function# Requires: collection must have embedding_function setcollection.add(
ids=["doc1", "doc2"],
documents=["Text document 1", "Text document 2"],
metadatas=[{"tag": "A"}, {"tag": "B"}]
)
# The collection's embedding_function will automatically convert documents to embeddings

Parameters:

  • ids (str or List[str]): Single ID or list of IDs (required)
  • embeddings (List[float] or List[List[float]], optional): Single embedding or list of embeddings
    • If provided, used directly (embedding_function is ignored)
    • If not provided, must provide documents and collection must have embedding_function
  • documents (str or List[str], optional): Single document or list of documents
    • If embeddings not provided, documents will be converted to embeddings using collection's embedding_function
  • metadatas (dict or List[dict], optional): Single metadata dict or list of metadata dicts

Note: The embedding_function used is the one associated with the collection (set during create_collection() or get_collection()). You cannot override it per-operation.

4.2 Update Data

The update() method updates existing records in a collection. Records must exist, otherwise an error will be raised.

Behavior with Embedding Function:

  1. If embeddings are provided: Embeddings are used directly, embedding_function is NOT called
  2. If embeddings are NOT provided but documents are provided:
    • If collection has an embedding_function, it will automatically generate embeddings from documents
    • If collection does NOT have an embedding_function, a ValueError will be raised
  3. If neither embeddings nor documents are provided: Only metadata will be updated (metadata-only update is allowed)
# Update single item - metadata only (embedding_function not used)collection.update(
ids="item1",
metadatas={"category": "AI", "score": 98} # Update metadata only
)
# Update multiple items with embeddings (embedding_function not used)collection.update(
ids=["item1", "item2"],
embeddings=[[0.9, 0.8, 0.7], [0.6, 0.5, 0.4]], # Update embeddingsdocuments=["Updated document 1", "Updated document 2"] # Update documents
)
# Update with documents only - embeddings auto-generated by embedding_function# Requires: collection must have embedding_function setcollection.update(
ids="item1",
documents="New document text", # Embeddings will be auto-generatedmetadatas={"category": "AI"}
)
# Update specific fields - only document (embeddings auto-generated)collection.update(
ids="item1",
documents="New document text"# Only update document, embeddings auto-generated
)

Parameters:

  • ids (str or List[str]): Single ID or list of IDs to update (required)
  • embeddings (List[float] or List[List[float]], optional): New embeddings
    • If provided, used directly (embedding_function is ignored)
    • If not provided, can provide documents to auto-generate embeddings
  • documents (str or List[str], optional): New documents
    • If embeddings not provided, documents will be converted to embeddings using collection's embedding_function
  • metadatas (dict or List[dict], optional): New metadata

Note: Metadata-only updates (no embeddings, no documents) are allowed. The embedding_function used is the one associated with the collection.

4.3 Upsert Data

The upsert() method inserts new records or updates existing ones. If a record with the given ID exists, it will be updated; otherwise, a new record will be inserted.

Behavior with Embedding Function:

  1. If embeddings are provided: Embeddings are used directly, embedding_function is NOT called
  2. If embeddings are NOT provided but documents are provided:
    • If collection has an embedding_function, it will automatically generate embeddings from documents
    • If collection does NOT have an embedding_function, a ValueError will be raised
  3. If neither embeddings nor documents are provided: Only metadata will be upserted (metadata-only upsert is allowed)
# Upsert single item with embeddings (embedding_function not used)collection.upsert(
ids="item1",
embeddings=[0.1, 0.2, 0.3],
documents="Document text",
metadatas={"category": "AI", "score": 95}
)
# Upsert multiple items with embeddings (embedding_function not used)collection.upsert(
ids=["item1", "item2", "item3"],
embeddings=[
[0.1, 0.2, 0.3],
[0.4, 0.5, 0.6],
[0.7, 0.8, 0.9]
],
documents=["Doc 1", "Doc 2", "Doc 3"],
metadatas=[
{"category": "AI"},
{"category": "ML"},
{"category": "DL"}
]
)
# Upsert with documents only - embeddings auto-generated by embedding_function# Requires: collection must have embedding_function setcollection.upsert(
ids=["item1", "item2"],
documents=["Document 1", "Document 2"],
metadatas=[{"category": "AI"}, {"category": "ML"}]
)
# The collection's embedding_function will automatically convert documents to embeddings

Parameters:

  • ids (str or List[str]): Single ID or list of IDs (required)
  • embeddings (List[float] or List[List[float]], optional): Embeddings
    • If provided, used directly (embedding_function is ignored)
    • If not provided, can provide documents to auto-generate embeddings
  • documents (str or List[str], optional): Documents
    • If embeddings not provided, documents will be converted to embeddings using collection's embedding_function
  • metadatas (dict or List[dict], optional): Metadata

Note: Metadata-only upserts (no embeddings, no documents) are allowed. The embedding_function used is the one associated with the collection.

4.4 Delete Data

The delete() method removes records from a collection. You can delete by IDs, metadata filters, or document filters.

# Delete by IDscollection.delete(ids=["item1", "item2", "item3"])
# Delete by single IDcollection.delete(ids="item1")
# Delete by metadata filtercollection.delete(where={"category": {"$eq": "AI"}})
# Delete by comparison operatorcollection.delete(where={"score": {"$lt": 50}})
# Delete by document filtercollection.delete(where_document={"$contains": "obsolete"})
# Delete with combined filterscollection.delete(
where={"category": {"$eq": "AI"}},
where_document={"$contains": "deprecated"}
)

Parameters:

  • ids (str or List[str], optional): Single ID or list of IDs to delete
  • where (dict, optional): Metadata filter conditions (see Filter Operators section)
  • where_document (dict, optional): Document filter conditions

Note: At least one of ids, where, or where_document must be provided.

5. DQL Operations

DQL (Data Query Language) operations allow you to retrieve data from collections using various query methods.

5.1 Query (Vector Similarity Search)

The query() method performs vector similarity search to find the most similar documents to the query vector(s).

Behavior with Embedding Function:

  1. If query_embeddings are provided: embeddings are used directly, embedding_function is NOT called
  2. If query_embeddings are NOT provided but query_texts are provided:
    • If collection has an embedding_function, it will automatically generate query embeddings from texts
    • If collection does NOT have an embedding_function, a ValueError will be raised
  3. If neither query_embeddings nor query_texts are provided: A ValueError will be raised
# Basic vector similarity query (embedding_function not used)results=collection.query(
query_embeddings=[1.0, 2.0, 3.0],
n_results=3
)
# Iterate over resultsforiinrange(len(results["ids"][0])):
print(f"ID: {results['ids'][0][i]}, Distance: {results['distances'][0][i]}")
ifresults.get("documents"):
print(f"Document: {results['documents'][0][i]}")
ifresults.get("metadatas"):
print(f"Metadata: {results['metadatas'][0][i]}")
# Query by texts - embeddings auto-generated by embedding_function# Requires: collection must have embedding_function setresults=collection.query(
query_texts=["my query text"],
n_results=10
)
# The collection's embedding_function will automatically convert query_texts to query_embeddings# Query by multiple texts (batch query)results=collection.query(
query_texts=["query text 1", "query text 2"],
n_results=5
)
# Returns dict with lists of lists, one list per query textforiinrange(len(results["ids"])):
print(f"Query {i}: {len(results['ids'][i])} results")
# Query with metadata filter (using query_texts)results=collection.query(
query_texts=["AI research"],
where={"category": {"$eq": "AI"}},
n_results=5
)
# Query with comparison operator (using query_texts)results=collection.query(
query_texts=["machine learning"],
where={"score": {"$gte": 90}},
n_results=5
)
# Query with document filter (using query_texts)results=collection.query(
query_texts=["neural networks"],
where_document={"$contains": "machine learning"},
n_results=5
)
# Query with combined filters (using query_texts)results=collection.query(
query_texts=["AI research"],
where={"category": {"$eq": "AI"}, "score": {"$gte": 90}},
where_document={"$contains": "machine"},
n_results=5
)
# Query with multiple embeddings (batch query)results=collection.query(
query_embeddings=[[1.0, 2.0, 3.0], [2.0, 3.0, 4.0]],
n_results=2
)
# Returns dict with lists of lists, one list per query embeddingforiinrange(len(results["ids"])):
print(f"Query {i}: {len(results['ids'][i])} results")
# Query with specific fieldsresults=collection.query(
query_embeddings=[1.0, 2.0, 3.0],
include=["documents", "metadatas", "embeddings"],
n_results=3
)

Parameters:

  • query_embeddings (List[float] or List[List[float]], optional): Single embedding or list of embeddings for batch queries
    • If provided, used directly (embedding_function is ignored)
    • If not provided, must provide query_texts and collection must have embedding_function
  • query_texts (str or List[str], optional): Query text(s) to be embedded
    • If query_embeddings not provided, query_texts will be converted to embeddings using collection's embedding_function
  • n_results (int, required): Number of similar results to return (default: 10)
  • where (dict, optional): Metadata filter conditions (see Filter Operators section)
  • where_document (dict, optional): Document content filter
  • include (List[str], optional): List of fields to include: ["documents", "metadatas", "embeddings"]

Returns: Dict with keys (chromadb-compatible format):

  • ids: List[List[str]] - List of ID lists, one list per query
  • documents: Optional[List[List[str]]] - List of document lists, one list per query (if included)
  • metadatas: Optional[List[List[Dict]]] - List of metadata lists, one list per query (if included)
  • embeddings: Optional[List[List[List[float]]]] - List of embedding lists, one list per query (if included)
  • distances: Optional[List[List[float]]] - List of distance lists, one list per query

Usage:

# Single queryresults=collection.query(query_embeddings=[0.1, 0.2, 0.3], n_results=5)
# results["ids"][0] contains IDs for the query# results["documents"][0] contains documents for the query# results["distances"][0] contains distances for the query# Multiple queriesresults=collection.query(query_embeddings=[[0.1, 0.2], [0.3, 0.4]], n_results=5)
# results["ids"][0] contains IDs for first query# results["ids"][1] contains IDs for second query

Note: The embedding_function used is the one associated with the collection. You cannot override it per-query.

5.2 Get (Retrieve by IDs or Filters)

The get() method retrieves documents from a collection without vector similarity search. It supports filtering by IDs, metadata, and document content.

# Get by single IDresults=collection.get(ids="123")
# Get by multiple IDsresults=collection.get(ids=["1", "2", "3"])
# Get by metadata filter (simplified equality - both forms are supported)results=collection.get(
where={"category": "AI"},
limit=10
)
# Or use explicit $eq operator:# where={"category": {"$eq": "AI"}}# Get by comparison operatorresults=collection.get(
where={"score": {"$gte": 90}},
limit=10
)
# Get by $in operatorresults=collection.get(
where={"tag": {"$in": ["ml", "python"]}},
limit=10
)
# Get by logical operators ($or) - simplified equalityresults=collection.get(
where={
"$or": [
{"category": "AI"},
{"tag": "python"}
]
},
limit=10
)
# Get by document content filterresults=collection.get(
where_document={"$contains": "machine learning"},
limit=10
)
# Get with combined filtersresults=collection.get(
where={"category": {"$eq": "AI"}},
where_document={"$contains": "machine"},
limit=10
)
# Get with paginationresults=collection.get(limit=2, offset=1)
# Get with specific fieldsresults=collection.get(
ids=["1", "2"],
include=["documents", "metadatas", "embeddings"]
)
# Get all data (up to limit)results=collection.get(limit=100)

Parameters:

  • ids (str or List[str], optional): Single ID or list of IDs to retrieve
  • where (dict, optional): Metadata filter conditions (see Filter Operators section)
  • where_document (dict, optional): Document content filter using $contains for full-text search
  • limit (int, optional): Maximum number of results to return
  • offset (int, optional): Number of results to skip for pagination
  • include (List[str], optional): List of fields to include: ["documents", "metadatas", "embeddings"]

Returns: Dict with keys (chromadb-compatible format):

  • ids: List[str] - List of IDs
  • documents: Optional[List[str]] - List of documents (if included)
  • metadatas: Optional[List[Dict]] - List of metadata dictionaries (if included)
  • embeddings: Optional[List[List[float]]] - List of embeddings (if included)

Usage:

# Get by single IDresults=collection.get(ids="123")
# results["ids"] contains ["123"]# results["documents"] contains document for ID "123"# Get by multiple IDsresults=collection.get(ids=["1", "2", "3"])
# results["ids"] contains ["1", "2", "3"]# results["documents"] contains documents for all IDs# Get by filterresults=collection.get(where={"category": {"$eq": "AI"}}, limit=10)
# results["ids"] contains all matching IDs# results["documents"] contains all matching documents

Note: If no parameters provided, returns all data (up to limit).

5.3 Hybrid Search

The hybrid_search() method combines full-text search and vector similarity search with ranking.

Behavior with Embedding Function:

In the knn parameter:

  1. If query_embeddings are provided: embeddings are used directly, embedding_function is NOT called
  2. If query_embeddings are NOT provided but query_texts are provided:
    • If collection has an embedding_function, it will automatically generate query embeddings from texts
    • If collection does NOT have an embedding_function, a ValueError will be raised
  3. If neither query_embeddings nor query_texts are provided in knn: Only full-text search will be performed (if query is provided)
# Hybrid search with both full-text and vector search (using query_texts)results=collection.hybrid_search(
query={
"where_document": {"$contains": "machine learning"},
"where": {"category": {"$eq": "science"}},
"n_results": 10
},
knn={
"query_texts": ["AI research"], # Will be embedded automatically"where": {"year": {"$gte": 2020}},
"n_results": 10
},
rank={"rrf": {}}, # Reciprocal Rank Fusionn_results=5,
include=["documents", "metadatas", "embeddings"]
)
# Hybrid search with query_embeddings (embedding_function not used)results=collection.hybrid_search(
query={
"where_document": {"$contains": "machine learning"},
"n_results": 10
},
knn={
"query_embeddings": [[0.1, 0.2, 0.3], [0.4, 0.5, 0.6]], # Used directly"n_results": 10
},
rank={"rrf": {}},
n_results=5
)
# Hybrid search with multiple query texts (batch)results=collection.hybrid_search(
query={
"where_document": {"$contains": "AI"},
"n_results": 10
},
knn={
"query_texts": ["machine learning", "neural networks"], # Multiple queries"n_results": 10
},
rank={"rrf": {}},
n_results=5
)

Parameters:

  • query (dict, optional): Full-text search configuration with:
    • where_document: Document filter conditions
    • where: Metadata filter conditions
    • n_results: Number of results for full-text search
  • knn (dict, optional): Vector search configuration with:
    • query_texts (str or List[str], optional): Query text(s) to be embedded
      • If query_embeddings not provided, query_texts will be converted to embeddings using collection's embedding_function
    • query_embeddings (List[float] or List[List[float]], optional): Query vector(s)
      • If provided, used directly (embedding_function is ignored)
    • where: Metadata filter conditions (optional)
    • n_results: Number of results for vector search (optional)
  • rank (dict, optional): Ranking configuration (e.g., {"rrf": {"rank_window_size": 60, "rank_constant": 60}})
  • n_results (int): Final number of results to return after ranking (default: 10)
  • include (List[str], optional): Fields to include in results

Returns: Dict with keys (query-compatible format):

  • ids: List[List[str]] - List of ID lists (one list for hybrid search result)
  • documents: Optional[List[List[str]]] - List of document lists (if included)
  • metadatas: Optional[List[List[Dict]]] - List of metadata lists (if included)
  • embeddings: Optional[List[List[List[float]]]] - List of embedding lists (if included)
  • distances: Optional[List[List[float]]] - List of distance lists

Usage:

# Hybrid search returns results in query-compatible formatresults=collection.hybrid_search(
query={"where_document": {"$contains": "machine learning"}},
knn={"query_texts": ["AI research"]},
rank={"rrf": {}},
n_results=5
)
# results["ids"][0] contains IDs for the hybrid search# results["documents"][0] contains documents for the hybrid search# results["distances"][0] contains distances for the hybrid search

Note: The embedding_function used is the one associated with the collection. You cannot override it per-search.

5.4 Filter Operators

Metadata Filters (where parameter)

  • $eq: Equal to (simplified form is also supported)

    # Simplified form (recommended for equality)where={"category": "AI"}
# Explicit $eq operator (also supported)where={"category": {"$eq": "AI"}}

  • $ne: Not equal to

    where={"status": {"$ne": "deleted"}}

  • $gt: Greater than

    where={"score": {"$gt": 90}}

  • $gte: Greater than or equal to

    where={"score": {"$gte": 90}}

  • $lt: Less than

    where={"score": {"$lt": 50}}

  • $lte: Less than or equal to

    where={"score": {"$lte": 50}}

  • $in: Value in array

    where={"tag": {"$in": ["ml", "python", "ai"]}}

  • $nin: Value not in array

    where={"tag": {"$nin": ["deprecated", "old"]}}

  • $or: Logical OR

    # Simplified equality formwhere={
"$or": [
{"category": "AI"},
{"tag": "python"}
]
}
# Or with explicit $eq:# where={"$or": [{"category": {"$eq": "AI"}}, {"tag": {"$eq": "python"}}]}

  • $and: Logical AND

    # Simplified equality formwhere={
"$and": [
{"category": "AI"},
{"score": {"$gte": 90}}
]
}
# Or with explicit $eq:# where={"$and": [{"category": {"$eq": "AI"}}, {"score": {"$gte": 90}}]}

Document Filters (where_document parameter)

  • $contains: Full-text search (contains substring)

    where_document={"$contains": "machine learning"}

  • $regex: Regular expression matching (if supported)

    where_document={"$regex": "pattern.*"}

  • $or: Logical OR for document filters

    where_document={
"$or": [
{"$contains": "machine learning"},
{"$contains": "artificial intelligence"}
]
}

  • $and: Logical AND for document filters

    where_document={
"$and": [
{"$contains": "machine"},
{"$contains": "learning"}
]
}

5.5 Collection Information Methods

# Get item countcount=collection.count()
print(f"Collection has {count} items")
# Get detailed collection informationinfo=collection.describe()
print(f"Name: {info['name']}, Dimension: {info['dimension']}")
# Preview first few items in collection (returns all columns by default)preview=collection.peek(limit=5)
foriinrange(len(preview["ids"])):
print(f"ID: {preview['ids'][i]}, Document: {preview['documents'][i]}")
print(f"Metadata: {preview['metadatas'][i]}, Embedding: {preview['embeddings'][i]}")
# Count collections in databasecollection_count=client.count_collection()
print(f"Database has {collection_count} collections")

Methods:

  • collection.count() - Get the number of items in the collection
  • collection.describe() - Get detailed collection information
  • collection.peek(limit=10) - Quickly preview the first few items in the collection
  • client.count_collection() - Count the number of collections in the current database

6. Embedding Functions

Embedding functions convert text documents into vector embeddings for similarity search. SeekDBClient supports both built-in and custom embedding functions.

6.1 Default Embedding Function

The DefaultEmbeddingFunction uses sentence-transformers and is the default embedding function if none is specified.

frompyseekdbimportDefaultEmbeddingFunction# Use default model (all-MiniLM-L6-v2, 384 dimensions)ef=DefaultEmbeddingFunction()
# Use custom modelef=DefaultEmbeddingFunction(model_name='all-MiniLM-L6-v2')
# Get embedding dimensionprint(f"Dimension: {ef.dimension}") # 384# Generate embeddingsembeddings=ef(["Hello world", "How are you?"])
print(f"Generated {len(embeddings)} embeddings, each with {len(embeddings[0])} dimensions")

Note: The DefaultEmbeddingFunction requires the sentence-transformers package. Install it with:

pip install sentence-transformers

6.2 Creating Custom Embedding Functions

You can create custom embedding functions by implementing the EmbeddingFunction protocol. The function must:

  1. Implement __call__ method that accepts Documents (str or List[str]) and returns Embeddings (List[List[float]])
  2. Optionally implement a dimension property to return the vector dimension

Example 1: Simple Hash-Based Embedding Function

fromtypingimportList, UnionimporthashlibfrompyseekdbimportEmbeddingFunction, Documents, EmbeddingsclassSimpleHashEmbeddingFunction(EmbeddingFunction[Documents]):
""" A simple custom embedding function that uses hash-based vectorization. This creates fixed-dimensional embeddings by hashing the text. """def__init__(self, dimension: int=128):
""" Initialize the hash-based embedding function. Args: dimension: The dimension of the embedding embeddings (default: 128) """self._dimension=dimension@propertydefdimension(self) ->int:
"""Get the dimension of embeddings produced by this function"""returnself._dimensiondef__call__(self, input: Documents) ->Embeddings:
""" Generate embeddings for the given documents using hash-based vectorization. Args: input: Single document (str) or list of documents (List[str]) Returns: List of embedding embeddings (List[List[float]]) """# Handle single string inputifisinstance(input, str):
input= [input]
# Handle empty inputifnotinput:
return []
embeddings= []
fordocininput:
# Create hash-based embeddinghash_obj=hashlib.md5(doc.encode('utf-8'))
hash_hex=hash_obj.hexdigest()
# Convert hash to vectorvector= []
foriinrange(0, min(len(hash_hex), self._dimension*2), 2):
# Convert hex pair to float in range [0, 1]hex_pair=hash_hex[i:i+2]
value=int(hex_pair, 16) /255.0vector.append(value)
# Pad or truncate to exact dimensionwhilelen(vector) <self._dimension:
vector.append(0.0)
vector=vector[:self._dimension]
embeddings.append(vector)
returnembeddings# Use the custom embedding functionef=SimpleHashEmbeddingFunction(dimension=128)
collection=client.create_collection(
name="my_collection",
configuration=HNSWConfiguration(dimension=128, distance='cosine'),
embedding_function=ef
)

Example 2: Sentence-Transformer Custom Embedding Function

fromtypingimportList, UnionfrompyseekdbimportEmbeddingFunction, Documents, EmbeddingsclassSentenceTransformerCustomEmbeddingFunction(EmbeddingFunction[Documents]):
""" A custom embedding function using sentence-transformers with a specific model. """def__init__(self, model_name: str="all-MiniLM-L6-v2", device: str="cpu"):
""" Initialize the sentence-transformer embedding function. Args: model_name: Name of the sentence-transformers model to use device: Device to run the model on ('cpu' or 'cuda') """self.model_name=model_nameself.device=deviceself._model=Noneself._dimension=Nonedef_ensure_model_loaded(self):
"""Lazy load the embedding model"""ifself._modelisNone:
try:
fromsentence_transformersimportSentenceTransformerself._model=SentenceTransformer(self.model_name, device=self.device)
# Get dimension from modeltest_embedding=self._model.encode(["test"], convert_to_numpy=True)
self._dimension=len(test_embedding[0])
exceptImportError:
raiseImportError(
"sentence-transformers is not installed. ""Please install it with: pip install sentence-transformers"
)
@propertydefdimension(self) ->int:
"""Get the dimension of embeddings produced by this function"""self._ensure_model_loaded()
returnself._dimensiondef__call__(self, input: Documents) ->Embeddings:
""" Generate embeddings for the given documents. Args: input: Single document (str) or list of documents (List[str]) Returns: List of embedding embeddings """self._ensure_model_loaded()
# Handle single string inputifisinstance(input, str):
input= [input]
# Handle empty inputifnotinput:
return []
# Generate embeddingsembeddings=self._model.encode(
input,
convert_to_numpy=True,
show_progress_bar=False
)
# Convert numpy arrays to listsreturn [embedding.tolist() forembeddinginembeddings]
# Use the custom embedding functionef=SentenceTransformerCustomEmbeddingFunction(
model_name='all-MiniLM-L6-v2',
device='cpu'
)
collection=client.create_collection(
name="my_collection",
configuration=HNSWConfiguration(dimension=384, distance='cosine'),
embedding_function=ef
)

Example 3: OpenAI Embedding Function

fromtypingimportList, UnionimportosimportopenaifrompyseekdbimportEmbeddingFunction, Documents, EmbeddingsclassOpenAIEmbeddingFunction(EmbeddingFunction[Documents]):
""" A custom embedding function using OpenAI's embedding API. """def__init__(self, model_name: str="text-embedding-ada-002", api_key: str=None):
""" Initialize the OpenAI embedding function. Args: model_name: Name of the OpenAI embedding model api_key: OpenAI API key (if not provided, uses OPENAI_API_KEY env var) """self.model_name=model_nameself.api_key=api_keyoros.environ.get('OPENAI_API_KEY')
ifnotself.api_key:
raiseValueError("OpenAI API key is required")
# Dimension for text-embedding-ada-002 is 1536self._dimension=1536if"ada-002"inmodel_nameelseNone@propertydefdimension(self) ->int:
"""Get the dimension of embeddings produced by this function"""ifself._dimensionisNone:
# Call API to get dimension (or use known values)raiseValueError("Dimension not set for this model")
returnself._dimensiondef__call__(self, input: Documents) ->Embeddings:
""" Generate embeddings using OpenAI API. Args: input: Single document (str) or list of documents (List[str]) Returns: List of embedding embeddings """# Handle single string inputifisinstance(input, str):
input= [input]
# Handle empty inputifnotinput:
return []
# Call OpenAI APIresponse=openai.Embedding.create(
model=self.model_name,
input=input,
api_key=self.api_key
)
# Extract embeddingsembeddings= [item['embedding'] foriteminresponse['data']]
returnembeddings# Use the custom embedding functionef=OpenAIEmbeddingFunction(
model_name='text-embedding-ada-002',
api_key='your-api-key'
)
collection=client.create_collection(
name="my_collection",
configuration=HNSWConfiguration(dimension=1536, distance='cosine'),
embedding_function=ef
)

6.3 Embedding Function Requirements

When creating a custom embedding function, ensure:

  1. Implement __call__ method:

    • Accepts: str or List[str] (single document or list of documents)
    • Returns: List[List[float]] (list of embeddings)
    • Each vector must have the same dimension

  2. Implement dimension property (recommended):

    • Returns: int (the dimension of embeddings produced by this function)
    • This helps validate dimension consistency when creating collections

  3. Handle edge cases:

    • Single string input should be converted to list
    • Empty input should return empty list
    • All embeddings in the output must have the same dimension

6.4 Using Custom Embedding Functions

Once you've created a custom embedding function, use it when creating or getting collections:

# Create collection with custom embedding functionef=MyCustomEmbeddingFunction()
collection=client.create_collection(
name="my_collection",
configuration=HNSWConfiguration(dimension=ef.dimension, distance='cosine'),
embedding_function=ef
)
# Get collection with custom embedding functioncollection=client.get_collection("my_collection", embedding_function=ef)
# Use the collection - documents will be automatically embeddedcollection.add(
ids=["doc1", "doc2"],
documents=["Document 1", "Document 2"], # Embeddings auto-generatedmetadatas=[{"tag": "A"}, {"tag": "B"}]
)
# Query with texts - query embeddings auto-generatedresults=collection.query(
query_texts=["my query"],
n_results=10
)

Testing

# Run all tests
python3 -m pytest pyseekdb/tests/ -v
# Run tests with log output
python3 -m pytest pyseekdb/tests/ -v -s
# Run specific test
python3 -m pytest pyseekdb/tests/test_client_creation.py::TestClientCreation::test_create_server_client -v
# Run specific test file
python3 -m pytest pyseekdb/tests/test_client_creation.py -v

Environment Variables (Optional)

Password Environment Variables

For security purposes, you can set passwords via environment variables instead of passing them directly in code:

  • SEEKDB_PASSWORD: Password for remote server connections (used by Client() and AdminClient() when password parameter is not provided or is empty). Works for both SeekDB Server and OceanBase Server.
# Set password for remote server connections (SeekDB Server or OceanBase Server)export SEEKDB_PASSWORD="your_password"

Test Configuration Environment Variables

test_client_creation.py honors the following overrides:

export SEEKDB_PATH=/data/seekdb
export SEEKDB_DATABASE=demo
export SERVER_HOST=127.0.0.1
export SERVER_PORT=2881 # SeekDB Server portexport SERVER_USER=root
export SERVER_PASSWORD=secret
export OB_HOST=127.0.0.1
export OB_PORT=2881 # OceanBase port (same as SeekDB Server)export OB_TENANT=test # OceanBase tenantexport OB_USER=root
export OB_PASSWORD= # Uses SEEKDB_PASSWORD if not set

Architecture

  • ClientAPI: Collection operations interface
  • AdminAPI: Database operations interface
  • ServerAPI (BaseClient): Implements both interfaces
  • _ClientProxy: Exposes only collection operations
  • _AdminClientProxy: Exposes only database operations
Client() → _ClientProxy → BaseClient (ServerAPI)
AdminClient() → _AdminClientProxy → BaseClient (ServerAPI)

License

This package is licensed under Apache 2.0.

About

No description, website, or topics provided.

Resources

Readme
License
Activity
Custom properties

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages