Skip to content
BranchesTags

Repository files navigation

PyPI versionStargazersIssuesWebsite


Logo

VideoDB Python SDK

Video Database for your AI Applications
Explore the docs »

View Demo · Report Bug · Request Feature

VideoDB Python SDK

VideoDB Python SDK provides programmatic access to VideoDB's serverless video infrastructure. Build AI applications that understand and process video as structured data with support for semantic search, scene extraction, transcript generation, and multimodal content generation.

📑 Table of Contents

Installation

pip install videodb

Requirements:

  • Python 3.8 or higher
  • Dependencies: requests>=2.25.1, backoff>=2.2.1, tqdm>=4.66.1

Quick Start

Establishing a Connection

Get your API key from VideoDB Console. Free for first 50 uploads (no credit card required).

importvideodb# Connect using API keyconn=videodb.connect(api_key="YOUR_API_KEY")
# Or set environment variable VIDEO_DB_API_KEY# conn = videodb.connect()

Uploading Media

Upload videos, audio files, or images from various sources:

# Upload video from YouTube URLvideo=conn.upload(url="https://www.youtube.com/watch?v=VIDEO_ID")
# Upload from public URLvideo=conn.upload(url="https://example.com/video.mp4")
# Upload from local filevideo=conn.upload(file_path="./my_video.mp4")
# Upload with metadatavideo=conn.upload(
file_path="./video.mp4",
name="My Video",
description="Video description"
)

The upload() method returns Video, Audio, or Image objects based on the media type.

Updating Video Metadata

# Update video namevideo.update(name="New Video Title")

Viewing and Streaming Videos

# Generate stream URLstream_url=video.generate_stream()
# Play stream using VideoDB playervideodb.play_stream(stream_url)
# Play in browser/notebookvideo.play()

Understanding Videos

VideoDB 0.5 separates the retrieval pipeline into three primitives: Understand → Index → Retrieve. Understanding runs analyzers once and stores reusable, timestamped artifacts; indexing then decides how those artifacts can be retrieved.

understanding=video.understand(
analyzers=[
{"type": "spoken_words", "name": "transcript"},
{"type": "vlm", "name": "scene"},
]
)
understanding.wait_until_complete()
# Inspect analyzer status or outputforanalyzerinunderstanding.list_analyzers():
print(analyzer.name, analyzer.type, analyzer.status)
scene=understanding.get_analyzer("scene")
scene_output=scene.get_output()

Built-in analyzer types include spoken_words, vlm, object_detection, ocr, brand_detection, activity_recognition, and location_detection. An understanding run can be reopened with video.get_understanding(id), listed with video.list_understandings(), or deleted independently of the video.

Creating Indexes

Create one or more retrieval-ready indexes from the stored analyzer artifacts without analyzing the video again:

fromvideodbimportIndexCapabilitytranscript=understanding.get_analyzer("transcript")
scene=understanding.get_analyzer("scene")
transcript_index=video.index(
source=transcript,
name="transcript",
use_for=[IndexCapability.semantic, IndexCapability.query],
)
scene_index=video.index(
source=scene,
name="scene",
use_for=[
IndexCapability.semantic,
IndexCapability.query,
IndexCapability.aggregate,
],
)
scene_index.wait_until_complete()
print(scene_index.status, scene_index.fields, scene_index.field_schema)

use_for declares whether an index supports semantic search, structured queries, and aggregation. The optional fields argument maps artifact fields into semantic, filter, aggregate, and sort groups; when omitted, VideoDB derives sensible groups from the artifact.

You can also index your own timestamped records:

fromvideodbimportFieldGroupchapters=video.index(
name="chapters",
source=[
{"start": 0.0, "end": 12.4, "summary": "Opening city skyline", "kind": "intro"},
{"start": 12.4, "end": 45.0, "summary": "CEO discusses Q4 results", "kind": "presentation"},
],
use_for=[IndexCapability.semantic, IndexCapability.query, IndexCapability.aggregate],
fields={
FieldGroup.semantic: ["summary"],
FieldGroup.filter: ["kind"],
FieldGroup.aggregate: ["kind"],
},
)

Manage and inspect indexes through their manifests:

indexes=video.list_indexes()
same_index=video.get_index(index_id=chapters.index_id)
page=same_index.records(limit=20)
same_index.delete() # Deletes the index, not its video or understanding artifact

Retrieving Indexed Content

Use high-level search() when VideoDB should plan across the available indexes, or choose a direct retrieval primitive when your application knows the operation:

# Natural-language retrieval; VideoDB selects and combines indexesresponse=video.search(
query="someone discussing a product while holding a phone",
top_k=10,
)
forshotinresponse:
print(shot.start, shot.end, shot.generate_stream())
# Direct vector retrieval over selected semantic indexesresults=video.semantic_search(
query="a presentation about financial results",
index_names=["scene", "transcript"],
top_k=10,
)
# Exact structured filtering over one indexresults=video.query(
index_name="chapters",
filter=[{"field": "kind", "op": "==", "value": "presentation"}],
limit=20,
)
# Counts and facets over one indexcounts=video.aggregate(
index_name="chapters",
group_by="kind",
metric="count",
)
# A grounded answer with optional timestamped sourcesanswer=video.ask(
question="What financial results were discussed?",
include_sources=True,
)

The same search(), semantic_search(), query(), aggregate(), and ask() methods are available on collections for retrieval across multiple videos. Existing applications can continue using index_spoken_words(), index_scenes(), and legacy_search() for legacy indexes.

Working with Transcripts

# Generate transcriptvideo.generate_transcript()
# Generate transcript with language hintvideo.generate_transcript(language_code="en")
# Get transcript with timestampstranscript=video.get_transcript()
# Get plain text transcripttext=video.get_transcript_text()
# Get transcript for specific time rangetranscript=video.get_transcript(start=10, end=60)
# Translate transcripttranslated=video.translate_transcript(
language="Spanish",
additional_notes="Formal tone"
)

Segmentation Options:

  • videodb.Segmenter.word - Word-level timestamps
  • videodb.Segmenter.sentence - Sentence-level timestamps
  • videodb.Segmenter.time - Time-based segments

Legacy Scene Extraction and Indexing

Extract and analyze scenes with the legacy indexing API. New applications should prefer video.understand(...) followed by video.index(...) as shown above:

fromvideodbimportSceneExtractionType# Extract scenes using shot detectionscene_collection=video.extract_scenes(
extraction_type=SceneExtractionType.shot_based,
extraction_config={"threshold": 20, "frame_count": 1}
)
# Extract scenes at time intervalsscene_collection=video.extract_scenes(
extraction_type=SceneExtractionType.time_based,
extraction_config={
"time": 10,
"frame_count": 1,
"select_frames": ["first"]
}
)
# Describe individual scenes with custom model configscenes=video.get_scene_index(scene_collection.scene_index_id)
scene=scenes[0]
scene.describe(
prompt="Describe this scene",
model_config={"model_name": "pro", "temperature": 0.5}
)
# Index scenes for semantic searchscene_index_id=video.index_scenes(
extraction_type=SceneExtractionType.shot_based,
prompt="Describe the visual content of this scene"
)
# Search within scenesresults=video.search(
query="outdoor landscape",
search_type=SearchType.scene,
index_type=IndexType.scene
)
# List scene indexesscene_indexes=video.list_scene_index()
# Get specific scene indexscenes=video.get_scene_index(scene_index_id)
# Delete scene collectionvideo.delete_scene_collection(scene_collection.id)

Adding Subtitles

fromvideodbimportSubtitleStyle# Add subtitles with default stylestream_url=video.add_subtitle()
# Customize subtitle appearancestyle=SubtitleStyle(
font_name="Arial",
font_size=24,
primary_colour="&H00FFFFFF",
bold=True
)
stream_url=video.add_subtitle(style=style)

Generating Thumbnails

# Get default thumbnailthumbnail_url=video.generate_thumbnail()
# Generate thumbnail at specific timestampthumbnail_image=video.generate_thumbnail(time=30.5)
# Get all thumbnailsthumbnails=video.get_thumbnails()

Working with Collections

Organize and search across multiple videos:

# Get default collectioncoll=conn.get_collection()
# Create new collectioncoll=conn.create_collection(
name="My Collection",
description="Collection description",
is_public=False
)
# List all collectionscollections=conn.get_collections()
# Update collectioncoll=conn.update_collection(
id="collection_id",
name="Updated Name",
description="Updated description"
)
# Upload to collectionvideo=coll.upload(url="https://example.com/video.mp4")
# Get videos in collectionvideos=coll.get_videos()
video=coll.get_video(video_id)
# Search across collectionresults=coll.search(query="specific content")
# Search by titleresults=coll.search_title("video title")
# Make collection public/privatecoll.make_public()
coll.make_private()
# Delete collectioncoll.delete()

Audio and Image Management

# Get audio filesaudios=coll.get_audios()
audio=coll.get_audio(audio_id)
# Generate audio URLaudio_url=audio.generate_url()
# Get imagesimages=coll.get_images()
image=coll.get_image(image_id)
# Generate image URLimage_url=image.generate_url()
# Delete mediaaudio.delete()
image.delete()

Advanced Features

Realtime Video Editor

Build multi-track video compositions programmatically using VideoDB's 4-layer architecture: Assets (raw media), Clips (how assets appear), Tracks (timeline lanes), and Timeline (final canvas).

Example: Video with background music

fromvideodbimportconnectfromvideodb.editorimportTimeline, Track, Clip, VideoAsset, AudioAssetconn=connect(api_key="YOUR_API_KEY")
video=conn.upload(url="https://www.youtube.com/watch?v=VIDEO_ID")
audio=conn.upload(file_path="./music.mp3")
# Create timelinetimeline=Timeline(conn)
# Video trackvideo_track=Track()
video_asset=VideoAsset(id=video.id, start=10)
video_clip=Clip(asset=video_asset, duration=30)
video_track.add_clip(0, video_clip)
# Audio trackaudio_track=Track()
audio_asset=AudioAsset(id=audio.id, start=0, volume=0.3)
audio_clip=Clip(asset=audio_asset, duration=30)
audio_track.add_clip(0, audio_clip)
# Compose and rendertimeline.add_track(video_track)
timeline.add_track(audio_track)
stream_url=timeline.generate_stream()

Asset Types:

  • VideoAsset - Video clips with trim control (start, volume)
  • AudioAsset - Background music, voiceovers, sound effects
  • ImageAsset - Logos, watermarks, static overlays
  • TextAsset - Custom text with typography (Font, Background, Alignment)
  • CaptionAsset - Auto-generated subtitles synced to speech

Clip Controls:

  • Position & Scale: position=Position.topRight, scale=0.5, offset=Offset(x=0.1, y=-0.2)
  • Visual Effects: opacity=0.8, fit=Fit.cover, filter=Filter.greyscale
  • Transitions: transition=Transition(in_="fade", out="fade", duration=1)

Track Layering:

  • Clips on the same track play sequentially
  • Clips on different tracks at the same time play simultaneously (overlays)

For advanced patterns (picture-in-picture, multi-audio layers, auto-captions), see the Editor SDK documentation.

Real-Time Streams (RTStream)

Process live video streams in real-time:

fromvideodbimportSceneExtractionType# Connect to real-time streamrtstream=coll.connect_rtstream(
url="rtsp://example.com/stream",
name="Live Stream"
)
# Start or Stop processingrtstream.stop()
rtstream.start()
# Index scenes from streamscene_index=rtstream.index_scenes(
extraction_type=SceneExtractionType.time_based,
extraction_config={"time": 2, "frame_count": 5},
prompt="Describe the scene"
)
# Start or Stop scene indexingscene_index.stop()
scene_index.start()
# Get scenesscenes=scene_index.get_scenes(page=1, page_size=100)
# Create alerts for eventsalert_id=scene_index.create_alert(
event_id=event_id,
callback_url="https://example.com/callback"
)
# Enable/disable alertsscene_index.disable_alert(alert_id)
scene_index.enable_alert(alert_id)
# Generate stream with player metadatastream_url=rtstream.generate_stream(
start=1711000000,
end=1711003600,
player_config={
"title": "Live Feed",
"description": "Stream recording",
"slug": "live-feed"
}
)
# Export a stopped stream as a video/audio assetrtstream.stop()
export_result=rtstream.export(name="my_recording")
# List streamsstreams=coll.list_rtstreams()

Capture Sessions (Desktop Recording)

Record screen, microphone, and system audio from desktop applications using native capture binaries:

# Install capture dependencies
pip install 'videodb[capture]'
fromvideodb.captureimportCaptureClient# Backend: Create a capture sessioncap=coll.create_capture_session(
end_user_id="user_abc",
callback_url="https://example.com/webhook"
)
# Generate a client token for secure desktop authtoken=conn.generate_client_token(expires_in=86400)
# Desktop client: Start captureclient=CaptureClient(session_token=token)
# Request permissionsawaitclient.request_permission("microphone")
awaitclient.request_permission("screen")
# Configure channels and start recordingawaitclient.start_capture_session(
session_id=cap.id,
channels=[
{"type": "mic", "name": "mic:default"},
{"type": "system_audio", "name": "system_audio:default"},
{"type": "display", "name": "display:1"},
]
)
# Stop captureawaitclient.stop_capture_session()
# Get session details and exportcap=coll.get_capture_session(cap.id)
export_result=cap.export()
# List all capture sessionssessions=coll.list_capture_sessions()

WebSocket Events

Receive real-time transcript and indexing events via WebSocket:

# Connect to WebSocketws=conn.connect_websocket()
awaitws.connect()
print(f"Connection ID: {ws.connection_id}")
# Stream eventsasyncforeventinws.receive():
print(event)
# Close connectionawaitws.close()

Meeting Recording

Record and process virtual meetings:

# Start meeting recordingmeeting=conn.record_meeting(
meeting_url="https://meet.google.com/xxx-yyyy-zzz",
bot_name="Recorder Bot",
meeting_title="Team Meeting",
callback_url="https://example.com/callback"
)
# Check meeting statusmeeting.refresh()
print(meeting.status) # initializing, processing, or done# Wait for completionmeeting.wait_for_status("done", timeout=14400, interval=120)
# Get meeting detailsifmeeting.is_completed:
video_id=meeting.video_idvideo=coll.get_video(video_id)
# Get meeting from videomeeting_info=video.get_meeting()

Sandbox Compute

Create dedicated compute for supported open-weight models:

fromvideodbimportSandboxTiersandbox=conn.create_sandbox(
tier=SandboxTier.small,
name="my-sandbox",
models=["rtdetr-v2-r50vd"],
)
sandbox.wait_for_ready(timeout=1200, interval=5)
# Retrieve or list existing sandboxes.same_sandbox=conn.get_sandbox(sandbox.id)
active_sandboxes=conn.list_sandboxes(status="active")
# Keep the sandbox active while submitting inference work, then stop it.sandbox.stop(grace=True)
sandbox.wait_for_stop(timeout=300, interval=5)

Generative Media

Generate images, audio, and videos using AI:

# Generate imageimage=coll.generate_image(
prompt="A beautiful sunset over mountains",
aspect_ratio="16:9"
)
# Generate musicaudio=coll.generate_music(
prompt="Upbeat electronic music",
duration=30
)
# Generate sound effectsaudio=coll.generate_sound_effect(
prompt="Door closing sound",
duration=2
)
# Generate voice from textaudio=coll.generate_voice(
text="Hello, welcome to VideoDB",
voice_name="Default"
)
# Generate videovideo=coll.generate_video(
prompt="A cat playing with a ball",
duration=5
)
# Generate text using LLMresponse=coll.generate_text(
prompt="Summarize this content",
model_name="pro", # basic, pro, or ultraresponse_type="text"# text or json
)
# Large prompts are uploaded automatically with a unique filename and# sent as prompt_url instead of inline JSON to avoid request payload limits.

Video Dubbing and Translation

# Dub video to another languagedubbed_video=coll.dub_video(
video_id=video.id,
language_code="es",
callback_url="https://example.com/callback"
)

Transcoding

fromvideodbimportTranscodeMode, VideoConfig, AudioConfig# Start transcoding jobjob_id=conn.transcode(
source="https://example.com/video.mp4",
callback_url="https://example.com/callback",
mode=TranscodeMode.economy,
video_config=VideoConfig(resolution=1080, quality=23),
audio_config=AudioConfig(mute=False)
)
# Check transcode statusstatus=conn.get_transcode_details(job_id)

YouTube Integration

# Search YouTuberesults=conn.youtube_search(
query="machine learning tutorial",
result_threshold=10,
duration="medium"
)
forresultinresults:
print(result["title"], result["url"])

Billing and Usage

# Check usageusage=conn.check_usage()
# Get invoicesinvoices=conn.get_invoices()

Download Streams

# Download compiled streamdownload_info=conn.download(
stream_link="https://stream.videodb.io/...",
name="my_compilation"
)

Configuration Options

Subtitle Customization

fromvideodbimportSubtitleStyle, SubtitleAlignment, SubtitleBorderStylestyle=SubtitleStyle(
font_name="Arial",
font_size=18,
primary_colour="&H00FFFFFF", # Whitesecondary_colour="&H000000FF", # Blueoutline_colour="&H00000000", # Blackback_colour="&H00000000", # Blackbold=False,
italic=False,
underline=False,
strike_out=False,
scale_x=1.0,
scale_y=1.0,
spacing=0,
angle=0,
border_style=SubtitleBorderStyle.outline,
outline=1.0,
shadow=0.0,
alignment=SubtitleAlignment.bottom_center,
margin_l=10,
margin_r=10,
margin_v=10
)

Text Overlay Styling

fromvideodbimportTextStylestyle=TextStyle(
fontsize=24,
fontcolor="black",
font="Sans",
box=True,
boxcolor="white",
boxborderw="10"
)

Error Handling

fromvideodb.exceptionsimport (
VideodbError,
AuthenticationError,
InvalidRequestError,
SearchError
)
try:
conn=videodb.connect(api_key="invalid_key")
exceptAuthenticationErrorase:
print(f"Authentication failed: {e}")
try:
video=conn.upload(url="invalid_url")
exceptInvalidRequestErrorase:
print(f"Invalid request: {e}")
try:
results=video.search("query")
exceptSearchErrorase:
print(f"Search error: {e}")

API Reference

Core Objects

  • Connection: Main client for API interaction
  • Collection: Container for organizing media
  • Video: Video file with processing methods
  • Audio: Audio file representation
  • Image: Image file representation
  • Timeline: Multi-track video editor
  • SearchResult: Search results with shots
  • Shot: Time-segmented video clip
  • Understanding: A reusable video analysis run containing analyzer artifacts
  • UnderstandingAnalyzer: Status, output, and index-source handle for one analyzer
  • Index: Retrieval-ready index manifest with status, capabilities, and schema
  • IndexRecord: One timestamped record stored in an index
  • Scene: Visual scene with frames
  • SceneCollection: Collection of extracted scenes
  • Meeting: Meeting recording session
  • RTStream: Real-time stream processor
  • CaptureSession: Desktop capture session with export
  • CaptureClient: Native binary client for screen/audio recording
  • WebSocketConnection: Real-time event streaming
  • Sandbox: Dedicated compute for supported open-weight models
  • GenerationJob: Asynchronous image or audio generation job
  • VoiceClone: Reusable cloned-voice reference

Constants and Enums

  • IndexCapability: semantic, query, aggregate
  • FieldGroup: semantic, filter, aggregate, sort
  • IndexType: spoken_word, scene (legacy)
  • SearchType: semantic, keyword (legacy)
  • SceneExtractionType: shot_based, time_based (legacy)
  • Segmenter: word, sentence, time
  • TranscodeMode: lightning, economy
  • MediaType: video, audio, image
  • SandboxTier: small, medium
  • SandboxStatus: provisioning, active, alert, stopping, stopped, failed

For detailed API documentation, visit docs.videodb.io.

Examples and Tutorials

Explore practical examples and use cases in the VideoDB Cookbook:

  • Semantic video search
  • Scene-based indexing and retrieval
  • Custom video compilations
  • Meeting transcription and analysis
  • Real-time stream processing
  • Multi-language video dubbing

Contributing

Contributions are welcome! To contribute:

  1. Fork the repository
  2. Create your feature branch (git checkout -b feature/AmazingFeature)
  3. Commit your changes (git commit -m 'Add AmazingFeature')
  4. Push to the branch (git push origin feature/AmazingFeature)
  5. Open a Pull Request

Resources

License

Apache License 2.0 - see LICENSE file for details.

About

VideoDB Python SDK

pypi.org/project/videodb

Topics

agentaicopilotdatabasegptinfrastructurellmmultimediamultimodalpythonragsdkvideovideo-processing

Resources

Readme
Apache-2.0 license
Activity
Custom properties

Stars

97 stars

Watchers

2 watching

Forks

16 forks
Report repository

Releases

Packages

Used by

Contributors

Languages