VideoDB Node.js SDK

Video Database for your AI Applications
Explore the docs »

Report Bug · Request Feature

VideoDB Node.js SDK

VideoDB Node.js SDK allows you to interact with the VideoDB serverless database. Manage videos as intelligent data, not files. It's scalable, cost efficient & optimized for AI applications and LLM integration.

Table of Contents

• About The Project
• Installation
• Quick Start
  • Creating a Connection
  • Getting a Connection
• Working with a single video
  • ⬆️ Upload Video
  • 📺 View your Video
  • ⛓️ Stream Sections of videos
  • 🗂️ Indexing a Video
  • 🔍 Searching inside a video
  • Viewing Search Results
• RAG: Search inside Multiple Videos
  • 🔄 Using Collection to Upload Multiple Videos
  • 📂 Search inside multiple videos in a collection
• Timeline And Assets
  • Understanding Assets
  • Creating Assets
    • VideoAsset
    • AudioAsset
    • ImageAsset
    • TextAsset
  • Understanding Timeline
  • Creating Timeline
• More on Video object
  • Get the video's transcript
  • Get the video's thumbnail
  • Overlay Subtitle on video
  • Delete the video
• More on Collection object
  • Get all videos
  • Get a video given videoId
  • Delete a video
• Roadmap
• Contributing
• License

Installation

To install the package, run the following command in your terminal:

npm install videodb

Quick Start

Creating a Connection

Get an API key from the VideoDB console. Free for first 50 uploads (No credit card required).

import{connect}from'videodb';// create a connection to the VideoDB APIconstconn=connect('YOUR_API_KEY');

Getting a Collection

A default collection is created when you create your first connection. Use the getCollection method on the established database connection to get the Collection object.

// Get Default collectionconn.getCollection().then(coll=>console.log('Collection Id : ',coll.meta.id));

Or using async/await

// Get Default Collection(async()=>{constcoll=awaitconn.getCollection();console.log('Collection Id : ',coll.meta.id);})();

Working with a single video

Upload Video

Now that you have established a connection to VideoDB, you can upload your videos using coll.uploadURL() or coll.uploadFile().
You can directly upload files from youtube, any public url, S3 bucket or a local file path. A default collection is created when you create your first connection.

// upload to the default collection using url which returns an upload jobconstuploadJob=awaitcoll.uploadURL({url: 'https://www.youtube.com/watch?v=WDv4AWk0J3U',});// Attach optional event listenersuploadJob.on('success',uploadedVideo=>console.log('Video Uploaded. VideoID : ',uploadedVideo.meta.id));uploadJob.on('error',err=>{console.error(err);});// Call the start function to commence the uploaduploadJob.start();

View your Video

Once uploaded, your video is immediately available for viewing in 720p resolution. ⚡️

Generate a streamable url for the video using video.generateStream()
Get a browser playable link using video.play()

// Replace {VIDEO_ID} with your video idconstvideo=awaitcoll.getVideo('VIDEO_ID');// Generate a preview stream for videoconstplayerUrl=awaitvideo.play();console.log('Video Preview : ',playerUrl);

Stream Sections of videos

You can easily clip specific sections of a video by passing a timeline of the start and end timestamps (in seconds) as a parameter. For example, this will generate a streaming URL for a compilation of the fist 10 seconds, and the part between the 120th and the 140th second.

import{playStream}from'videodb';conststreamLink=awaitvideo.generateStream([[0,10],[120,140],]);conststreamPreview=playStream(streamLink);console.log('Clipped Video Preview : ',streamPreview);

Indexing a Video

To search bits inside a video, you have to first index the video. This can be done by a invoking the index function on the Video. VideoDB offers two type of indexes currently.

1. indexSpokenWords: Indexes spoken words in the video. It automatically generate the transcript and makes it ready for search.
2. indexScenes: Indexes visual concepts and events of the video.

(Note: This feature is currently available only for beta users, join our discord for early access)

// best for podcasts, elearning, news, etc.constjob1=video.indexSpokenWords();job1.start();// best for camera feeds, moderation usecases etc.constjob2=video.indexScenes();job2.start();

In future you can also index videos using:

1. Faces : Upload image of the person and find them in a video.
2. Specific domain Index like Football, Baseball, Drone footage, Cricket etc.

⏱️ Indexing may take some time for longer videos, structure it as a batch job in your application.

Searching inside a video

Search the segments inside a video. While searching you have options to choose the type of search. VideoDB offers following type of search :

• semantic: Perfect for question answer kind of queries. This is also the default type of search.

• keyword: It matches the exact occurance of word or sentence you pass in the query parameter of the search function. keyword search is only available to use with single videos.

• scene : It search the visual information of the video, Always Index the videousing index_scenes function before using this search.

constindexJob=video.indexSpokenWords();indexJob.on('success',async()=>{constresults=awaitvideo.search('Morning Sunlight','semantic');constresultsUrl=awaitresults.play();console.log('Search results preview : ',resultsUrl);});indexJob.start();

Similarly, you can index and search from scenes using Video.indexScenes()

Viewing Search Results

video.search() will return a SearchResult object, which contains the sections or as we call them, shots of videos which semantically match your search query.

• result.shots Returns a list of Shot(s) that matched the search query. You can call generateStream() on each shot to get the corresponding streaming URL.
• result.play() Compiles and returns a playable url for the compiled shots (similar to video.play()). You can open this link in the browser, or embed it into your website using an iframe.

RAG: Search inside Multiple Videos

VideoDB can store and search inside multiple videos with ease. By default, videos are uploaded to your default collection.

Using Collection to Upload Multiple Videos

constuploadJobHandler=video=>{console.log(`Video uploaded :${video.meta.name}`);};// Upload Video1 to VideoDBconstjob1=awaitcoll.uploadURL({url: 'https://www.youtube.com/watch?v=lsODSDmY4CY',});job1.on('success',uploadJobHandler);job1.start();// Upload Video2 to VideoDBconstjob2=awaitcoll.uploadURL({url: 'https://www.youtube.com/watch?v=vZ4kOr38JhY',});job2.on('success',uploadJobHandler);job2.start();// Upload Video3 to VideoDBconstjob3=awaitcoll.uploadURL({url: 'https://www.youtube.com/watch?v=uak_dXHh6s4',});job3.on('success',uploadJobHandler);job3.start();

• Connection.getCollection() : Returns Collection object, the default collection
• Collection.getVideo() : Returns list of Video, all videos in collections
• Collection.getVideo(videoId): Returns Video, respective video object from given videoId
• Collection.deleteVideo(videoId): Deletes the video from Collection

Search inside multiple videos in a collection

You can simply Index all the videos in a collection and use the search method to find relevant results. Here we are indexing the spoken content of a collection and performing semantic search.

constindexJobHandler=res=>{console.log(`Video Indexed : `,res);};constvideos=awaitcoll.getVideos();console.log('Total videos',videos.length);for(letvideoofvideos){constindexJob=awaitvideo.indexSpokenWords();indexJob.on('success',indexJobHandler);indexJob.start();}

Semantic Search in the collection

constsearchRes=awaitcoll.search('What is dopamine');constresultsUrl=awaitsearchRes.play();console.log('Search Result Preview : ',resultsUrl);

The result here has all the matching bits in a single stream from your collection. You can use these results in your application right away.

As you can see VideoDB fundamentally removes the limitation of files and gives you power to access and stream videos in a very seamless way. Stay tuned for exciting features in our upcoming version and keep building awesome stuff with VideoDB 🤘

Timeline And Assets

Timeline and Assets lets you create programmatic compilation streams with audio, image and text overlays using your video data in VideoDB.

Understanding Assets

Assets are objects that you can use in your video timeline. These Assets have lots of configurable setting available.

Creating Assets

To define any asset, you must provide the identifier of the media and specify the segment of the media with start and end parameters, some assets have lots of other configurable settings available.

VideoAsset

A Video Asset can be created by calling VideoAsset()

import{VideoAsset}from'videodb';constvideoAsset=newVideoAsset('MEDIA_ID',{start: 0,end: 20});

AudioAsset

An Audio Asset can be created by calling AudioAsset()

import{AudioAsset}from'videodb';constaudioAsset=newAudioAsset('MEDIA_ID',{start: 0,end: 10});

ImageAsset

An Image Asset can be created by calling ImageAsset()

import{ImageAsset}from'videodb';constimageAsset=newImageAsset('MEDIA_ID',{x: 10,y: 10,duration: 5});

TextAsset

A Text Asset can be created by calling TextAsset()

import{TextAsset}from'videodb';// Defult StyleconsttextAsset1=newTextAsset({text: 'Hello World!'});// Configured StyleconsttextAsset2=newTextAsset({text: 'Hello World!',style: {fontsize: 16,alpha: 0.8},});

Understanding Timeline

Timeline let's you add your organise your asset in a Video Timeline and generate stream for your timeline.

Creating Timeline

Timeline can be created by calling Timeline() constructor

The Timeline object provides you with following methods:

• addInline() : adds VideoAsset inline
• addOverlay(): adds AudioAsset, ImageAsset and TextAsset as overlay
• generateStream(): generates stream for your timeline

import{Timeline,playStream}from'videodb';// Create a Timeline consttimeline=Timeline(conn);// Add VideoAsset inlinetimeline.addInline(videoAsset);// Add AudioAsset overlaytimeline.addOverlay(0,audioAsset);// Generate and play streamconststreamUrl=timeline.generateStream();console.log(playStream(streamUrl));

🌟 More on Video object

There are multiple useful functions available on a Video Object:

Get the video's transcript

consttranscriptJob=video.getTranscript();transcriptJob.on('success',transcript=>{console.log(transcript);});transcriptJob.start();

Get the video's thumbnail

// Get thumbnail of the videoconstthumbnail=awaitvideo.generateThumbnail();console.log(thumbnail);

Overlay Subtitle on video

letsubtitleStream=awaitvideo.addSubtitle();letplayerUrl=playStream(subtitleStream);console.log(playerUrl);

Subtitles Can be styled by passing a Object of Type Partial<SubtitleStyleProps> in Video.addSubtitle()

subtitleStream=awaitvideo.addSubtitle({fontSize: 12});playerUrl=awaitplayStream(subtitleStream);console.log(playerUrl);

Delete the video

// Delete the video from the collectionawaitvideo.delete();

🌟 More on Collection object

Get all videos

constallVideos=coll.getVideo();

Get a video given videoId

constmyVideo=coll.getVideo(id);

Delete a video

awaitcoll.deleteVideo();

---

Roadmap

• Adding More Indexes : Face, Scene, Security, Events, and Sports
• Give prompt support to generate thumbnails using GenAI.
• Give prompt support to access content.
• Give prompt support to edit videos.
• See the open issues for a list of proposed features (and known issues).

---

Contributing

Contributions are what make the open source community such an amazing place to be learn, inspire, and create. Any contributions you make are greatly appreciated.

1. Fork the Project
2. Create your Feature Branch (git checkout -b feature/AmazingFeature)
3. Commit your Changes (git commit -m 'Add some AmazingFeature')
4. Push to the Branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

---

License