Vespa - Multivector video retrieval
Vespa - Multivector video retrieval
Vespa - Multivector video retrieval

Summary: This integration combines TwelveLabs’ Analyze and Embed APIs with Vespa to create an efficient solution for semantic video search. It captures rich video content as multimodal embeddings and utilizes Vespa’s robust indexing and hybrid ranking capabilities to deliver precise and relevant search results.
Description: The process of performing a semantic video search using TwelveLabs and Vespa involves three main steps:
Code explanation: Our blog post, Multivector Video Retrieval with TwelveLabs and Vespa, guides you through the process of building a semantic video search solution.
Colab Notebook: video_search_twelvelabs_cloud
This section shows how to use the Generate and Embed APIs for creating video embeddings and metadata, which facilitate the efficient retrieval of relevant video segments.
The code below uploads videos to an index and monitors the processing status:
Once the videos are processed, you can generate rich metadata using the /summarize and /analyze endpoints. This code creates summaries and lists of keywords for each video to enhance search capabilities:
The code below creates multimodal embeddings for each video. These embeddings capture the temporal and contextual nuances of the video content:
See the Create video embeddings section for details.
The code below generates an embedding for your text query:
See the Create text embeddings section for details.
The code below uses Vespa’s approximate nearest neighbor (ANN) search capabilities to combine lexical search (BM25) with vector similarity ranking. The query retrieves the top hit based on hybrid ranking:
After reading this page, you have the following options: