MongoDB - Semantic video search

Summary: This integration combines TwelveLabs’ Embed API with MongoDB Atlas Vector Search to create an efficient semantic video search solution. It captures rich video content as multimodal embeddings, enabling precise and relevant search results.

Description: The process of performing a semantic video search using TwelveLabs and MongoDB Atlas involves two main steps:

  1. Create embeddings for your video content and query.
  2. Use the embeddings to perform a vector search in MongoDB Atlas.

Step-by-step guide: Our blog post, Building Semantic Video Search with TwelveLabs Embed API and MongoDB Atlas, guides you through the process of creating a video search application, from setup to performing vector searches.

Colab Notebook: TwelveLabs-EmbedAPI-MongoDB-Atlas

Integration with TwelveLabs

This section describes how you can use the TwelveLabs Python SDK to create embeddings for semantic video search. The integration involves creating two types of embeddings:

  • Video embeddings from your video content
  • Text embeddings from queries.

These embeddings form the basis for vector search operations.

Video embeddings

The code below creates a video embedding task that handles the uploading and processing of a video. It periodically checks the status of the task and retrieves the embeddings upon completion:

Python
1# Create a video embedding task for the uploaded video
2task = tl_client.embed.task.create(
3    engine_name="Marengo-retrieval-2.6",
4    video_url="your-video-url"
5)
6print(
7    f"Created task: id={task.id} engine_name={task.engine_name} status={task.status}"
8)
9
10# Monitor the status of the video embedding task
11def on_task_update(task: EmbeddingsTask):
12    print(f"  Status={task.status}")
13
14status = task.wait_for_done(
15    sleep_interval=2,
16    callback=on_task_update
17)
18print(f"Embedding done: {status}")
19
20# Retrieve the video embeddings
21task_result = tl_client.embed.task.retrieve(task.id)

For more details, see the Create video embeddings page.

Text embeddings

The code below creates a text embedding for the query provided in the text parameter:

Python
1# Create a text embedding task for the text
2embedding = tl_client.embed.create(
3  engine_name="Marengo-retrieval-2.6",
4  text="your-text"
5)
6
7print("Created a text embedding")
8print(f" Engine: {embedding.engine_name}")
9print(f" Embedding: {embedding.text_embedding.float}")

For more details, see the Create text embeddings page.

Next steps

After reading this page, you have the following options:

  • Customize and use the example: Use the TwelveLabs-EmbedAPI-MongoDB-Atlas notebook to understand how the integration works. You can make changes and add more functionalities to suit your specific use case.
  • Explore further: Try the applications built by the community or our sample applications to get more insights into the TwelveLabs Video Understanding Platform’s diverse capabilities and learn more about integrating the platform into your applications.
Built with