The platform allows you to retrieve embeddings for videos you’ve already uploaded and indexed. The embeddings are generated using video scene detection. Video scene detection enables the segmentation of videos into semantically meaningful parts. It involves identifying boundaries between scenes, defined as a series of frames depicting a continuous action or theme. Each segment is between 2 and 10 seconds.

Prerequisites

Your video must be indexed with the Marengo video understanding model version 2.7 or later. For details on enabling this model for an index, see the Create an index page.

Complete example

1 from twelvelabs import TwelveLabs
2 from typing import List
3 from twelvelabs.models.embed import SegmentEmbedding
4 
5 #1. Retrieve the embeddings
6 video = client.index.video.retrieve(
7     index_id="<YOUR_INDEX_ID>", id="<YOUR_VIDEO_ID>", embedding_option=["visual-text", "audio"])
8 
9 #2. Process the results
10 def print_segments(segments: List[SegmentEmbedding], max_elements: int = 5):
11     for segment in segments:
12         print(
13             f"  embedding_scope={segment.embedding_scope} embedding_option={segment.embedding_option} start_offset_sec={segment.start_offset_sec} end_offset_sec={segment.end_offset_sec}"
14         )
15         print(f"  embeddings: {segment.embeddings_float[:max_elements]}")
16 
17 if video.embedding:
18     print(f"Model_name={video.embedding.model_name}")
19     print("Embeddings:")
20     print_segments(video.embedding.video_embedding.segments)

Step-by-step guide

Python

Node.js

Retrieve the embedding

Function call: You call the index.video.retrieve function.
Parameters:

index_id: The unique identifier of the index containing your video.
id: The unique identifier of your video.
embedding_option: The types of embeddings to retrieve. This example uses ["visual-text", "audio"].
Return value: The response contains, among other information, an object named embedding that contains the embedding data for your video.

Process the results

This example iterates over the results and prints the key properties and a portion of the embedding vectors for each segment.