GuidesCreate embeddingsVideo embeddings

Embeddings for indexed videos

The platform allows you to retrieve embeddings for videos you’ve already uploaded and indexed. The embeddings are generated using video scene detection. Video scene detection enables the segmentation of videos into semantically meaningful parts. It involves identifying boundaries between scenes, defined as a series of frames depicting a continuous action or theme. Each segment is between 2 and 10 seconds.

Prerequisites

Your video must be indexed with the Marengo video understanding model version 2.7 or later. For details on enabling this model for an index, see the Create an index page.

Complete example

1from twelvelabs import TwelveLabs
2
3# 1. Retrieve the embeddings
4video = client.indexes.indexed_assets.retrieve(
5    index_id="<YOUR_INDEX_ID>", indexed_asset_id="<YOUR_INDEXED_ASSET_ID>", embedding_option=["visual", "audio", "transcription"])
6
7# 2. Process the results
8segments = video.embedding.video_embedding.segments
9print(f"\n{'='*80}")
10print(f"EMBEDDINGS SUMMARY: {len(segments)} total embeddings")
11print(f"{'='*80}\n")
12
13for idx, segment in enumerate(segments, 1):
14    print(f"[{idx}/{len(segments)}] {segment.embedding_option.upper()} | {segment.embedding_scope.upper()}")
15    print(
16        f"├─ Time range: {segment.start_offset_sec}s - {segment.end_offset_sec}s")
17    print(f"├─ Dimensions: {len(segment.float_)}")
18    print(f"└─ First 10 values: {segment.float_[:10]}")
19    print()

Code explanation

1

Retrieve the embeddings

Retrieve the embeddings for an indexed video.
Function call: You call the indexes.indexed_assets.retrieve function.
Parameters:

  • index_id: The unique identifier of the index containing your video.

  • indexed_asset_id: The unique identifier of your indexed video.

  • embedding_option: The types of embeddings to retrieve. Valid values are visual, audio, and transcription. You can specify multiple options. This example uses ["visual", "audio", "transcription"].

    See the Embedding options section for details.

Return value: The response contains, among other information, an object named embedding that contains the embedding data for your video. The embedding.video_embedding.segments field is a list of segment objects. Each segment object includes:

  • float_: The embedding vector (a list of floats).
  • embedding_option: The type of embedding (visual, audio, or transcription).
  • embedding_scope: The scope of the embedding (clip).
  • start_offset_sec: The start time of the segment in seconds.
  • end_offset_sec: The end time of the segment in seconds.
2

Process the results

This example iterates through the embeddings in the segments field and prints the embedding type, scope, time range, dimensions, and the first 10 vector values for each segment.