Create embeddings

This quickstart guide provides a simplified introduction to creating embeddings using the TwelveLabs Video Understanding Platform. It includes the following:

  • A working example for each method: embed a query and embed content at scale
  • Minimal implementation details
  • Core parameters for common use cases

For comprehensive guides, see the Create embeddings section.

Key concepts

This section explains the key concepts and terminology used in this guide:

  • Asset: Your uploaded content. Once created, you can reference the same asset across multiple operations without uploading the file again.
  • Embedding: Vector representation of your content.
  • Embedding task: An asynchronous operation for processing your content and creating embeddings. Contains a status and the resulting embeddings when complete.

Workflow

The platform provides two methods to create embeddings. Choose the method that fits your use case:

Use these embeddings for similarity search, content classification, clustering, recommendations, or Retrieval-Augmented Generation (RAG).

Prerequisites

  • To use the platform, you need an API key:

    1

    If you don’t have an account, sign up for a free account.

    2

    Go to the API Keys page.

    3

    If you need to create a new key, select the Create API Key button. Enter a name and set the expiration period. The default is 12 months.

    4

    Select the Copy icon next to your key to copy it to your clipboard.

  • Depending on the programming language you are using, install the TwelveLabs SDK by entering one of the following commands:

    $pip install --upgrade twelvelabs
  • Your media files must meet the following requirements:

    • Embed a query: Up to 10 media sources (images, video, or audio). Each media source can be up to 32 MB, and video and audio can be up to 30 seconds. Your text can be up to 2,000 tokens.
    • Embed content at scale: Public video and audio URLs up to 4 GB, local video and audio files up to 200 MB, and images up to 32 MB. For local files up to 4 GB, see the Upload and processing methods page. For documents, local files up to 200 MB or public URLs up to 512 MB.
    • Model capabilities: See the complete input requirements for Marengo 3.5.

Embed a query

Create a single embedding from your text. To create a combined embedding, add up to 10 image, video, or audio files to your request. The platform processes your request synchronously and returns the embedding in the response.

Starter code

Copy and paste the code below, replacing the placeholders surrounded by <> with your values.

1from twelvelabs import (
2 TwelveLabs,
3 MultiInputRequest,
4 # To combine your text with media, uncomment the next line:
5 # MultiInputMediaSource,
6)
7
8# 1. Initialize the client
9client = TwelveLabs(api_key="<YOUR_API_KEY>")
10
11# 2. Create an embedding for your query
12response = client.embed.v_2.create(
13 input_type="multi_input",
14 model_name="marengo3.5",
15 multi_input=MultiInputRequest(
16 input_text="<YOUR_TEXT>",
17 # To combine your text with media, uncomment the following lines:
18 # media_sources=[
19 # MultiInputMediaSource(
20 # media_type="image", # Or "video" or "audio"
21 # asset_id="<YOUR_ASSET_ID>", # Upload your file first to create a reusable asset
22 # # Or use url="<YOUR_MEDIA_URL>" for a direct link to a raw media file. Video hosting platforms and cloud storage sharing links are not supported
23 # ),
24 # ],
25 ),
26)
27
28# 3. Process the results
29print(f"Number of embeddings: {len(response.data)}")
30for embedding_data in response.data:
31 print(f"Embedding dimensions: {len(embedding_data.embedding)}")
32 print(f"First 10 values: {embedding_data.embedding[:10]}")

Code explanation

1

Import the SDK and initialize the client

Create a client instance to interact with the TwelveLabs Video Understanding Platform.

2

Create an embedding for your query

Create an embedding for your text. To combine your text with an image, video, or audio file, uncomment the media source lines.

3

Process the results

Process and display the embeddings. This example prints the embedding dimensions and first 10 values to the standard output.

Embed content at scale

Create embeddings for your media files: video, audio, images, and documents. The platform processes your files asynchronously, one file per request. Use this method for long files and large collections. This example embeds one video; repeat the request for each file. The request differs for each type of content.

Retention policy

Embeddings created with the asynchronous method are stored for seven days. After this, you must recreate them to obtain the results again.

Starter code

Copy and paste the code below, replacing the placeholders surrounded by <> with your values.

1import time
2from twelvelabs import TwelveLabs, AsyncVideoInputRequest, MediaSource
3
4# 1. Initialize the client
5client = TwelveLabs(api_key="<YOUR_API_KEY>")
6
7# 2. Upload a video
8asset = client.assets.create(
9 method="url",
10 url="<YOUR_VIDEO_URL>" # Use direct links to raw media files. Video hosting platforms and cloud storage sharing links are not supported
11 # Or use method="direct" and file=open("<PATH_TO_VIDEO_FILE>", "rb") to upload a local file up to 200 MB
12)
13print(f"Created asset: id={asset.id}")
14
15# 3. Check the status of the asset
16print("Waiting for asset to be ready...")
17while True:
18 asset = client.assets.retrieve(asset.id)
19 if asset.status == "ready":
20 print("Asset is ready")
21 break
22 if asset.status == "failed":
23 raise RuntimeError(f"Asset processing failed: id={asset.id}")
24 time.sleep(5)
25
26# 4. Create an embedding task
27task = client.embed.v_2.tasks.create(
28 input_type="video",
29 model_name="marengo3.5",
30 video=AsyncVideoInputRequest(
31 media_source=MediaSource(
32 asset_id=asset.id,
33 # url="<YOUR_VIDEO_URL>", # Use direct links to raw media files. Video hosting platforms and cloud storage sharing links are not supported
34 # base_64_string="<BASE_64_ENCODED_DATA>",
35 ),
36 embedding_option=["visual", "audio"],
37 ),
38)
39print(f"Task ID: {task.id}")
40
41# 5. Monitor the status
42while True:
43 task = client.embed.v_2.tasks.retrieve(task_id=task.id)
44 if task.status == "ready":
45 print("Task completed")
46 break
47 elif task.status == "failed":
48 print("Task failed")
49 break
50 else:
51 print("Task still processing...")
52 time.sleep(5)
53
54# 6. Process the results
55print(f"Number of embeddings: {len(task.data)}")
56for embedding_data in task.data:
57 print(f"[{embedding_data.embedding_option}] {embedding_data.start_sec}s - {embedding_data.end_sec}s")
58 print(f"Embedding dimensions: {len(embedding_data.embedding)}")
59 print(f"First 10 values: {embedding_data.embedding[:10]}")

Code explanation

1

Import the SDK and initialize the client

Create a client instance to interact with the TwelveLabs Video Understanding Platform.

2

Upload a video

Upload a video to create an asset.

3

Check the status of the asset

Asset processing is asynchronous. Poll the status of the asset until it is ready before you use it.

4

Create an embedding task

Create an embedding task using the identifier of the asset.

5

Monitor the status

Poll the task until it reaches the ready state.

6

Process the results

Process and display the embeddings. This example prints the time range, embedding dimensions, and first 10 values for each segment to the standard output.