GuidesCreate embeddingsEmbed content at scale

Document embeddings

This guide shows how you can create document embeddings using the Marengo 3.5 video understanding model. For complete specifications and input requirements, see the Marengo 3.5 page.

The Marengo video understanding model generates embeddings for all modalities in the same latent space. This shared space enables any-to-any searches across different types of content.

For details on how your usage is measured and billed, see the Pricing page.

Key concepts

This section explains the key concepts and terminology used in this guide:

  • Asset: Your uploaded content. Once created, you can reference the same asset across multiple operations without uploading the file again.
  • Embedding: Vector representation of your content.
  • Embedding task: An asynchronous operation for processing your content and creating embeddings. Contains a status and the resulting embeddings when complete.

Workflow

This guide shows how to create embeddings for a document file, so your queries can match its content. The example uploads the PDF file as an asset. You can also pass a URL or base64-encoded data inline instead of creating an asset; both are shown as commented-out lines in the code examples.

The platform processes your files asynchronously, one file per request. This example embeds one document; repeat the request for each file in your collection. To embed a query instead, see the Embed a query guide.

The platform embeds the rendered pages of your PDF file, one embedding per page.

Use these embeddings for similarity search, content classification, clustering, recommendations, or Retrieval-Augmented Generation (RAG).

Retention policy

Embeddings created with the asynchronous method are stored for seven days. After this, you must recreate them to obtain the results again.

Prerequisites

  • To use the platform, you need an API key:

    1

    If you don’t have an account, sign up for a free account.

    2

    Go to the API Keys page.

    3

    If you need to create a new key, select the Create API Key button. Enter a name and set the expiration period. The default is 12 months.

    4

    Select the Copy icon next to your key to copy it to your clipboard.

  • Depending on the programming language you are using, install the TwelveLabs SDK by entering one of the following commands:

    $pip install --upgrade twelvelabs

  • Your document files must meet the following requirements:

    • Upload limits: Local document files up to 200 MB, or public document URLs up to 512 MB.

    • Formats: PDF.

    • Model capabilities: See the complete input requirements for Marengo 3.5.

Complete example

Copy and paste the code below, replacing the placeholders surrounded by <> with your values.

1import time
2from twelvelabs import TwelveLabs, AsyncDocumentInputRequest, MediaSource
3
4# 1. Initialize the client
5client = TwelveLabs(api_key="<YOUR_API_KEY>")
6
7# 2. Upload a document
8asset = client.assets.create(
9    method="url",
10    url="<YOUR_DOCUMENT_URL>" # Use direct links to raw files. Cloud storage sharing links are not supported
11    # Or use method="direct" and file=open("<PATH_TO_DOCUMENT_FILE>", "rb") to upload a local file up to 200 MB
12)
13print(f"Created asset: id={asset.id}")
14
15# 3. Check the status of the asset
16print("Waiting for asset to be ready...")
17while True:
18    asset = client.assets.retrieve(asset.id)
19    if asset.status == "ready":
20        print("Asset is ready")
21        break
22    if asset.status == "failed":
23        raise RuntimeError(f"Asset processing failed: id={asset.id}")
24    time.sleep(5)
25
26# 4. Create an embedding task
27task = client.embed.v_2.tasks.create(
28    input_type="document",
29    model_name="marengo3.5",
30    document=AsyncDocumentInputRequest(
31        media_source=MediaSource(
32            asset_id=asset.id,
33            # url="<YOUR_DOCUMENT_URL>", # Use direct links to raw files. Cloud storage sharing links are not supported
34            # base_64_string="<BASE_64_ENCODED_DATA>",
35        ),
36        # embedding_scope=["local"], # One embedding per page
37    ),
38    # embedding_uncertainty=True,
39)
40print(f"Task ID: {task.id}")
41
42# 5. Monitor the status
43while True:
44    task = client.embed.v_2.tasks.retrieve(task_id=task.id)
45    if task.status == "ready":
46        print("Task completed")
47        break
48    elif task.status == "failed":
49        print("Task failed")
50        break
51    else:
52        print("Task still processing...")
53        time.sleep(5)
54
55# 6. Process the results
56print(f"Number of embeddings: {len(task.data)}")
57for embedding_data in task.data:
58    print(f"[{embedding_data.embedding_scope}] pages {embedding_data.start_page_number}-{embedding_data.end_page_number}")
59    print(f"Embedding dimensions: {len(embedding_data.embedding)}")
60    print(f"First 10 values: {embedding_data.embedding[:10]}")

Code explanation

1

Import the SDK and initialize the client

Create a client instance to interact with the TwelveLabs Video Understanding Platform.
Function call: You call the constructor of the TwelveLabs class.
Parameters:

  • api_key: The API key to authenticate your requests to the platform.

Return value: An object of type TwelveLabs configured for making API calls.

2

Upload a document

Upload a document to create an asset.
Function call: You call the assets.create function.
Parameters:

  • method: The upload method for your asset. Use url for a publicly accessible file or direct to upload a local file. This example uses url.
  • url or file: The publicly accessible URL of your document file or an opened file object in binary read mode. This example uses url.

Return value: An object of type Asset. This object contains, among other information, a field named id representing the unique identifier of your asset.

3

Check the status of the asset

Asset processing is asynchronous. Poll the status of the asset until it is ready before you use it.
Function call: You call the assets.retrieve function.
Parameters:

  • asset_id: The unique identifier of your asset.

Return value: An object of type Asset containing, among other information, a field named status representing the current status of the asset. Check this field until its value is ready.

4

Create an embedding task

Create an embedding task to start processing your document.
Function call: You call the embed.v_2.tasks.create function.
Parameters:

  • input_type: The type of content. Set this parameter to document.
  • model_name: The embedding model to use. This example uses marengo3.5.
  • (Optional) embedding_uncertainty: Set this parameter to true to receive a data[].embedding_uncertainty field in the response. This field is a per-dimension uncertainty vector with the same length as the embedding array. A higher value shows lower confidence in that dimension.
  • document: An object containing the following properties:
    • media_source: An object specifying the source of the document file. Specify one of the following:

      • asset_id: The unique identifier of an asset from a previous upload.

      • url: The publicly accessible URL of the document file.

      • base_64_string: The base64-encoded document data.

        This example uses the identifier of the asset created in the previous step.

    • (Optional) embedding_option: The type of content to embed. Valid values are the following:
      • visual: Embeds the rendered pages. Valid for PDF files.
      • text: Not supported. Returns a 400 error.
    • (Optional) embedding_scope: The scope for which to generate embeddings. Valid values are the following:
      • local: Returns one embedding per page. The only supported scope for PDF files, and the default.
      • asset: Not supported for PDF files.
    • (Optional) embedding_type: Specifies how to structure the embedding. The only valid value is separate_embedding. Documents have a single modality, so fused_embedding returns a 400 error.

Return value: An object of type TasksCreateResponse containing, among other information, a field named id, which represents the unique identifier of your embedding task. You can use this identifier to track the status of your embedding task.

5

Monitor the status

The platform requires some time to process documents. Poll the status of the embedding task until it is ready. This example uses a loop to check the status every 5 seconds.
Function call: You repeatedly call the embed.v_2.tasks.retrieve function until the task completes.

Parameters:

  • task_id: The unique identifier of your embedding task.

Return value: An object of type EmbeddingTaskResponse containing, among other information, the following fields:

  • status: The current status of the task. The possible values are:
    • processing: The platform is creating the embeddings.
    • ready: Processing is complete. Embeddings are available in the data field.
    • failed: The task failed.
  • data: When the status is ready, this field contains a list of embedding objects. Each embedding object includes:
    • embedding: The embedding vector (a list of floats).
    • embedding_scope: The scope of the embedding. A request that sets the local scope returns page.
    • start_page_number: The first page this embedding covers, counting from 1. The platform returns this field only for page-level embeddings of a PDF file.
    • end_page_number: The last page this embedding covers, counting from 1 and including that page.
6

Process the results

This example iterates through the embeddings in the data field and prints the scope, page range, dimensions, and the first 10 vector values for each embedding.