GuidesCreate embeddings

Image embeddings

This guide shows how you can create image embeddings using the Marengo video understanding model. For a list of available versions, complete specifications and input requirements for each version, see the Marengo page.

The Marengo video understanding model generates embeddings for all modalities in the same latent space. This shared space enables any-to-any searches across different types of content.

Prerequisites

  • To use the platform, you need an API key:

    1

    If you don’t have an account, sign up for a free account.

    2

    Go to the API Keys page.

    3

    Select the Copy icon next to your key.

  • Depending on the programming language you are using, install the TwelveLabs SDK by entering one of the following commands:

    $pip install twelvelabs

  • Your image files must meet the format requirements.

Complete example

This complete example shows how you can create image embeddings. Ensure you replace the placeholders surrounded by <> with your values.

1from typing import List
2
3from twelvelabs import TwelveLabs
4from twelvelabs.types import BaseSegment
5
6
7# 1. Initialize the client
8client = TwelveLabs(api_key="<YOUR_API_KEY>")
9
10# 2. Create image embeddings
11res = client.embed.create(
12    model_name="marengo3.0",
13    image_url="<YOUR_IMAGE_URL>"
14    # Or for a local file: image_file=open("<PATH_TO_IMAGE_FILE>", "rb")
15)
16
17# 3. Process the results
18def print_segments(segments: List[BaseSegment], max_elements: int = 5):
19    for segment in segments:
20        first_few = segment.float_[:max_elements]
21        print(
22            f"  embeddings: [{', '.join(str(x) for x in first_few)}...] (total: {len(segment.float_)} values)"
23        )
24
25
26print("Created image embedding")
27if res.image_embedding is not None and res.image_embedding.segments is not None:
28    print_segments(res.image_embedding.segments)

Step-by-step guide

1

Import the SDK and initialize the client

Create a client instance to interact with the TwelveLabs Video Understanding Platform.
Function call: You call the constructor of the TwelveLabs class.
Parameters:

  • api_key: The API key to authenticate your requests to the platform.

Return value: An object of type TwelveLabs configured for making API calls.

2

Create image embeddings

Function call: You call the embed.create function.
Parameters:

  • model_name: The name of the model you want to use (example: “marengo3.0”).
  • image_url or image_file:
    • image_url: The publicly accessible URL of your image file (string)
    • image_file: An opened file object in binary read mode. Use open(path, 'rb') to open your local file

Return value: The response contains the following fields:

  • image_embedding: An object that contains the embedding data for your image file. It includes the following fields:
    • segments: An object that contains the following:
      • float_: An array of floats representing the embedding
    • metadata: An object that contains metadata about the embedding.
  • model_name: The name of the video understanding model the platform has used to create this embedding.
3

Process the results

This example prints the results to the standard output.