Image embeddings

This guide shows how you can create image embeddings.

The following table lists the available models for generating image embeddings and their key characteristics:

ModelDescriptionDimensionsMax sizeSimilarity metric
Marengo-retrieval-2.7Use this model to create embeddings that you can use in various downstream tasks10245 MBCosine similarity

The Marengo video understanding model generates embeddings for all modalities in the same latent space. This shared space enables any-to-any searches across different types of content.

Prerequisites

  • To use the platform, you need an API key:

    1

    If you don’t have an account, sign up for a free account.

    2

    Go to the API Key page.

    3

    Select the Copy icon next to your key.

  • Ensure the TwelveLabs SDK is installed on your computer:

    $pip install twelvelabs
  • The images you wish to use must meet the following requirements:

    • Format: JPEG and PNG.
    • Dimension: Must be at least 128 x 128 pixels.
    • Size: Must not exceed 5MB.

Complete example

This complete example shows how you can create image embeddings. Ensure you replace the placeholders surrounded by <> with your values.

1from typing import List
2
3from twelvelabs import TwelveLabs
4from twelvelabs.types import BaseSegment
5
6
7# 1. Initialize the client
8client = TwelveLabs(api_key="<YOUR_API_KEY>")
9
10# 2. Create image embeddings
11res = client.embed.create(
12 model_name="Marengo-retrieval-2.7",
13 image_url="<YOUR_IMAGE_URL>"
14)
15
16# 3. Process the results
17def print_segments(segments: List[BaseSegment], max_elements: int = 5):
18 for segment in segments:
19 first_few = segment.float_[:max_elements]
20 print(
21 f" embeddings: [{', '.join(str(x) for x in first_few)}...] (total: {len(segment.float_)} values)"
22 )
23
24
25print("Created image embedding")
26if res.image_embedding is not None and res.image_embedding.segments is not None:
27 print_segments(res.image_embedding.segments)

Step-by-step guide

1

Import the SDK and initialize the client

Create a client instance to interact with the TwelveLabs Video Understanding Platform.
Function call: You call the constructor of the TwelveLabs class.
Parameters:

  • api_key: The API key to authenticate your requests to the platform.

Return value: An object of type TwelveLabs configured for making API calls.

2

Create image embeddings

Function call: You call the embed.create function.
Parameters:

  • model_name: The name of the model you want to use (“Marengo-retrieval-2.7”).
  • image_url or image_file: The publicly accessible URL or the path of your image file.

Return value: The response contains the following fields:

  • image_embedding: An object that contains the embedding data for your image file. It includes the following fields:
    • segments: An object that contains the following:
      • float_: An array of floats representing the embedding
    • metadata: An object that contains metadata about the embedding.
  • model_name: The name of the video understanding model the platform has used to create this embedding.
3

Process the results

This example prints the results to the standard output.