Image embeddings

This guide shows how you can create image embeddings.

The following table lists the available models for generating text embeddings and their key characteristics:

ModelDescriptionDimensionsMax sizeSimilarity metric
Marengo-retrieval-2.7Use this model to create embeddings that you can use in various downstream tasks10245 MBCosine similarity

The “Marengo-Retrieval-2.7” video understanding model generates embeddings for all modalities in the same latent space. This shared space enables any-to-any searches across different types of content.

Prerequisites

  • To use the platform, you need an API key:

    1

    If you don’t have an account, sign up for a free account.

    2

    Go to the API Key page.

    3

    Select the Copy icon next to your key.

  • Ensure the TwelveLabs SDK is installed on your computer:

    $pip install twelvelabs
  • The images you wish to use use must meet the following requirements:

    • Format: JPEG and PNG.
    • Dimension: Must be at least 128 x 128 pixels.
    • Size: Must not exceed 5MB.

Complete example

This complete example shows how you can create image emebddings. Ensure you replace the placeholders surrounded by <> with your values.

1from twelvelabs import TwelveLabs
2from typing import List
3from twelvelabs.models.embed import SegmentEmbedding
4
5# 1. Initialize the client
6client = TwelveLabs(api_key="<YOUR_API_KEY>")
7
8# 2. Create audio embeddings
9res = client.embed.create(
10 model_name="Marengo-retrieval-2.7", image_url="<YOUR_IMAGE_URL>")
11
12# 3. Process the results
13print(f"Created image embedding: model_name={res.model_name}")
14def print_segments(segments: List[SegmentEmbedding], max_elements: int = 5):
15 for segment in segments:
16 print(f" embeddings: {segment.embeddings_float[:max_elements]}")
17
18if res.image_embedding is not None and res.image_embedding.segments is not None:
19 print_segments(res.image_embedding.segments)

Step-by-step guide

1

Import the SDK and initialize the client

Create a client instance to interact with the TwelveLabs Video Understanding platform.
Function call: You call the constructor of the TwelveLabs class.
Parameters:

  • api_key: The API key to authenticate your requests to the platform.

Return value: An object of type TwelveLabs configured for making API calls.

2

Create image embeddings

Function call: You call the embed.create function.
Parameters:

  • model_name: The name of the model you want to use (“Marengo-retrieval-2.7”).
  • image_file or image_url: The path or the publicly accessible URL of your image file.

Return value: The response contains the following fields:

  • image_embedding: An object that contains the embedding data for your image file. It includes the following fields:
    • segments: An object that contains the following:
      • float: An array of floats representing the embedding
    • metadata: An object that contains metadata about the embedding.
  • model_name: The name ofhe video understanding model the platform has used to create this embedding.
3

Process the results

This example prints the results to the standard output.