This guide shows how you can create image embeddings.

The following table lists the available models for generating image embeddings and their key characteristics:

Model	Description	Dimensions	Max size	Similarity metric
Marengo-retrieval-2.7	Use this model to create embeddings that you can use in various downstream tasks	1024	5 MB	Cosine similarity

The Marengo video understanding model generates embeddings for all modalities in the same latent space. This shared space enables any-to-any searches across different types of content.

Prerequisites

To use the platform, you need an API key:

1
If you don’t have an account, sign up for a free account.
2
Go to the API Keys page.
3
Select the Copy icon next to your key.

Ensure the TwelveLabs SDK is installed on your computer:

$ pip install twelvelabs

The images you wish to use must meet the following requirements:
- Format: JPEG and PNG.
- Dimension: Must be at least 128 x 128 pixels.
- Size: Must not exceed 5MB.

Complete example

This complete example shows how you can create image embeddings. Ensure you replace the placeholders surrounded by <> with your values.

1 from typing import List
2 
3 from twelvelabs import TwelveLabs
4 from twelvelabs.types import BaseSegment
5 
6 
7 # 1. Initialize the client
8 client = TwelveLabs(api_key="<YOUR_API_KEY>")
9 
10 # 2. Create image embeddings
11 res = client.embed.create(
12     model_name="Marengo-retrieval-2.7",
13     image_url="<YOUR_IMAGE_URL>" # Or use image_file to upload a file from the local file system
14 )
15 
16 # 3. Process the results
17 def print_segments(segments: List[BaseSegment], max_elements: int = 5):
18     for segment in segments:
19         first_few = segment.float_[:max_elements]
20         print(
21             f"  embeddings: [{', '.join(str(x) for x in first_few)}...] (total: {len(segment.float_)} values)"
22         )
23 
24 
25 print("Created image embedding")
26 if res.image_embedding is not None and res.image_embedding.segments is not None:
27     print_segments(res.image_embedding.segments)

Step-by-step guide

Python

Node.js

Import the SDK and initialize the client

Create a client instance to interact with the TwelveLabs Video Understanding Platform.
Function call: You call the constructor of the TwelveLabs class.
Parameters:

api_key: The API key to authenticate your requests to the platform.

Return value: An object of type TwelveLabs configured for making API calls.

Create image embeddings

Function call: You call the embed.create function.
Parameters:

model_name: The name of the model you want to use (“Marengo-retrieval-2.7”).
image_url or image_file: The publicly accessible URL or the path of your image file.

Return value: The response contains the following fields:

image_embedding: An object that contains the embedding data for your image file. It includes the following fields:
- segments: An object that contains the following:
  - float_: An array of floats representing the embedding
- metadata: An object that contains metadata about the embedding.
model_name: The name of the video understanding model the platform has used to create this embedding.

Process the results

This example prints the results to the standard output.