Create video embeddings

The resources.EmbedTask class provides methods to create embeddings for your videos.

To create video embeddings:

  1. Create a video embedding task that uploads and processes a video.
  2. Monitor the status of your task.
  3. Retrieve the embeddings once the task is completed.

Related quickstart notebook: Open In Colab

Methods

Create a video embedding task

Description: This method creates a new video embedding task.

Function signature and example:

def create(
    self,
    engine_name: str,
    *,
    video_file: Union[str, BinaryIO, None] = None,
    video_url: Optional[str] = None,
    video_start_offset_sec: Optional[float] = None,
    video_end_offset_sec: Optional[float] = None,
    video_clip_length: Optional[int] = None,
    video_embedding_scopes: Optional[List[Literal["clip", "video"]]] = None,
    **kwargs
) -> models.EmbeddingsTask
task = client.embed.task.create(
    engine_name="Marengo-retrieval-2.6",
    # video_file="<YOUR_VIDEO_FILE>",
    video_url="<YOUR_VIDEO_URL>",
    video_start_offset_sec=0,
    video_end_offset_sec=10,
    video_clip_length=5,
    video_embedding_scopes=["clip", "video"]
)

print(f"Task ID: {task.id}")
print(f"Engine Name: {task.engine_name}")
print(f"Status: {task.status}")

Parameters:

NameTypeRequiredDescription
engine_namestrYesThe name of the video understanding engine to use. Example: "Marengo-retrieval-2.6".
video_fileUnion[str, BinaryIO, None]NoPath to the video file or a file-like object.
video_urlOptional[str]NoThe publicly accessible URL of the video.
video_start_offset_secOptional[float]NoThe start offset in seconds from the beginning of the video where processing should begin.
video_end_offset_secOptional[float]NoThe end offset in seconds from the beginning of the video where processing should stop.
video_clip_lengthOptional[int]NoThe desired duration in seconds for each clip for which the platform generates an embedding.
video_embedding_scopesOptional[List[Literal["clip", "video"]]]NoSpecifies the embedding scope. Valid values are:

- ["clip"]
- ["clip", "video"]
**kwargsdictNoAdditional keyword arguments for the request.

Return value: Returns a models.EmbeddingsTask object representing the new video embedding task.

API Reference: For a description of each field in the request and response, see the Create a video embedding task page.

Related guide: Create video embeddings.

Retrieve the status of a video embedding task

Description: This method retrieves the status of a video embedding task.

Function signature and example:

def status(self, task_id: str, **kwargs) -> models.EmbeddingsTaskStatus
task = client.embed.task.status(task_id="<YOUR_TASK_ID>")

print(f"Task ID: {task.id}")
print(f"Engine Name: {task.engine_name}")
print(f"Status: {task.status}")

Parameters:

NameTypeRequiredDescription
task_idstringYesThe unique identifier of the video embedding task.
**kwargsdictNoAdditional keyword arguments for the request.

Return value: Returns a models.EmbeddingsTaskStatus object containing the current status of the embedding task.

📘

Note:

You can also use the wait_for_done method of the models.EmbeddingTask class to monitor the status of a video indexing task until it completes.

API Reference: For a description of each field in the response, see the Retrieve the status of a video embedding task page.

Related guide: Create video embeddings.

Retrieve video embeddings

Description: This method retrieves embeddings for a specific video embedding task. Ensure the task status is ready before retrieving your embeddings.

Function signature and example:

def retrieve(self, id: str, **kwargs) -> models.EmbeddingsTask
def print_segments(segments: List[SegmentEmbedding], max_elements: int = 5):
    for segment in segments:
        print(
            f"  embedding_scope={segment.embedding_scope} start_offset_sec={segment.start_offset_sec} end_offset_sec={segment.end_offset_sec}"
        )
        print(f"  embeddings: {segment.embeddings_float[:max_elements]}")
        
task = client.embed.task.retrieve(id="<YOUR_TASK_ID>")
if task.video_embedding is not None and task.video_embedding.segments is not None:
    print_segments(task.video_embedding.segments)

Parameters:

NameTypeRequiredDescription
idstringYesThe unique identifier of the video embedding task.
**kwargsdictNoAdditional keyword arguments for the request.

Return value: Returns a models.EmbeddingsTask object containing the details of the embedding task, including the embeddings if available. The video_embeddings property of the returned object is a RootModelList of VideoEmbedding objects when the task is completed, or None if the embeddings are not yet available.

API Reference: For a description of each field in the response, see the Retrieve video embeddings page.

Related guide: Create video embeddings.