Python SDK

Create video embeddings

The resources.EmbedTask class provides methods to create embeddings for your videos.

To create video embeddings:

  1. Create a video embedding task that uploads and processes a video.
  2. Monitor the status of your task.
  3. Retrieve the embeddings once the task is completed.

Related quickstart notebook

Methods

Create a video embedding task

Description: This method creates a new video embedding task.

The videos you wish to upload must meet the following requirements:

  • Video resolution: Must be at least 360x360 and must not exceed 3840x2160.
  • Aspect ratio: Must be one of 1:1, 4:3, 4:5, 5:4, 16:9, 9:16, or 17:9.
  • Video and audio formats: Your video files must be encoded in the video and audio formats listed on the FFmpeg Formats Documentation page. For videos in other formats, contact us at support@twelvelabs.io.
  • Duration: Must be between 4 seconds and 2 hours (7,200s).
  • File size: Must not exceed 2 GB. If you require different options, contact us at support@twelvelabs.io.

Function signature and example:

1def create(
2    self,
3    model_name: str,
4    *,
5    video_file: Union[str, BinaryIO, None] = None,
6    video_url: Optional[str] = None,
7    video_start_offset_sec: Optional[float] = None,
8    video_end_offset_sec: Optional[float] = None,
9    video_clip_length: Optional[int] = None,
10    video_embedding_scopes: Optional[List[Literal["clip", "video"]]] = None,
11    **kwargs
12) -> models.EmbeddingsTask

Parameters:

NameTypeRequiredDescription
model_nameLiteral["Marengo-retrieval-2.7"]YesThe name of the video understanding model to use. Example: “Marengo-retrieval-2.7”.
video_fileUnion[str, BinaryIO, None]NoPath to the video file or a file-like object.
video_urlOptional[str]NoThe publicly accessible URL of the video.
video_start_offset_secOptional[float]NoThe start offset in seconds from the beginning of the video where processing should begin.
video_end_offset_secOptional[float]NoThe end offset in seconds from the beginning of the video where processing should stop.
video_clip_lengthOptional[int]NoThe desired duration in seconds for each clip for which the platform generates an embedding.
video_embedding_scopesOptional[List[Literal["clip", "video"]]]NoSpecifies the embedding scope. Valid values are:

- ["clip"]
- ["clip", "video"]
**kwargsdictNoAdditional keyword arguments for the request.

Return value: Returns a models.EmbeddingsTask object representing the new video embedding task.

API Reference: For a description of each field in the request and response, see the Create a video embedding task page.

Related guide: Create video embeddings.

Retrieve the status of a video embedding task

Description: This method retrieves the status of a video embedding task.

Function signature and example:

1def status(self, task_id: str, **kwargs) -> models.EmbeddingsTaskStatus

Parameters:

NameTypeRequiredDescription
task_idstringYesThe unique identifier of the video embedding task.
**kwargsdictNoAdditional keyword arguments for the request.

Return value: Returns a models.EmbeddingsTaskStatus object containing the current status of the embedding task.

API Reference: For a description of each field in the response, see the Retrieve the status of a video embedding task page.

Related guide: Create video embeddings.

Wait for a video embedding task to complete

Description: This method waits until a video embedding task is completed by periodically checking its status. If you provide a callback function, it calls the function after each status update with the current task object, allowing you to monitor progress.

1def wait_for_done(
2    self,
3    *,
4    sleep_interval: float = 5.0,
5    callback: Optional[Callable[[EmbeddingsTask], None]] = None,
6    **kwargs,
7) -> str

Parameters

NameTypeRequiredDescription
sleep_intervalfloatNoSets the time in seconds to wait between status checks. Must be greater than 0. Default is 5.0.
callbackOptional[Callable[[EmbeddingsTask], None]]NoProvides an optional function to call after each status check. The function receives the current task object. Use this to monitor progress.
**kwargsdictNoPasses additional keyword arguments to the update_status method when checking the task status.

Return value: Returns a string representing the status of the task.

Retrieve video embeddings

Description: This method retrieves embeddings for a specific video embedding task. Ensure the task status is ready before retrieving your embeddings.

Function signature and example:

1def retrieve(self, **kwargs) -> EmbeddingsTask

Parameters:

NameTypeRequiredDescription
**kwargsdictNoAdditional keyword arguments for the request.

Return value: Returns a models.EmbeddingsTask object containing the details of the embedding task, including the embeddings if available. The video_embeddings property of the returned object is a RootModelList of VideoEmbedding objects when the task is completed, or None if the embeddings are not yet available.

API Reference: For a description of each field in the request and response, see the Retrieve video embeddings page.

Related guide: Create video embeddings.

Error codes

This section lists the most common error messages you may encounter while creating video embeddings.

  • parameter_invalid
    • The video_clip_length parameter is invalid. video_clip_length should be within 2-10 seconds long
    • The video_end_offset_sec parameter is invalid. video_end_offset_sec should be greater than video_start_offset_sec