API Reference
SDKsSample appsIntegrationsDiscordPlayground

Create video embeddings

The Resources.EmbedTask class provides methods to create embeddings for your videos.

To create video embeddings:

  1. Create a video embedding task that uploads and processes a video.
  2. Monitor the status of your task.
  3. Retrieve the embeddings once the task is completed.

Methods

Create a video embedding task

Description: This method creates a new video embedding task.

Function signature and example:

async create(
  modelName: string,
  { file, url, startOffsetSec, endOffsetSec, clipLength, scopes }: CreateEmbeddingsTaskVideoParams,
  options: RequestOptions = {},
): Promise<Models.EmbeddingsTask | undefined>

const params = {
  file: '<YOUR_VIDEO_FILE>',
  url: '<YOUR_VIDEO_URL>',
  startOffsetSec: 5,
  endOffsetSec: 15,
  clipLength: 5,
  scopes: ['clip', 'video'],
};
const task = await client.embed.task.create('Marengo-retrieval-2.7', params);

console.log(`Task ID: ${task.id}`);
console.log(`Model Name: ${task.modelName}`);
console.log(`Status: ${task.status}`);

Parameters:

NameTypeRequiredDescription
modelNamestringYesThe name of the model you want to use (example: "Marengo-retrieval-2.7".
paramsCreateEmbeddingsTaskVideoParamsYesParameters for creating the video embedding task.
optionsRequestOptionsNoAdditional options for the request. Defaults to {}.

The CreateEmbeddingsTaskVideoParams interface defines the parameters for creating a video embedding task:

NameTypeRequiredDescription
fileBuffer | NodeJS.ReadableStream | stringNoThe video file you want to upload. This parameter can be a Buffer, a ReadableStream, or a string representing the path to the video file.
urlstringNoThe publicly accessible URL of the video file.
startOffsetSecnumberNoThe start offset in seconds from the beginning of the video where processing should begin.
endOffsetSecnumberNoThe end offset in seconds from the beginning of the video where processing should stop.
clipLengthnumberNoThe desired duration in seconds for each clip for which the platform generates an embedding.
scopesArray<'clip' | 'video'>NoSpecifies the embedding scope. Valid values are:

- ['clip']\
- ['clip', 'video']

Return value: Returns a Promise that resolves to a Models.EmbeddingsTask instance.

API Reference: For a description of each field in the request and response, see the Create a video embedding task page.

Related guide: Create video embeddings.

Retrieve the status of a video embedding task

Description: This method retrieves the status of a video embedding task.

Function signature and example:

async status(taskId: string, options: RequestOptions = {}): Promise<Models.EmbeddingsTaskStatus>

const task = await client.embed.task.status('<YOUR_TASK_ID>')

console.log(`Task ID: ${task.id}`);
console.log(`Model name: ${task.modelName}`);
console.log(`Status: ${task.status}`);

Parameters:

NameTypeRequiredDescription
taskIdstringYesThe unique identifier of a video embedding task.
optionsRequestOptionsNoAdditional options for the request. Defaults to {}.

Return value: Returns a Promise that resolves to a Models.EmbeddingsTaskStatus instance.

📘

Note:

You can also use the [waitForDone] method of the Models.EmbeddingTask class to monitor the status of a video indexing task until it completes.

API Reference: For a description of each field in the response, see the Retrieve the status of a video embedding task page.

Related guide: Create video embeddings.

Retrieve video embeddings

Description: This method retrieves embeddings for a specific video embedding task. Ensure the task status is ready before retrieving your embeddings.

Function signature and example:

async retrieve(id: string, options: RequestOptions = {}): Promise<Models.EmbeddingsTask>

const printSegments = (segments: SegmentEmbedding[], maxElements = 5) => {
  segments.forEach((segment) => {
    console.log(
      `  embedding_scope=${segment.embeddingScope} start_offset_sec=${segment.startOffsetSec} end_offset_sec=${segment.endOffsetSec}`
    );
    console.log(
      "  embeddings: ",
      segment.embeddingsFloat.slice(0, maxElements)
    );
  });
};

let task = await client.embed.task.retrieve('<YOUR_TASK_ID>');

console.log(`Task ID: ${task.id}`);
console.log(`Model name: ${task.modelName}`);
console.log(`Status: ${task.status}`);


task = await task.retrieve();
if (task.videoEmbedding) {
  if (task.videoEmbedding.segments) {
    printSegments(task.videoEmbedding.segments);
  }
}

Parameters:

NameTypeRequiredDescription
idstringYesThe unique identifier of the video embedding task.
optionsRequestOptionsNoAdditional options for the request. Defaults to {}.

Return value: Returns a Promise that resolves to a Models.EmbeddingsTask instance.

API Reference: For a description of each field in the response, see the Retrieve video embeddings page.

Related guide: Create video embeddings.