Create text image, and audio embeddings

The Resources.Embed class provides methods to create text, image, and audio embeddings.

Create text, image, and audio embeddings

Description: This method creates a new embedding.

Note that you must specify at least the following parameters:

  • engineName: The name of the video understanding engine to use.
  • One or more of the following input types:
    • text: For text embeddings
    • audioUrl or audioFile: For audio embeddings. If you specify both, the audioUrl parameter takes precedence.
    • imageUrl or imageFile: For image embeddings. If you specify both, the imageUrl parameter takes precedence.

You must provide at least one input type, but you can include multiple types in a single function call.

Function signature and example:

async create(
  { engineName, text, textTruncate, audioUrl, audioFile, imageUrl, imageFile }: CreateEmbedParams,
  options: RequestOptions = {},
): Promise<Models.CreateEmbeddingsResult>
const printSegments = (segments: SegmentEmbedding[], maxElements = 5) => {
  segments.forEach((segment) => {
    console.log(
      `  embedding_scope=${segment.embeddingScope} start_offset_sec=${segment.startOffsetSec} end_offset_sec=${segment.endOffsetSec}`
    );
    console.log(
      "  embeddings: ",
      segment.embeddingsFloat.slice(0, maxElements)
    );
  });
};

cont res = await client.embed.create({
  engineName: "Marengo-retrieval-2.6",
  text: "<YOUR_TEXT>",
});

console.log(`Created text embedding: engineName=${res.engineName}`);
if (res.textEmbedding?.segments) {
  printSegments(res.textEmbedding.segments);
}

Parameters:

NameTypeRequiredDescription
paramsCreateEmbedParamsYesParameters for creating the text embedding.
optionsRequestOptionsNoAdditional options for the request. Defaults to {}.

The CreateEmbedParams interface defines the parameters for creating a text embedding.

NameTypeRequiredDescription
engineNamestringYesThe name of the video understanding engine to use. Example: "Marengo-retrieval-2.6".
textstringYesThe text for which you want to create an embedding.
textTruncate'none' | 'start' | 'end'NoSpecifies how to truncate the text if it exceeds the maximum length of 77 tokens.
audioUrl

Return value: Returns a Promise that resolves to a Models.CreateEmbeddingsResult instance.

API Reference: For a description of each field in the request and response, see the Create text embeddings page.

Related guide: Create text embeddings.