The TwelvelabsApiClient interface provides methods to analyze videos synchronously and generate text based on their content. These methods return results immediately in the response.

When to use these methods:

Analyze videos up to 1 hour
Retrieve immediate results without polling for task completion
Stream text fragments in real time for immediate processing and feedback

Do not use these methods for:

Videos longer than 1 hour. Use the analyzeAsync.tasks.create method instead.
Video segmentation with custom segment definitions. Use the analyzeAsync.tasks.create method with modelName set to "pegasus1.5" instead.

Sync analysis

Description: This method analyzes your videos and returns the results directly in the response. It generates text based on your prompts and supports both Pegasus 1.2 and Pegasus 1.5 for general analysis (prompt-based text generation).

Note

This method is rate-limited. For details, see the Rate limits page.

Function signature and example:

1 analyze(request: TwelvelabsApi.AnalyzeRequest, requestOptions?: TwelvelabsApiClient.RequestOptions): core.HttpResponsePromise<TwelvelabsApi.NonStreamAnalyzeResponse>;

Parameters:

Name	Type	Required	Description
`request`	`TwelvelabsApi.AnalyzeRequest`	Yes	The request object containing the parameters for performing an analysis.
`requestOptions`	`RequestOptions`	No	Request-specific configuration.

The AnalyzeRequest interface contains the following properties:

Name	Type	Required	Description
`modelName`	`string`	No	The video understanding model to use for analysis. Values: - `"pegasus1.2"`: General analysis (prompt-based text generation). - `"pegasus1.5"`: General analysis (prompt-based text generation) with video clipping, structured prompts with reference images, extended token limits, and video segmentation (async only). Default: `"pegasus1.2"`
`videoId`	`string`	No	The unique identifier of the video to analyze. Use this parameter when the `modelName` parameter is `"pegasus1.2"`. Not supported with `"pegasus1.5"`. This parameter will be deprecated and removed in a future version. Use the `video` parameter instead.
`video`	`TwelvelabsApi.VideoContext`	No	An object specifying the source of the video content. Include exactly one source. See VideoContext.
`prompt`	`string`	No	A text prompt that guides the model on the desired format or content. Works with both Pegasus 1.2 and Pegasus 1.5. To include reference images in your prompt, use the `promptV2` parameter instead (Pegasus 1.5 only). Mutually exclusive with the `promptV2` parameter.
`promptV2`	`TwelvelabsApi.AnalyzePromptV2`	No	A structured prompt with `<@name>` placeholders for referencing images. Requires the `modelName` parameter set to `"pegasus1.5"`. Mutually exclusive with the `prompt` parameter. See AnalyzePromptV2.
`temperature`	`number`	No	Controls the randomness of the text output. Default: 0.2, Min: 0, Max: 1
`startTime`	`number`	No	Start of the analysis window, in seconds. Use with `endTime` to analyze only a portion of the video. Requires `modelName` set to `"pegasus1.5"`. If omitted, defaults to `0`. Must be less than `endTime` and less than the video duration. The clip (`endTime` − `startTime`) must be at least `4` seconds.
`endTime`	`number`	No	End of the analysis window, in seconds. Use with `startTime` to analyze only a portion of the video. Requires `modelName` set to `"pegasus1.5"`. If omitted, defaults to the video duration. Must be greater than `startTime` and less than or equal to the video duration. The clip (`endTime` − `startTime`) must be at least `4` seconds.
`responseFormat`	`TwelvelabsApi.SyncResponseFormat`	No	Specifies the format of the response. When you omit this parameter, the platform returns unstructured text. Only the `json_schema` type is supported for synchronous analysis.
`maxTokens`	`number`	No	The maximum number of tokens to generate. The allowed range depends on the model: Pegasus 1.2 — Min: 1, Max: 4,096, Default: 4,096. Pegasus 1.5 — Min: 512, Max: 65,536, Default: 4,096.

The SyncResponseFormat class contains the following properties:

Name	Type	Description
`type`	`TwelvelabsApi.SyncResponseFormatType`	The response format to use. Value: `"json_schema"` (structured JSON conforming to a provided schema).
`jsonSchema`	`Record<string, unknown>`	Contains the JSON schema that defines the response structure. The schema must adhere to the JSON Schema Draft 2020-12 specification. For details, see the `json_schema` parameter in the API Reference section.

VideoContext

The VideoContext type specifies the source of the video content. Provide exactly one of the following. Set the type discriminator field to "url", "asset_id", or "base64_string" to indicate which source you are providing.

Object	Description
`{ type: "url", url: string }`	The publicly accessible URL of the video file. Use direct links to raw media files. Video hosting platforms and cloud storage sharing links are not supported.
`{ type: "asset_id", assetId: string }`	The unique identifier of an asset from a direct or multipart upload. The asset status must be `ready`. Use `assets.retrieve` to check the status.
`{ type: "base64_string", base64String: string }`	The base64-encoded video data. The maximum size is 30MB.

AnalyzePromptV2

The AnalyzePromptV2 interface defines a structured prompt with image references.

Name	Type	Required	Description
`inputText`	`string`	Yes	The text of the prompt. Use `<@name>` placeholders to reference images declared in `mediaSources` (Example: `"Is there a <@tiger-1> in the video?"`). The maximum length is 2,000 tokens.
`mediaSources`	`TwelvelabsApi.SmeMediaSource[]`	No	Reference images for the `<@name>` placeholders in the prompt. Maximum 4 sources. See SmeMediaSource.

SmeMediaSource

A reference image that provides visual context. Provide exactly one of url, assetId, or base64String.

Name	Type	Required	Description
`name`	`string`	Yes	A descriptive name for this media source.
`mediaType`	`TwelvelabsApi.SmeMediaSourceMediaType`	Yes	The media type. Value: `"image"`.
`url`	`string`	No	A publicly accessible HTTPS URL of the image.
`assetId`	`string`	No	The unique identifier of an uploaded asset.
`base64String`	`string`	No	Base64-encoded image data. The maximum size is 30MB.

Return value: Returns a Promise that resolves to a NonStreamAnalyzeResponse object containing the generated text.

The NonStreamAnalyzeResponse interface contains the following properties:

Name	Type	Description
`id`	`string`	Unique identifier of the response.
`data`	`string`	The generated text based on the prompt you provided.
`finishReason`	`TwelvelabsApi.FinishReason`	The reason for the generation to finish. Values: - `null`: The generation hasn’t finished yet. - `stop`: The generation stopped because the model reached the maximum number of tokens. For JSON responses, this may result in truncated, invalid JSON that fails to parse.
`usage`	`TokenUsage`	The number of tokens used in the generation.

The TokenUsage interface contains the following properties:

Name	Type	Description
`outputTokens`	`number`	The number of tokens in the generated text.
`inputTokens`	`number`	The number of tokens consumed by the input (prompt and video). Omitted for Pegasus 1.5.

The maximum response length is 4,096 tokens for Pegasus 1.2 and up to 65,536 tokens for Pegasus 1.5. Set the maxTokens parameter to change this limit.

API Reference: Sync analysis.

Related guide: Analyze videos.

Sync analysis with streaming responses

Note

This method is rate-limited. For details, see the Rate limits page.

Function signature and example:

1 async analyzeStream(
2   request: TwelvelabsApi.AnalyzeStreamRequest,
3   requestOptions?: TwelvelabsApiClient.RequestOptions
4 ): core.HttpResponsePromise<core.Stream<TwelvelabsApi.StreamAnalyzeResponse>>

Parameters:

Name	Type	Required	Description
`request`	`TwelvelabsApi.AnalyzeStreamRequest`	Yes	The request object containing the parameters for performing an analysis.
`requestOptions`	`RequestOptions`	No	Request-specific configuration.

The AnalyzeStreamRequest interface contains the following properties:

Name	Type	Required	Description
`modelName`	`string`	No	The video understanding model to use for analysis. Values: - `"pegasus1.2"`: General analysis (prompt-based text generation). - `"pegasus1.5"`: General analysis (prompt-based text generation) with video clipping, structured prompts with reference images, extended token limits, and video segmentation (async only). Default: `"pegasus1.2"`
`videoId`	`string`	No	The unique identifier of the video to analyze. Use this parameter when the `modelName` parameter is `"pegasus1.2"`. Not supported with `"pegasus1.5"`. This parameter will be deprecated and removed in a future version. Use the `video` parameter instead.
`video`	`TwelvelabsApi.VideoContext`	No	An object specifying the source of the video content. Include exactly one source. See VideoContext.
`prompt`	`string`	No	A text prompt that guides the model on the desired format or content. Works with both Pegasus 1.2 and Pegasus 1.5. To include reference images in your prompt, use the `promptV2` parameter instead (Pegasus 1.5 only). Mutually exclusive with the `promptV2` parameter.
`promptV2`	`TwelvelabsApi.AnalyzePromptV2`	No	A structured prompt with `<@name>` placeholders for referencing images. Requires the `modelName` parameter set to `"pegasus1.5"`. Mutually exclusive with the `prompt` parameter. See AnalyzePromptV2.
`temperature`	`number`	No	Controls the randomness of the text output. Default: 0.2, Min: 0, Max: 1
`startTime`	`number`	No	Start of the analysis window, in seconds. Use with `endTime` to analyze only a portion of the video. Requires `modelName` set to `"pegasus1.5"`. If omitted, defaults to `0`. Must be less than `endTime` and less than the video duration. The clip (`endTime` − `startTime`) must be at least `4` seconds.
`endTime`	`number`	No	End of the analysis window, in seconds. Use with `startTime` to analyze only a portion of the video. Requires `modelName` set to `"pegasus1.5"`. If omitted, defaults to the video duration. Must be greater than `startTime` and less than or equal to the video duration. The clip (`endTime` − `startTime`) must be at least `4` seconds.
`responseFormat`	`TwelvelabsApi.SyncResponseFormat`	No	Specifies the format of the response. When you omit this parameter, the platform returns unstructured text. Only the `json_schema` type is supported for synchronous analysis.
`maxTokens`	`number`	No	The maximum number of tokens to generate. The allowed range depends on the model: Pegasus 1.2 — Min: 1, Max: 4,096, Default: 4,096. Pegasus 1.5 — Min: 512, Max: 65,536, Default: 4,096.

For details about VideoContext, AnalyzePromptV2, and SmeMediaSource, see the Sync analysis section above.

The SyncResponseFormat class contains the following properties:

Name	Type	Description
`type`	`TwelvelabsApi.SyncResponseFormatType`	The response format to use. Value: `"json_schema"` (structured JSON conforming to a provided schema).
`jsonSchema`	`Record<string, unknown>`	Contains the JSON schema that defines the response structure. The schema must adhere to the JSON Schema Draft 2020-12 specification. For details, see the `json_schema` parameter in the API Reference section.

Return value: Returns a promise that resolves to a Stream<TwelvelabsApi.StreamAnalyzeResponse> object that can be iterated over to receive streaming text chunks.

The StreamAnalyzeResponse can be either a StreamAnalyzeResponse.StreamStart, a StreamAnalyzeResponse.TextGeneration, or a StreamAnalyzeResponse.StreamEnd.

The StreamAnalyzeResponse.StreamStart interface contains the following properties:

Name	Type	Description
`eventType`	`"stream_start"`	This field is always set to `stream_start` for this event.
`metadata`	`StreamStartResponseMetadata`	An object containing metadata about the stream.

The StreamAnalyzeResponse.TextGeneration interface contains the following properties:

Name	Type	Description
`eventType`	`"text_generation"`	This field is always set to `text_generation` for this event.
`text`	`string`	A fragment of the generated text. Text fragments may be split at arbitrary points, not necessarily at word or sentence boundaries.

The StreamAnalyzeResponse.StreamEnd interface contains the following properties:

Name	Type	Description
`eventType`	`"stream_end"`	This field is always set to `stream_end` for this event.
`finishReason`	`TwelvelabsApi.FinishReason`	The reason for the generation to finish. Values: - `null`: The generation hasn’t finished yet. - `stop`: The generation stopped because the model reached the maximum number of tokens. For JSON responses, this may result in truncated, invalid JSON that fails to parse.
`metadata`	`StreamEndResponseMetadata`	An object containing metadata about the stream.

The StreamStartResponseMetadata interface contains the following properties:

Name	Type	Description
`generationId`	`string`	A unique identifier for the generation session.

The StreamEndResponseMetadata interface contains the following properties:

Name	Type	Description
`generationId`	`string`	The same unique identifier provided in the `stream_start` event.
`usage`	`TokenUsage`	The number of tokens used in the generation.

The TokenUsage interface contains the following properties:

Name	Type	Description
`outputTokens`	`number`	The number of tokens in the generated text.
`inputTokens`	`number`	The number of tokens consumed by the input (prompt and video). Omitted for Pegasus 1.5.

The maximum response length is 4,096 tokens for Pegasus 1.2 and up to 65,536 tokens for Pegasus 1.5. Set the maxTokens parameter to change this limit.

API Reference: Sync analysis.

Related guide: Analyze videos.

Error codes

This section lists the most common error messages you may encounter while analyzing videos.

token_limit_exceeded
- Your request could not be processed due to exceeding maximum token limit. Please try with another request or another video with shorter duration.
index_not_supported_for_generate
- You can only summarize videos uploaded to an index with an engine from the Pegasus family enabled.