The TwelveLabs class provides methods to analyze videos synchronously and generate text based on their content. These methods return results immediately in the response.

When to use these methods:

• Analyze videos up to 1 hour
• Retrieve immediate results without polling for task completion
• Stream text fragments in real time for immediate processing and feedback

Do not use these methods for:

• Videos longer than 1 hour. Use the analyze_async.tasks.create method instead.
• Video segmentation with custom segment definitions. Use the analyze_async.tasks.create method with model_name set to "pegasus1.5" instead.

Sync analysis

Description: This method analyzes your videos and returns the results directly in the response. It generates text based on your prompts and supports both Pegasus 1.2 and Pegasus 1.5 for general analysis (prompt-based text generation).

Function signature and example:

1
def analyze(
2
    self,
3
    *,
4
    model_name: typing.Optional[AnalyzeRequestModelName] = OMIT,
5
    video_id: typing.Optional[str] = OMIT,
6
    video: typing.Optional[VideoContext] = OMIT,
7
    prompt: typing.Optional[AnalyzeTextPrompt] = OMIT,
8
    prompt_v_2: typing.Optional[AnalyzePromptV2] = OMIT,
9
    temperature: typing.Optional[AnalyzeTemperature] = OMIT,
10
    response_format: typing.Optional[SyncResponseFormat] = OMIT,
11
    max_tokens: typing.Optional[int] = OMIT,
12
    start_time: typing.Optional[float] = OMIT,
13
    end_time: typing.Optional[float] = OMIT,
14
    request_options: typing.Optional[RequestOptions] = None,
15
) -> NonStreamAnalyzeResponse:

Parameters:

| response_format | typing.Optional[SyncResponseFormat] | No | Specifies the format of the response. When you omit this parameter, the platform returns unstructured text. Only the json_schema type is supported for synchronous analysis. | | max_tokens | typing.Optional[int] | No | The maximum response length, in tokens. The allowed range depends on the model: Pegasus 1.2 — Min: 1, Max: 4,096, Default: 4,096. Pegasus 1.5 — Min: 512, Max: 98,304, Default: 4,096. For Pegasus 1.5, the input and response must fit within the context window. | | request_options | typing.Optional[RequestOptions] | No | Request-specific configuration. |

The SyncResponseFormat class contains the following properties:

VideoContext

The VideoContext type specifies the source of the video content. Provide exactly one of the following:

AnalyzePromptV2

The AnalyzePromptV2 class defines a structured prompt with image references.

SmeMediaSource

A reference image that provides visual context. Provide exactly one of url, asset_id, or base64_string.

Return value: Returns a NonStreamAnalyzeResponse object containing the generated text.

The NonStreamAnalyzeResponse class contains the following properties:

The TokenUsage class contains the following properties:

API Reference: Sync analysis.

Related guide: Analyze videos.

Sync analysis with streaming responses

Description: This method analyzes your videos and returns the results directly in the response. It generates text based on your prompts and supports both Pegasus 1.2 and Pegasus 1.5 for general analysis (prompt-based text generation).

Function signature and example:

1
def analyze_stream(
2
    self,
3
    *,
4
    model_name: typing.Optional[AnalyzeStreamRequestModelName] = OMIT,
5
    video_id: typing.Optional[str] = OMIT,
6
    video: typing.Optional[VideoContext] = OMIT,
7
    prompt: typing.Optional[AnalyzeTextPrompt] = OMIT,
8
    prompt_v_2: typing.Optional[AnalyzePromptV2] = OMIT,
9
    temperature: typing.Optional[AnalyzeTemperature] = OMIT,
10
    response_format: typing.Optional[SyncResponseFormat] = OMIT,
11
    max_tokens: typing.Optional[int] = OMIT,
12
    start_time: typing.Optional[float] = OMIT,
13
    end_time: typing.Optional[float] = OMIT,
14
    request_options: typing.Optional[RequestOptions] = None,
15
) -> typing.Iterator[StreamAnalyzeResponse]

Parameters:

| response_format | typing.Optional[SyncResponseFormat] | No | Specifies the format of the response. When you omit this parameter, the platform returns unstructured text. Only the json_schema type is supported for synchronous analysis. | | max_tokens | typing.Optional[int] | No | The maximum response length, in tokens. The allowed range depends on the model: Pegasus 1.2 — Min: 1, Max: 4,096, Default: 4,096. Pegasus 1.5 — Min: 512, Max: 98,304, Default: 4,096. For Pegasus 1.5, the input and response must fit within the context window. | | request_options | typing.Optional[RequestOptions] | No | Request-specific configuration. |

For details about VideoContext, AnalyzePromptV2, and SmeMediaSource, see the Sync analysis section above.

The SyncResponseFormat class contains the following properties:

Return value: Returns an iterator of StreamAnalyzeResponse objects. Each response can be a StreamAnalyzeResponse_StreamStart, StreamAnalyzeResponse_TextGeneration, or StreamAnalyzeResponse_StreamEnd.

The StreamAnalyzeResponse_StreamStart class contains the following properties:

The StreamAnalyzeResponse_TextGeneration class contains the following properties:

The StreamAnalyzeResponse_StreamEnd class contains the following properties:

The StreamStartResponseMetadata class contains the following properties:

The StreamEndResponseMetadata class contains the following properties:

The TokenUsage class contains the following properties:

API Reference: Sync analysis.

Related guide: Analyze videos.

Error codes

This section lists the most common error messages you may encounter while analyzing videos.

• token_limit_exceeded
  • The request exceeds the context window and cannot be processed. Reduce the prompt length, use a shorter video, or lower the max_tokens value.
• output truncated: the generation reached the configured max_tokens. The partial output is returned; raise max_tokens (up to 98304) if you need a longer response.
  • The response reached the maximum response length. The platform returns the partial output and sets finish_reason to "length". This message appears in the error.message field.
• output truncated: combined input and output tokens reached the model's context limit. The partial output is returned; consider reducing input size (shorter prompt, smaller video clip, fewer media bindings) or lowering max_tokens.
  • The request reached the context window. The platform returns the partial output and sets finish_reason to "length". This message appears in the error.message field.
• index_not_supported_for_generate
  • You can only summarize videos uploaded to an index with an engine from the Pegasus family enabled.

For a list of general errors that apply to all endpoints, see the Error codes page.