This guide shows how to analyze videos with a prompt. Upload your video as an asset and analyze it asynchronously, or pass a URL or base64-encoded data directly to the analysis call. For videos under 1 hour, synchronous processing returns the results immediately without polling and also supports streaming responses.

On the Free plan, analyzed video hours count toward a shared limit that also covers indexing. On paid plans, you pay based on how much video you process.

For details on how your usage is measured and billed, see the Pricing page.

Key concepts

This section explains the key concepts and terminology used in this guide:

Asset: Your uploaded content. Once created, you can reference the same asset across multiple operations without uploading the file again.
Analysis task: An asynchronous operation for processing your content and generating text. Contains a status and the resulting text when complete.

Prerequisites

To use the platform, you need an API key:

1
If you don’t have an account, sign up for a free account.
2
Go to the API Keys page.
3
If you need to create a new key, select the Create API Key button. Enter a name and set the expiration period. The default is 12 months.
4
Select the Copy icon next to your key to copy it to your clipboard.
Depending on the programming language you are using, install the TwelveLabs SDK by entering one of the following commands:
```
pip install --upgrade twelvelabs
```
Your videos must meet the following requirements:
- Upload limits: Public video URLs up to 4 GB or local videos up to 200 MB. For a video up to 10 GB, use multipart uploads. See the Upload and processing methods page for details.
- Analysis method: The video can be up to 2 hours long, or up to 4 hours when you analyze only a portion of it. You can analyze between 1 second and 2 hours (asynchronous approach). For videos under 1 hour, see the synchronous approach below.
- Model capabilities: See the complete requirements for resolution, aspect ratio, and supported formats.

Complete example

Copy and paste the code below, replacing the placeholders surrounded by <> with your values.

import time
from twelvelabs import TwelveLabs
from twelvelabs.types import VideoContext_AssetId, VideoContext_Url, VideoContext_Base64String, AnalyzePromptV2, SmeMediaSource
# 1. Initialize the client
client = TwelveLabs(api_key="<YOUR_API_KEY>")
# 2. Upload a video
asset = client.assets.create(
    method="url",
    url="<YOUR_VIDEO_URL>" # Use direct links to raw media files. Video hosting platforms and cloud storage sharing links are not supported
    # Or use method="direct" and file=open("<PATH_TO_VIDEO_FILE>", "rb") to upload a local file up to 200 MB
)
print(f"Created asset: id={asset.id}")
# 3. Check the status of the asset
print("Waiting for asset to be ready...")
while True:
    asset = client.assets.retrieve(asset.id)
    if asset.status == "ready":
        print("Asset is ready")
        break
    if asset.status == "failed":
        raise RuntimeError(f"Asset processing failed: id={asset.id}")
    time.sleep(5)
# 4. Analyze your video
video = VideoContext_AssetId(asset_id=asset.id)
# Or instead of creating an asset, pass video inline:
# video = VideoContext_Url(url="<YOUR_VIDEO_URL>")
# video = VideoContext_Base64String(base_64_string="<YOUR_BASE64_DATA>")
task = client.analyze_async.tasks.create(
    model_name="pegasus1.6",
    video=video,
    prompt_v_2=AnalyzePromptV2(
        input_text="<YOUR_PROMPT>",  # To use reference images: "Is there a <@product> in this video?"
        # media_sources=[
        #     SmeMediaSource(name="product", media_type="image", url="<YOUR_IMAGE_URL>"),
        # ],
    ),
    # temperature=0.2,
    # max_tokens=1024,  # Or "unlimited" to segment long videos without a token limit
    # analysis_mode="time_based_metadata",  # For video segmentation with in-segment events, also pass `response_format` (see the Segment videos guide)
    # You can also use `response_format` to request structured JSON responses
)
print(f"Task ID: {task.task_id}")
# 5. Monitor the status
while True:
    task = client.analyze_async.tasks.retrieve(task.task_id)
    if task.status == "ready":
        print("Task completed")
        break
    elif task.status in ("failed", "canceled"):
        message = task.error.message if task.error else f"Task ended with status: {task.status}"
        raise RuntimeError(message)
    else:
        print("Task still processing...")
        time.sleep(5)
# 6. Process the results
print(f"{task.result.data}")

Code explanation

Python

Node.js

Import the SDK and initialize the client

Create a client instance to interact with the TwelveLabs Video Understanding Platform.
Function call: You call the constructor of the TwelveLabs class.
Parameters:

api_key: The API key to authenticate your requests to the platform.

Return value: An object of type TwelveLabs configured for making API calls.

Upload a video

Upload a video to create an asset.
Function call: You call the assets.create function.
Parameters:

method: The upload method for your asset. Use url for a publicly accessible or direct to upload a local file. This example uses url.
url or file: The publicly accessible URL of your video or an opened file object in binary read mode. This example uses url.

Return value: An object of type Asset. This object contains, among other information, a field named id representing the unique identifier of your asset.

Note

For local files larger than 200 MB, use multipart uploads. Multipart uploads support automatic retry, progress tracking, parallel chunk uploads, and improved reliability, performance, and observability.

Check the status of the asset

Asset processing is asynchronous. Poll the status of the asset until it is ready before you use it.
Function call: You call the assets.retrieve function.
Parameters:

asset_id: The unique identifier of your asset.

Return value: An object of type Asset containing, among other information, a field named status representing the current status of the asset. Check this field until its value is ready.

Analyze your video

Create an analysis task to start processing your video. This operation is asynchronous.
Function call: You call the analyze_async.tasks.create method.
Parameters:

video: An object that specifies the source of the video. Provide one of the following:
- asset_id: The unique identifier of an asset from a previous upload.
- url: The publicly accessible URL of the video file.
- base_64_string: The base64-encoded video data.
This example uses the asset ID from the previous step.
prompt_v_2: A structured prompt. Set input_text to your prompt text. To include reference images, add entries to media_sources and use <@name> placeholders in input_text. See the commented lines in the code example above.
(Optional) temperature: Controls the randomness of the text output. A higher value generates more creative text, while a lower value produces more deterministic output.
(Optional) max_tokens: The maximum response length, in tokens. Set it to "unlimited" with analysis_mode set to "time_based_metadata" to segment long videos without a token limit. The platform returns the segments it produced, even when the output is incomplete.
(Optional) analysis_mode and response_format: Set analysis_mode to "time_based_metadata" to segment the video into structured, timestamped segments. Define a segment field with type set to "time_array" to extract a list of timestamped events inside each segment. Requires response_format.type set to "segment_definitions". For a full walkthrough, see the Segment videos page.
(Optional) response_format: Use this parameter to request structured JSON responses. For instructions, examples, and best practices, see the Structured responses page.

Return value: An object of type CreateAnalyzeTaskResponse containing a field named task_id, which represents the unique identifier of your analysis task. You can use this identifier to track the status of your task.

Monitor the status

The platform requires some time to process videos. Poll the status of the analysis task until processing completes. This example uses a loop to check the status every 5 seconds.
Function call: You repeatedly call the analyze_async.tasks.retrieve method until the task completes.

Parameters:

task_id: The unique identifier of your analysis task.

Return value: An object of type AnalyzeTaskResponse containing, among other information, the following fields:

status: The current status of the task. The possible values are:
- queued: The task is waiting to be processed.
- pending: The task is queued and waiting to start.
- processing: The platform is analyzing the video.
- ready: Processing is complete. Results are available in the result field.
- failed: The task failed.
- canceled: The task was canceled. No result is available.
result: When the status is ready, this field contains the generated text and usage information.

Process the results

This example prints the generated text to the standard output.

Short videos (synchronous)

For videos that are shorter than one hour, you can use a synchronous approach that returns results immediately without creating an analysis task.

Response methods

Streaming responses

Streaming responses deliver text fragments in real-time as they are generated, enabling immediate processing and feedback. This method is the default behavior of the platform and is ideal for applications requiring incremental updates.

Response format: A stream of JSON objects in NDJSON format, with three event types:
- stream_start: Marks the beginning of the stream.
- text_generation: Delivers a fragment of the generated text.
- stream_end: Signals the end of the stream.
Response handling:
- Iterate over the stream to process text fragments as they arrive.
Advantages:
- Real-time processing of partial results.
- Reduced perceived latency.
Use case: Live transcription, real-time analysis, or applications needing instant updates.

Non-streaming responses

Non-streaming responses deliver the complete generated text in a single response, simplifying processing when the full result is needed.

Response format: A single string containing the full generated text.
Response handling:
- Access the complete text directly from the response.
Advantages:
- Simplicity in handling the full result.
- Immediate access to the entire text.
Use case: Generating reports, summaries, or any scenario where the whole text is required at once.

Copy and paste the code below, replacing the placeholders surrounded by <> with your values.

Streaming responses

Non-streaming responses

import time
from twelvelabs import TwelveLabs
from twelvelabs.types import VideoContext_AssetId, AnalyzePromptV2, SmeMediaSource
# 1. Initialize the client
client = TwelveLabs(api_key="<YOUR_API_KEY>")
# 2. Upload a video
asset = client.assets.create(
    method="url",
    url="<YOUR_VIDEO_URL>" # Use direct links to raw media files. Video hosting platforms and cloud storage sharing links are not supported
    # Or use method="direct" and file=open("<PATH_TO_VIDEO_FILE>", "rb") to upload a local file up to 200 MB
)
print(f"Created asset: id={asset.id}")
# 3. Check the status of the asset
print("Waiting for asset to be ready...")
while True:
    asset = client.assets.retrieve(asset.id)
    if asset.status == "ready":
        print("Asset is ready")
        break
    if asset.status == "failed":
        raise RuntimeError(f"Asset processing failed: id={asset.id}")
    time.sleep(5)
# 4. Analyze your video
video = VideoContext_AssetId(asset_id=asset.id)
text_stream = client.analyze_stream(
    model_name="pegasus1.6",
    video=video,
    prompt_v_2=AnalyzePromptV2(
        input_text="<YOUR_PROMPT>",  # To use reference images: "Is there a <@product> in this video?"
        # media_sources=[
        #     SmeMediaSource(name="product", media_type="image", url="<YOUR_IMAGE_URL>"),
        # ],
    ),
    # temperature=0.2,
    # max_tokens=1024,
    # You can also use `response_format` to request structured JSON responses
)
# 5. Process the results
for text in text_stream:
    if text.event_type == "text_generation":
        print(text.text)

The video parameter and the optional model_name, temperature, and response_format parameters function the same as in the asynchronous approach above. This endpoint does not support video segmentation, so max_tokens only accepts the general mode range and does not accept unlimited.

Troubleshooting

Truncated responses

When the response reaches the maximum response length or the context window, the platform returns the partial output and sets finish_reason to "length". A warning appears in the error field. This can happen for two reasons:

The response reached the maximum response length. To fix this, increase the max_tokens value.
The combined input and response reached the context window. To fix this, reduce the input (shorter prompt, shorter video clip, fewer reference images) or lower the max_tokens value.

Check usage.input_tokens and usage.output_tokens in the response to determine which limit was reached. For truncation error messages, see Error codes.

Stay within the context window

Reduce input size:

Keep prompts focused, especially for longer videos.
Use start_time and end_time to analyze a portion of the video.
Reduce the number or size of reference images.

Control output size:

Set the max_tokens parameter to the length your application needs.
Request only the fields your application requires.

Split large tasks:

Break complex analysis into multiple requests.