This quickstart guide provides a simplified introduction to segmenting videos into structured, timestamped data using the TwelveLabs Video Understanding Platform. It includes the following:

A basic working example
Minimal implementation details
Core parameters for common use cases

For a comprehensive guide, see the Segment videos guide.

Key concepts

This section explains the key concepts and terminology used in this guide:

Asset: Your uploaded content. Once created, you can reference the same asset across multiple operations without uploading the file again.
Segment definition: A description of a type of segment you want to extract. Each definition includes a unique identifier, a natural language description, and optional custom fields.
Segment field: A custom metadata field to extract for each segment. Each field has a name, a type, and a description.

Workflow

This guide shows how to upload your video as an asset, create an asynchronous segmentation task with Pegasus 1.5, and parse the timestamped metadata from the results.

Prerequisites

To use the platform, you need an API key:

1
If you don’t have an account, sign up for a free account.
2
Go to the API Keys page.
3
If you need to create a new key, select the Create API Key button. Enter a name and set the expiration period. The default is 12 months.
4
Select the Copy icon next to your key to copy it to your clipboard.
Depending on the programming language you are using, install the TwelveLabs SDK by entering one of the following commands:
```
pip install --upgrade twelvelabs
```
Your videos must meet the following requirements:
- Upload limits: Public video URLs up to 2 GB or local videos up to 200 MB. For local files up to 2 GB, see the Upload and processing methods page.
- Analysis method: The video can be up to 2 hours long, or up to 4 hours when you analyze only a portion of it. You can analyze between 1 second and 2 hours.
- Model capabilities: See the complete requirements for videos

Starter code

Copy and paste the code below, replacing the placeholders surrounded by <> with your values. The example defines a single segment type (scenes) with three custom fields to illustrate the shape. Adapt the segment definitions to match what you want to extract from your videos.

import json
import time
from twelvelabs import TwelveLabs
from twelvelabs.types import AsyncResponseFormat, VideoContext_AssetId
# 1. Initialize the client
client = TwelveLabs(api_key="<YOUR_API_KEY>")
# 2. Upload a video
asset = client.assets.create(
    method="url",
    url="<YOUR_VIDEO_URL>" # Use direct links to raw media files. Video hosting platforms and cloud storage sharing links are not supported
    # Or use method="direct" and file=open("<PATH_TO_VIDEO_FILE>", "rb") to upload a local file up to 200 MB
)
print(f"Created asset: id={asset.id}")
# 3. Check the status of the asset
print("Waiting for asset to be ready...")
while True:
    asset = client.assets.retrieve(asset.id)
    if asset.status == "ready":
        print("Asset is ready")
        break
    if asset.status == "failed":
        raise RuntimeError(f"Asset processing failed: id={asset.id}")
    time.sleep(5)
# 4. Create a video segmentation task
video = VideoContext_AssetId(asset_id=asset.id)
task = client.analyze_async.tasks.create(
    video=video,
    model_name="pegasus1.5",
    analysis_mode="time_based_metadata",
    response_format=AsyncResponseFormat(
        type="segment_definitions",
        segment_definitions=[
            {
                "id": "scenes",
                "description": "Segment the video into distinct scenes based on changes in setting, topic, or visual composition",
                "fields": [
                    {
                        "name": "sentiment",
                        "type": "string",
                        "description": "The overall sentiment of this scene",
                        "enum": ["positive", "negative", "neutral"]
                    },
                    {
                        "name": "key_objects",
                        "type": "array",
                        "description": "Notable objects visible in the scene",
                        "items": {"type": "string"}
                    },
                    {
                        "name": "contains_speech",
                        "type": "boolean",
                        "description": "Whether the scene contains speech or dialogue"
                    }
                ]
            }
        ]
    )
)
print(f"Task ID: {task.task_id}")
# 5. Monitor the status
while True:
    task = client.analyze_async.tasks.retrieve(task.task_id)
    if task.status == "ready":
        print("Task completed")
        break
    elif task.status == "failed":
        print("Task failed")
        break
    else:
        print("Task still processing...")
        time.sleep(5)
# 6. Parse and process the results
data = json.loads(task.result.data)
for segment in data["scenes"]:
    print(f"\n[{segment['start_time']:.1f}s - {segment['end_time']:.1f}s]")
    meta = segment["metadata"]
    print(f"  Sentiment: {meta['sentiment']}")
    print(f"  Key objects: {', '.join(meta['key_objects'])}")
    print(f"  Contains speech: {meta['contains_speech']}")

Code explanation

Import the SDK and initialize the client

Create a client instance to interact with the TwelveLabs Video Understanding Platform.

Upload a video

Upload a video to create an asset.

Check the status of the asset

Asset processing is asynchronous. Poll the status of the asset until it is ready before you use it.

Create a video segmentation task

Define the segments and fields you want to extract. Set the analysis_mode parameter to time_based_metadata and pass your segment definitions in the response_format parameter.

Monitor the status

Poll the task until it reaches the ready state.

Parse and process the results

The result.data field is a JSON-encoded string. This example parses it, then displays the timestamps and custom metadata for each segment to the standard output.