For AI agents: a documentation index is available at the root level at /llms.txt and /llms-full.txt. Append /llms.txt to any URL for a page-level index, or .md for the markdown version of any page.
Pegasus is a generative model for video-to-text generation. Pegasus analyzes multiple modalities to generate contextually relevant text based on the content of your videos.
The current version is Pegasus 1.5, which supports both general analysis (prompt-based text generation) and video segmentation. Pegasus 1.2 remains available for general analysis only.
New in Pegasus 1.5
Pegasus 1.5 analyzes videos directly from a URL, asset, or base64 string, with no pre-indexing required. It introduces the following capabilities over Pegasus 1.2:
Video segmentation: Transform raw videos into structured, timestamped data. Define the types of segments you want to detect, such as editorial narratives, sports plays, speaker changes, or brand appearances, specify custom fields for each segment, and receive structured results in JSON format. For details, see the Segment videos page.
Multimodal prompting: Include reference images to provide visual context for your prompts or to help identify specific segments during video segmentation.
Video clipping: Analyze a specific portion of the video.
Per-definition time ranges: Restrict segment extraction to specific time windows within the video for finer control over video segmentation.
Context window: Pegasus 1.5 uses a shared context window of 261,120 tokens for input and output per request.
Longer responses: Pegasus 1.5 supports responses up to 98,304 tokens. Use max_tokens to control the response length.
Key features
Video-to-text generation: Creates detailed textual descriptions based on video content
Extended processing capacity: Processes videos up to 2 hours in length
Granular visual comprehension: Analyzes objects, on-screen text, and numerical content
Temporal grounding: Accurately identifies timestamps of specific events
Multimodal understanding: Combines visual, audio, and textual information for comprehensive analysis
Context window
Pegasus 1.5 uses a context window of 261,120 tokens. The context window is the maximum number of tokens a single request can use. This limit covers both the input and the response. Pegasus 1.2 enforces separate limits for each field: a 2,000-token prompt limit and a 4,096-token maximum response length.
The following inputs and outputs count toward the context window:
Video content
Audio transcription
Prompt text (the prompt or prompt_v2 parameter)
Reference images (in prompts or segment definitions)
JSON schema (if you request structured responses)
Segment definitions (for video segmentation)
Generated output (text or JSON)
Use cases
Content summarization: Generate concise summaries of video content
Detailed descriptions: Create comprehensive textual descriptions of visual scenes
Timestamp identification: Answer questions about when specific events occur in videos
Content analysis: Extract key information from video content for further processing
Image-guided analysis: Reference images in your prompt to ask about specific objects, people, or scenes in the video
Video segmentation: Detect and extract structured metadata for editorial segments, scene changes, sports plays, or brand appearances
Input requirements
The specifications on this page reflect the maximum capabilities of the model. Your actual requirements depend on the upload method and operation you choose. For details about the available upload methods and the corresponding limits, see the Upload and processing methods page.
Video file requirements
Duration: 4 sec to 2 hours
File size: ≤ 2 GB
Resolution: 360x360 to 5184x2160
Aspect ratio: Between 1:1 and 1:2.4, or between 2.4:1 and 1:1. For example, you can use 1:1, 4:3, 4:5, 5:4, 16:9, 9:16, or 17:9.
If you upload files using publicly accessible URLs, use direct links to raw video files that play without user interaction or custom video players (example: https://example.com/videos/sample-video.mp4). Video hosting platforms like YouTube and cloud storage sharing links are not supported.
For videos in other formats or if you require different options, contact us at support@twelvelabs.io.
Supported languages
Pegasus supports the following languages for processing visual and audio content, understanding prompts, and generating outputs:
This example prompt summarizes an educational video without any customization.
Summarize this video
Generating captions for social media
This example prompt generates a caption for a social media post.
Generate an attention-grabbing caption for a social media post. Keep it shorter than 200 characters.
Table of contents
This example prompt creates a table of contents detailing the main sections.
Provide a table of contents detailing the main sections of this video.
Company-wide memo
This example prompt generates a company-wide memo.
Generate a company-wide memo based on the content of this video.
Video annotations
This example prompt identifies and lists key visual elements, scene changes, and notable events, briefly describing each.
Identify and list key visual elements, scene changes, and notable events in the video, briefly describing each.
Video question answering
This example prompt identifies the key takeaways.
What are the key takeaways of this video?
Timestamp breakdown
This example prompt lists all timestamps in an advertisement where a close-up of the product appears.
Tell me all the timestamps in the advertisement where a closeup of the product appears.
Police report
This example prompt creates a police report using a specific template for a video showing a robbery.
Create a police report based on what happened in the video. Provide the exact time range where the suspect appears in the video
Using different languages
Spanish
This example prompt summarizes a video and indicates that the response should be in Spanish. Note that the prompt is in English, and the output is in Spanish.
Write a summary in Spanish.
French
This example prompt summarizes the three main takeaways of a video. Note that the prompt and the output are in French.
Résumez les trois principaux points à retenir de cette vidéo
