The sections below list all new features, enhancements, and changes to the platform, in chronological order. All dates and times are in the PT timezone, unless stated otherwise.

Version 1.3

May 22, 2026

User-defined metadata on asset upload

You can now attach user-defined metadata to your assets during upload. Previously, you could only add metadata after indexing a video. With this update, the following methods accept the user_metadata parameter:

• Create an asset
• Create a multipart upload session
• Index an asset

This release also introduces two new methods for managing user-defined metadata on existing assets:

• Update the user-defined metadata of an asset
• Delete the user-defined metadata of an asset

This feature is accessible through the REST API. SDK support is not yet available.

Timestamp formatting for analysis responses

You can now choose the output format for timestamps in both general analysis (prompt-based text generation) and video segmentation. This feature requires Pegasus 1.5 and is accessible through the REST API. SDK support is not yet available.

Timestamp formatting in structured responses

Declare a field as {"type": "timestamp", "format": "<format>"} to control the format of the returned value. Available formats: seconds, hh:mm:ss, and hh:mm:ss.fff.

For complete parameter specifications, see the following parameters in the API Reference section:

• Sync analysis: json_schema.
• Async analysis: json_schema and fields.

Format control for segment boundaries

Set the segment_time_format parameter to control the output format for the automatic start_time and end_time boundaries on each returned segment. Available formats: seconds, hh:mm:ss, and hh:mm:ss.fff.

For complete parameter specifications, see the segment_time_format property in the API Reference section.

May 7, 2026

Combined video hours limit on the Free plan

The Free plan now includes a single 10-hour limit shared across indexing and video analysis. Previously, only indexing hours counted toward this limit. You can split the 10 hours however you choose between indexing and analysis.

For details, see the Frequently asked questions page.

Video segmentation pricing update for paid plans

On paid plans, video segmentation pricing now factors in the number of segment definitions per request. The cost equals the billable video duration multiplied by the number of segment definitions. If you provide the start_time and end_time parameters, the billable duration covers that time range only. Otherwise, it covers the full video duration.

On the Free plan, the number of segment definitions does not affect your limit.

For details and examples, see the Frequently asked questions page.

SDK support for Pegasus 1.5

All Pegasus 1.5 features are now available in the Python and Node.js SDKs. Upgrade to the latest version:

1
pip install --upgrade twelvelabs

Breaking changes

The following change requires updates to your code when you upgrade the SDK:

• ResponseFormat type split: The ResponseFormat type has been split into SyncResponseFormat (for sync analysis) and AsyncResponseFormat (for async analysis). Update your imports and type references. See the Migration guide for details.

New features

The following features are opt-in. Your existing code continues to work without changes.

• Video segmentation: Extract structured, timestamped metadata with custom segment definitions. See the Segment videos guide.
• Structured prompts with reference images: Use the prompt_v2 parameter to include up to 4 reference images in your prompt.
• Video clipping: Use the start_time and end_time parameters to analyze a specific portion of the video.
• Per-definition time ranges for segmentation: Restrict segment extraction to specific time windows within the video.

Enhancements

The following improvements take effect automatically. No code changes required.

• Analysis task response echo: The request_params object in task responses now includes the full set of parameters you submitted when creating the task.
• Extended token limits: Generate responses up to 65,536 tokens with Pegasus 1.5.
• Custom task identifier: Attach a custom identifier to async analysis tasks to correlate them across responses.

For migration instructions, see the Migration guide. For details about each feature, see the original announcements on April 20, April 27, and May 7.

May 6, 2026

Pegasus 1.5 now supports synchronous analysis

You can now use Pegasus 1.5 with the synchronous analysis endpoint. Set the model_name parameter to pegasus1.5 when calling the POST method of the /analyze endpoint to analyze videos directly from a URL, asset, or base64 string and receive results in the response.

The following Pegasus 1.5 capabilities, previously available only through async analysis, are now available on the synchronous analysis endpoint:

• Structured prompts with reference images: Use the prompt_v2 parameter to include up to 4 reference images in your prompt.
• Video clipping: Use the start_time and end_time parameters to analyze a specific portion of the video. The clip must be at least 4 seconds long.

For complete parameter specifications, see the Sync analysis page in the API Reference section.

April 27, 2026

TwelveLabs is proud to introduce general analysis (prompt-based text generation) for Pegasus 1.5, along with the following new capabilities.

General analysis for Pegasus 1.5

Set model_name to pegasus1.5 and analysis_mode to general to analyze videos with a prompt. Pegasus 1.5 accepts videos directly from a URL, an asset, or a base64 string, with no pre-indexing required.

Structured prompts with reference images

Use the new prompt_v2 field to include up to 4 reference images in your prompt. Assign a name to each image and insert <@name> placeholders in the prompt text (Example: "Is there a <@tiger-1> in the video?"). Requires model_name set to pegasus1.5.

Video clipping

Use the start_time and end_time fields to analyze only a portion of the video. The clip must be at least 4 seconds long. Requires model_name set to pegasus1.5.

Per-definition time ranges for segmentation

When using video segmentation (analysis_mode set to time_based_metadata), add the time_ranges to individual segment definitions to restrict segment extraction to specific parts of the video. This gives you finer control over which portions are analyzed for each definition. Requires model_name set to pegasus1.5.

Analysis task response echo

The request_params object in task responses now includes the full set of parameters you submitted when creating the task: prompt, prompt_v2, json_schema, segment_definitions, time_ranges, start_time, and end_time. The list endpoint returns compact versions (truncated prompts, omitted schemas), while the retrieve endpoint returns the full data.

For complete parameter specifications, see the Create an async analysis task and Retrieve analysis task status and results pages in the API Reference section.

April 24, 2026

Track async analysis tasks with a custom identifier

You can now attach a custom identifier to each async analysis task. Set the custom_id field when you call the POST method of the /analyze/tasks endpoint, and the platform returns it unchanged in the following responses:

• The GET method of the /analyze/tasks/{task_id} endpoint
• The GET method of the /analyze/tasks endpoint
• The analyze.task.ready and analyze.task.failed webhook payloads

Use this field to correlate tasks, for example, to distinguish tasks by type or environment.

For complete parameter specifications, see the Create an async analysis task page in the API Reference section.

April 20, 2026

Introducing Pegasus 1.5 with video segmentation

Pegasus 1.5, a new video understanding model, transforms raw videos into structured, timestamped data. Define the types of segments you want to detect, such as editorial segments, sports plays, speaker changes, or brand appearances, specify custom fields for each segment, and receive structured results in JSON format.

Instead of manually reviewing footage or relying on brittle heuristics, you can define what matters to your application and automatically extract these events across entire video libraries. For general analysis (prompt-based text generation) support, see the April 27 announcement.

Video segmentation is available through the asynchronous analysis endpoint. To use this feature, upgrade to the latest version of the Python or Node.js SDK:

1
pip install --upgrade twelvelabs

SDK type changes (breaking change)

The ResponseFormat type has been split into two separate types:

1
# Before
2
from twelvelabs.types import ResponseFormat
3
4
# After — for async analysis / video segmentation
5
from twelvelabs.types import AsyncResponseFormat
6
7
# After — for sync analysis
8
from twelvelabs.types import SyncResponseFormat

For a how-to guide, see the Segment videos page. For complete parameter specifications, see the Create an async analysis task page in the API Reference section.

April 15, 2026

Introducing asset management

This release introduces new asset management capabilities. It also includes the first set of changes for the deletion safeguards announced on April 8.

HLS streaming and thumbnail generation

You can now request HLS playlists and thumbnail images when creating an asset. Set the enable_hls or enable_thumbnail field to true in your request to the POST method of the /assets endpoint or the POST method of the /assets/multipart-uploads endpoint. The platform generates these in the background. To check progress, call the GET method of the /assets/{asset_id} endpoint and inspect the hls.status and thumbnail.status fields.

New response fields

The following response schemas now include new fields:

• Asset: The size field contains the file size in bytes. The duration field contains the length in seconds for video and audio assets. The method field can now also return multipart for assets uploaded through the multipart upload flow.
• Indexed asset: The asset_id field identifies the source asset, so you can trace an indexed asset back to the asset it was created from.
• Entity: The entity_collection_id field identifies the collection the entity belongs to.

Filename filtering

The GET method of the /assets endpoint now accepts the filename query parameter to filter assets by filename. The match is case-insensitive and supports partial matches.

Look up indexed assets and entities by asset

As announced on April 8, the following are now available:

• The GET method of the /assets/{asset_id}/indexed-assets endpoint lists the indexed assets that reference a given asset.
• The GET method of the /assets/{asset_id}/entities endpoint lists the entities that reference a given asset.
• The force query parameter on the DELETE method of the /assets/{asset_id} endpoint lets you delete the asset even if indexed assets reference it.

On April 26, 2026, delete requests for referenced assets will be denied by default. See the April 8 announcement for required actions.

April 8, 2026

Deleting referenced assets will be denied by default (Effective April 26, 2026)

Currently, you can delete any asset, even if an indexed asset references it. Starting on April 26, 2026, the platform will deny delete requests for referenced assets by default, returning a 409 Conflict error.

These changes will roll out in two phases:

• April 15, 2026: The following become available, and you can begin updating your deletion workflow:
  • The GET method of the /assets/{asset_id}/indexed-assets endpoint
  • The force query parameter on the DELETE method of the /assets/{asset_id} endpoint
• April 26, 2026 (breaking change): Delete requests for referenced assets are denied by default.

Actions required

Complete these steps before April 26, 2026:

Support

For migration assistance, contact our support team at support@twelvelabs.io.

March 31, 2026

Introducing async analysis endpoints for longer videos

You can now analyze videos up to 2 hours using the new asynchronous analysis endpoints. Previously, the Analyze API only supported synchronous analysis for videos up to 1 hour long.

When to use async analysis:

• Videos longer than one hour (up to 2 hours)
• Background processing without blocking your application

When to use sync analysis:

• Videos up to one hour
• Immediate results or real-time streaming

For a how-to guide, see the Analyze videos page. For complete parameter specifications, see the API reference for async analysis or the SDK reference for Python and Node.js.

March 30, 2026

On March 30th, 2026 (7PM PT), Marengo 2.7 has been sunset. You no longer can index new content, perform search requests, or retrieve any embeddings from previously indexed content.

Update your application to ensure continuous service.

March 11, 2026

TwelveLabs is proud to introduce multiple-image search, multiple-image embeddings, and fused embeddings. To use these features, upgrade to version 1.2.1 or later of the Python or Node.js SDK:

1
pip install twelvelabs==1.2.1

Multiple-image search

You can now use up to 10 images, alone or combined with a text query, in a single search request.

For a how-to guide, see the Search page. For complete parameter specifications, see the Make any-to-video search requests page in the API Reference section.

Multiple-image embeddings

The Embed API v2 now creates a single embedding from up to 10 images. You can also include text for context. To reference a specific image in your text, assign a name to that image and use a placeholder in the format <@name>. For example: “A person wearing <@outfit> and holding <@accessory>.”

For a how-to guide, see the Text and image embeddings page. For complete parameter specifications, see the Create sync embeddings page in the API Reference section.

Fused embeddings

The Embed API v2 now lets you control how the platform returns embeddings for audio and video content. You can request separate embeddings per modality, a single combined embedding that integrates all modalities, or both in the same response.

For how-to guides, see the Audio embeddings and Video embeddings pages. For complete parameter specifications, see the embedding_type parameter in the API Reference section:

• Audio embeddings
• Video embeddings

February 28, 2026

Marengo 2.7 will be deprecated

On March 30th, 2026 (7PM PT), Marengo 2.7 has been sunset. You no longer can index new content, perform search requests, or retrieve any embeddings from previously indexed content.

Update your application to ensure continuous service.

Actions required

• Search operations: Update your application logic for Marengo 3.0 compatibility. Deprecated parameters are now ignored, and audio processing has changed. Your code will not return errors, but results may differ.
• Embeddings: Regenerate all embeddings. Marengo 2.7 embeddings are not compatible with Marengo 3.0.

Support

For migration assistance, contact our support team at support@twelvelabs.io.

February 15, 2026

Predefined formats for video analysis have been sunset and removed

The /gist and /summarize endpoints, which generate text based on predefined formats, have been sunset and removed on February 15, 2026. Instead, use the /analyze endpoint, which provides structured JSON responses.

If you have not migrated yet, follow the migration steps in the January 7, 2026 entry below.

January 26, 2026

Introducing custom API key expiration periods

API keys previously expired after 90 days. You can now choose from 3 months, 6 months, 12 months (default), a custom date, or never expire.

This change applies only to new API keys created after January 26, 2026. The expiration dates for existing API keys remain unchanged.

January 7, 2026

Predefined formats for video analysis will be sunset and removed

Update: This deprecation was completed on February 15, 2026. See the February 15, 2026 entry for details.

The /gist and /summarize endpoints, which generate text based on predefined formats, will be sunset and removed on February 15, 2026. Instead, use the /analyze endpoint, which provides structured JSON responses.

This consolidation provides the same capabilities with the following benefits:

• Reduced API surface area: A single endpoint simplifies integration and maintenance
• Fully customizable schemas: Define custom fields, naming conventions, and schema variations to match your requirements
• Improved chapterization quality: Significantly better chapter detection and segmentation
• Better adherence to rules: Enhanced accuracy when applying duration-based chapter rules
• More stable output: Predictable, consistent responses across requests

Actions required

Complete these migration steps before February 15, 2026:

1
import json
2
from twelvelabs import TwelveLabs
3
from twelvelabs.types import ResponseFormat
4
5
client = TwelveLabs(api_key="<YOUR_API_KEY>")
6
7
text = client.analyze(
8
    video_id="<YOUR_VIDEO_ID>",
9
    prompt="Provide the title of this video",
10
    response_format=ResponseFormat(
11
        type="json_schema",
12
        json_schema={
13
            "type": "object",
14
            "properties": {
15
                "title": {"type": "string"}
16
            },
17
            "required": ["title"],
18
        },
19
    ),
20
)
21
22
data = json.loads(text.data) if text.data else {}
23
print(f"Title: {data.get('title', 'N/A')}")

Support

For migration assistance, contact our support team at support@twelvelabs.io.

December 12, 2025

Enhanced rate limits (Effective January 12, 2026)

TwelveLabs is introducing an enhanced rate-limiting system that provides better visibility and more granular control over your usage:

• Multi-dimensional rate limiting: Rate limits will measure usage across multiple dimensions:
  • Request count
  • Video and audio duration processed
  • Token usage for text generation
• Tiered Developer plan: The Developer plan will include three tiers that automatically upgrade based on monthly spending.
• Modality-based limits: Separate rate limits for video, audio, image, and text processing.
• Enhanced response headers: New multi-dimensional headers will show your remaining capacity for each dimension.
• Automatic tier management: The platform will automatically upgrade your tier when you reach spending thresholds. Downgrades include a one-month grace period.

Actions required

December 5, 2025

Pegasus 1.2 is now available in 23 new AWS regions via global cross-region inference

Amazon Bedrock introduces global cross-region inference for Pegasus 1.2, expanding the model’s availability to 23 additional regions. You can now also access the model in all EU regions using geographic cross-region inference.

Use geographic cross-region inference for data residency or compliance requirements. Use global cross-region inference for high availability, performance, and cost efficiency.

For a complete list of supported regions and detailed information about cross-region inference, see the Analyze videos page.

November 17, 2025

Marengo 3.0 is now generally available

TwelveLabs is proud to announce that Marengo 3.0 is now generally available through the first-party API. This version improves accuracy and performance in the following areas:

• Composed text and image search: Combine text descriptions with images in a single search query for more precise results.
• Improved cinematography understanding: Enhanced search performance for cinematography terms like zoom, pan, and tracking shot.
• Sports intelligence: Improved recognition of soccer and basketball actions. Support for baseball, ice hockey, and American football.
• Faster indexing: Significant performance improvement with the new indexing technology.
• Extended text processing: Maximum text length increased from 77 to 500 tokens for both search queries and text embeddings.
• Optimized embeddings: 512-dimensional embeddings for faster processing and reduced storage.
• Long content support: Process up to four hours of video and audio content while maintaining context.
• Expanded language support: Query videos in 36 languages plus English (up from 12 plus English).

This release includes the following additional features:

• Entity search: Find specific people performing actions in your videos, such as locating players in sports footage, tracking characters in entertainment content, and analyzing individuals in surveillance videos.

  For details, see the Entity search page.

• Composed text and image search: Search using text descriptions and images in a single query.

  For concepts, see the Composed text and image queries page. For a how-to guide, see the Search page.

• New methods for uploading content: The platform now provides two new methods for uploading your videos, images, and audio files. These methods create reusable assets that you can use across different workflows, including search, analysis, and embedding creation.

  • Direct uploads: Upload whole files without splitting them. Use this method for a simple upload process. Direct uploads support files up to 4 GB.
  • Multipart uploads: Upload large local files by dividing them into smaller chunks. This method enables reliable uploads of large files and supports parallel processing and resuming interrupted transfers. TwelveLabs recommends this method for local files larger than 200 MB, with a maximum file size of 4 GB.

  For details, see the Upload content page.

• Embed API v2: Create embeddings for text, images, audio, and video content using the new Embed API v2.This API provides two endpoints:

  • Synchronous endpoint: Use for text, images, or audio and video content under 10 minutes.
  • Asynchronous endpoint: Use for audio and video content up to 4 hours.

  For details, see the Create embeddings v2 page.

October 30, 2025

Pegasus 1.2 expands to three additional AWS regions

Support for Pegasus 1.2 now includes US East (Ohio), US West (N. California), and Europe (Frankfurt), bringing the total number of supported regions to seven.

Deploy your applications in regions near your data and users to improve response times and streamline your infrastructure setup.

Regional availability: Pegasus 1.2 is now available in the following regions: US East (N. Virginia), US West (Oregon), US East (Ohio), US West (N. California), Europe (Ireland), Europe (Frankfurt), Asia Pacific (Seoul)

For details on using the model, see the Analyze videos page.

October 29, 2025

Marengo 3.0 is now available in Amazon Bedrock

Amazon Bedrock now supports the Marengo 3.0 video understanding model, which unifies videos, images, audio, and text into a single representation space. Use it to build any-to-any search, recommendation systems, and content analysis applications.

Marengo 3.0 introduces the following enhancements:

• Extended content processing: Process up to 4 hours of video or audio content and files up to 6 GB. Ideal for analyzing sporting events, training videos, and film productions.
• Enhanced sports analysis: Improved understanding of gameplay dynamics, player movements, and event detection. Better recognition of soccer and basketball actions with new support for baseball, ice hockey, and American football.
• Global multilingual support: Language support expanded from 12 to 36 languages (plus English), enabling unified search across diverse regions and markets.
• Multimodal search precision: Combine images and text in a single embedding request to merge visual similarity with semantic understanding for more accurate results.
• Expanded text processing: Maximum text length increased from 77 to 500 tokens for text embeddings. Supports a more detailed and richer context.
• Optimized embeddings: 512-dimensional embeddings deliver faster processing and reduced storage requirements.

Regional availability: US East (N. Virginia), Europe (Ireland), and Asia Pacific (Seoul)

To get started, visit the Amazon Bedrock console page.

For details on pricing, see the Amazon Bedrock pricing page.

October 14, 2025

TwelveLabs is proud to announce that Marengo 3.0 R2 is now available with the following enhancements:

Composed text and image search

Search using text descriptions and images in a single query. This feature allows you to specify both visual references and textual context for more precise results.

For concepts, see the Composed text and image queries page. For a how-to guide, see the Search page. For complete parameter specifications, see the Make any-to-video search requests page.

Improved cinematography understanding

The platform now provides enhanced search performance for cinematography terms like zoom, pan, and tracking shot.

Enhanced control over spoken word searches

Search spoken words independently from other audio using the new transcription value in the search_options parameter.

The transcription_options parameter offers two matching methods:

• Lexical matching: Finds exact words and phrases (ideal for product names and technical terms)
• Semantic matching: Finds concepts expressed differently (ideal for general topics)
• Combined: Uses both methods for comprehensive results

For more details and example use cases, see the Transcription options section. For complete parameter specifications, see the Make any-to-video search requests page.

Expanded language support for video search

The Marengo 3.0 video understanding model now supports 36 languages, in addition to English, for video search operations (up from 12 languages). Language support applies to your queries, regardless of the language used in the content.

• Find visual content: Query in any language to locate objects, actions, on-screen text, and brand logos.
• Find sounds and music: Query in any language to locate music, sound effects, and ambient audio.
• Find spoken words: Query in any language to locate dialogue and conversations. Use semantic search to query in one language and find content in another. Use a lexical search to find exact phrases in the same language as the content.

Supported languages: Arabic, Bengali, Chinese (Simplified), Croatian, Cusco, Czech, Danish, Dutch, English, Farsi, Filipino, Finnish, French, German, Greek, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Maori, Norwegian, Polish, Portuguese, Romanian, Russian, Spanish, Swahili, Swedish, Telugu, Thai, Turkish, Ukrainian, and Vietnamese.

Enhanced sports intelligence

The Marengo 3.0 video understanding model enhances the recognition of soccer and basketball actions, and provides support for baseball, ice hockey, and American football.

Extended audio processing

Process up to four hours of audio content while maintaining context.

September 17, 2025

The cloud-to-cloud integrations feature will be deprecated

After October 31, 2025, the cloud-to-cloud integrations feature will be deprecated, and files in your S3 buckets will no longer be uploaded to the platform automatically.

Actions required

This change affects your file processing workflow. Complete the migration steps before October 31, 2025.

1
task = client.tasks.create(index_id="<YOUR_INDEX_ID>", video_url="<YOUR_VIDEO_URL>")

Support

For migration assistance, contact our support team at support@twelvelabs.io.

September 16, 2025

Introducing structured JSON responses for video analysis

You can now request structured JSON responses when analyzing video content to generate open-ended text and summaries. This feature allows you to receive predictable, machine-readable outputs.

To learn how to use structured responses effectively with examples and best practices, refer to the Structured responses guide.

For complete parameter specifications, see the Sync analysis and Summaries, chapters, and highlights pages in the API Reference section.

September 12, 2025

TwelveLabs is proud to introduce the following new features and improvements:

Marengo 3 video understanding model

• Sports intelligence: Improved recognition of soccer and basketball actions.
• Faster indexing: Significant performance improvement with the new indexing technology.
• Extended text processing: Maximum text length increased from 77 to 500 tokens for both search queries and text embeddings.
• Optimized embeddings: 512-dimensional embeddings for faster processing and reduced storage.
• Long content support: Process up to four hours of video content while maintaining context.

API Changes:

When using Marengo 3, this update affects the following existing endpoints:

• POST /embed

  • The text_truncate parameter is deprecated and the maximum length for text embeddings is now 500 tokens.

• GET /indexes/:index-id/videos/:video-id and GET /embed/tasks/:task_id

  • The embedding_option parameter can now take the following values: visual, audio, transcription. For details, see the Embedding options section.

• POST /search

  • The audio search option now excludes human speech. A new search option named transcription for finding specific spoken words and phrases will be available in a future release.
  • The maximum query length is now 500 tokens.
  • The platform no longer returns the score and confidence fields. Use the new rank field instead to determine the relevance of a result.

Entity Search API

The Entity Search API enables you to search for specific people performing actions. Use cases include finding players in sports footage, tracking characters in entertainment content, and analyzing individuals in surveillance videos.

To get started, see the Entity Search page.

September 9, 2025

Amazon Bedrock adds synchronous inference for Marengo 2.7

Amazon Bedrock now supports synchronous inference for Marengo 2.7. You can now retrieve low-latency text and image embeddings directly in the API response.

This update enables you to build more responsive and interactive search and retrieval experiences. The synchronous inference maintains the same video understanding capabilities that Marengo 2.7 provides.

To generate embeddings from video, audio, and large-scale image files, continue using asynchronous inference for optimal performance.

Regional availability

Marengo 2.7 with synchronous inference is available in these regions:

• US East (N. Virginia)
• Europe (Ireland)
• Asia Pacific (Seoul)

Next steps

Review the Create embeddings guide to get started.

August 28, 2025

The PUT method of the /indexes/:index-id/videos/:video-id endpoint has been deprecated

The PUT method is now deprecated. Use the PATCH method instead, which provides identical functionality.

August 5, 2025

Version 1.0.0 of the Python and Node.js SDKs is now the stable version

TwelveLabs has released version 1.0.0 of the Python and Node.js SDKs as the stable version. You can still use versions 0.4.x and earlier, but they are no longer actively maintained.

Action required

Upgrade to version 1.0.0 for continued support and updates.

July 18, 2025

The pre-release version 1.0.0 of the Python and Node.js SDKs is now available

TwelveLabs is proud to introduce redesigned SDKs that incorporate modern development practices to enhance the developer experience.

Key Improvements

• Enhanced type safety and development experience: Complete typing support, full type hints, auto-completion, and Pydantic v1/v2 compatibility for superior IDE integration, safer code, and modern tooling.
• Auto-generated architecture: Built directly from the OpenAPI specifications, ensuring API consistency and automatic updates with new features.
• Improved pagination: Automatic page fetching, allowing easier handling of large datasets.
• Native async support: The Python SDK now fully supports async/await with a dedicated asynchronous client.
• Improved documentation: Detailed docstrings/JSDoc comments and thorough SDK Reference sections that outline all parameters and response fields.

Actions required

The pre-release version includes breaking changes from the stable 0.4.x release. Before you use version 1.0.0 in your application:

• Update your code to work with the new version
• Test all changes thoroughly
• Verify that everything functions as expected before you deploy to production.

Version support

TwelveLabs will stop maintaining versions up to 0.4.x after the stable 1.0.0 version is officially released.

Documentation

The documentation has been updated with current examples, detailed guides, and comprehensive SDK Reference sections.

July 16, 2025

Marengo 2.7 and Pegasus 1.2 are now available in Amazon Bedrock

Amazon Bedrock now supports the Marengo 2.7 and Pegasus 1.2 video understanding models.

Regional availability

• Marengo 2.7: Available in US East (N. Virginia), Europe (Ireland), and Asia Pacific (Seoul)
• Pegasus 1.2: Available in US West (Oregon) and Europe (Ireland) through cross-region inference

For details on accessing these models, see the Amazon Bedrock page.

For details on pricing, see the Amazon Bedrock pricing page.

July 11, 2025

Enhanced user metadata capabilities

The platform now provides advanced capabilities for managing user metadata, offering improved flexibility and precision:

• Upload videos with metadata: When you upload a video, you can now include user-defined metadata in the request body. This metadata will be attached to the video, helping you categorize it effectively.

• Update or delete metadata fields: After a video has been uploaded to the platform, you can update specific metadata fields, such as the file name, or delete them by setting the value to null.

• Controls metadata in search results: When you make search requests, you can specify whether user-defined metadata appears in the search results.

For more details on these enhancements, refer to the updated documentation:

• Create a video indexing task
• Partial update video information
• Make any-to-video search requests
• Retrieve a specific page of search results.

June 24, 2025

Organizations now support self-service creation and are available to all users

The Organizations feature has been improved to offer better accessibility and more control. This update includes several key changes:

• Self-service creation: You can now create and manage your organization directly from the Playground.
• Available to all users: The Organizations feature is no longer restricted to paid plans.
• Enhanced administrative features: Administrators now have the ability to upgrade or downgrade their organization’s plan.

For detailed instructions on setting up and managing your organization, see the updated Organizations section. If you previously had an organization set up through our sales team, your existing organization remains unchanged.

June 13, 2025

Marengo 2.6 and API v1.2 have been deprecated

The Marengo video understanding model version 2.6 has been deprecated. To ensure your applications remain compatible, please update them to use Marengo version 2.7.

Additionally, the API version 1.2 has also been deprecated. You must update your applications to utilize API version 1.3.

June 4, 2025

The Generate API has been renamed to the Analyze API

The Generate API has been renamed to the Analyze API to more accurately reflect its purpose of analyzing videos to generate text. This update includes changes to specific API endpoints and SDK methods, outlined below. You can continue using the Generate API until July 30, 2025. After this date, the Generate API will be deprecated, and you must transition to the Analyze API.

Changes to the API:

• The /generate endpoint is now the /analyze endpoint.
• The /gist endpoint remains unchanged.
• The /summarize endpoint remains unchanged.

Changes to SDKs:

The generate prefix has been removed from method names, and the methods below have been renamed as follows:

• generate.gist is now gist
• generate.summarize is now summarize
• generate.text is now analyze
• generate.text_stream is now analyze_stream (Python)
• generate.textStream is now analyzeStream (Node.js)

To maintain compatibility, update your API calls and SDK methods to the new names before July 30, 2025. For additional details, refer to the following resources:

• API Reference
• Python SDK Reference
• Node.js SDK Reference

May 16, 2025

Increased prompt length

The maximum prompt length for the /analyze and /summarize endpoints is now 2,000 tokens.

May 9, 2025

Support for retrieving transcriptions

The platform now allows you to retrieve transcriptions for your videos. This update adds new functionality without breaking existing code and affects the following endpoints:

• /indexes/:index-id/videos/:video-id: Set the new transcription query parameter to true to retrieve transcriptions. The platform will include a field named transcription in the response. It’s an array of objects, each consisting of the time range and spoken words for that segment. For more details, see the Retrieve video information page.
• /search: The response now includes a field named transcription for each match found. It’s a string that contains the transcription of the spoken words in the video. For more details, see the Make any-to-any search requests page.

April 15, 2025

New embedding retrieval options

The Embed API now provides more control over the types of embeddings you retrieve. This is a breaking change that requires updating your code and affects the following endpoints:

• /indexes/:index-id/videos/:video-id: The embed parameter has been deprecated and replaced with the new embedding_option parameter, allowing you to retrieve specific types of embeddings. For details, see the Retrieve video information page.
• /embed/tasks/:task_id: You can now use the optional embedding_option parameter to specify which types of embeddings to retrieve. For details, see the Retrieve video embeddings page.

In the responses from both endpoints, each segment now includes a new field named embedding_option located at video_embedding.segments[].embedding_option, which identifies the type of embedding as either “visual-text” or “audio.”

March 18, 2025

User-provided transcriptions have been deprecated

The POST method of the /tasks endpoint no longer processes transcription data submitted through the following parameters:

• provide_transcription
• transcription_file
• transcription_url

Note that the platform will not return an error if you include these parameters in a request.

March 3, 2025

Introducing the Organizations feature

TwelveLabs is excited to announce the launch of the Organizations feature, available exclusively for Enterprise customers. This feature enables the sharing of indexes, videos, and S3 integrations across the organization.

Enterprise customers interested in setting up an organization can do so by contacting our sales team at sales@twelvelabs.io. Once the organization is configured, administrators will be able to invite team members and start collaborating.

For more details about this feature, see the Organizations page.

February 11, 2025

Pegasus 1.2 has been released

TwelveLabs announces the official release of the Pegasus 1.2 video understanding model. For details on the new features and improvements in this version, refer to this blog post: Introducing Pegasus 1.2: An Industry-Grade Video Language Model for Scalable Applications.

Note that you can no longer use Pegasus 1.1 to create new indexes, and this version will be discontinued on February 25, 2025. All existing Pegasus 1.1 indexes will automatically be upgraded to Pegasus 1.2 on a rolling basis. No manual intervention is required for this migration process, and all indexes will utilize Pegasus 1.2 upon completion.

January 13, 2025

Pegasus 1.2 public preview

TwelveLabs announces the public preview release of Pegasus 1.2, our latest video understanding model.

Key improvements: The new model offers significant improvements over Pegasus 1.1:

• Extended video processing capacity from 30 minutes to 1 hour per video.
• Enhanced performance across video-language tasks compared to Pegasus 1.1 and other models of the same size.
• More granular visual comprehension of objects, on-screen text, and numerical content.
• More accurate temporal grounding and timestamp identification. For example, you can ask questions about the timestamps of certain events.

During the preview phase, the model is available only for new and sample indexes. Existing Pegasus 1.1 indexes remain fully supported. All current indexes will be automatically migrated to Pegasus 1.2 at no cost during the official release (date to be announced).

Note that the model may produce occasional errors or hallucinations. For support or feedback, contact support@twelvelabs.io.

December 2, 2024

TwelveLabs is proud to introduce the following new features and improvements:

• Marengo 2.7: This new version of the Marengo video understanding engine improves accuracy and performance in the following areas:
  • Multimodal processing that combines visual, audio, and text elements.
  • Fine-grained image-to-video search: detect brand logos, text, and small objects (as small as 10% of the video frame).
  • Improvement in motion search capability.
  • Counting capabilities.
  • More nuanced audio comprehension: music, lyrics, sound, and silence. For more details on the new features and improvements in this version, refer to this blog post: Introducing Marengo 2.7: Pioneering Multi-Vector Embeddings for Advanced Video Understanding.
• Simplified modalities:
  • visual: includes objects, actions, text OCR, logos.
  • audio: includes speech, music, and ambient sounds.
  • conversation has been deprecated.
  • text_in_video and logo are now part of visual.
• Streamlined endpoint structure: Several endpoints and parameters have been deprecated, removed, or renamed.

Version 1.2

If you have used the 1.1.2 version of the API, please refer to the following section for important information regarding the changes.

November 12, 2024

Improvements

• Cloud-to-cloud Integrations API: The API has been updated to provide a more intuitive experience. The /tasks/transfers endpoint will be deprecated. Use the following endpoints instead:
  • Import videos
  • Retrieve import status
  • Retrieve import logs
  Note

  Cloud-to-cloud integrations now require a paid plan. If you’re on the Free plan, you can find information on upgrading your plan in the  Upgrade your plan section.

November 5, 2024

Improvements

• Embed API: The structure of the responses has been streamlined across all endpoints to provide a more consistent and intuitive experience:
  • Standardized object naming:
    • The video_embeddings field has been renamed to video_embedding.
    • The video_embedding object now encapsulates the embeddings, related metadata, and additional information.
  • Enhanced response structure:
    • The embedding vectors are now nested under an array named segments.
    • The metadata objects have been moved under their respective parent embedding objects.
    • The is_success boolean has been removed.
  • Affected endpoints:
    • All video embedding endpoints the endpoint for creating video embedding tasks.
    • Create embeddings for text, image, and audio.
    • Retrieve video information.
• Embed API: You can now retrieve vector embeddings for any indexed video by setting embed=true in your GET /indexes/{index-id}/videos/{video-id} requests.

October 24, 2024

New features

• Embed API:
  • You can create image and audio embeddings in addition to its existing video and text capabilities. See the Create embeddings page for details.
  • You can now retrieve a list of the video embedding tasks in your account by invoking the GET method of the /embed/tasks endpoint.

July 7, 2024

New features

• Pegasus 1.1: The 1.1 version of the Pegasus video understanding engine has been released, introducing the following enhancements:

  • Improved model accuracy for video description and question-answering.
  • Fine-grained visual understanding and instruction following.
  • Streaming support when generating open-ended texts. For details, refer to the Streaming responses section.
  • Increased maximum prompt length to 375 tokens.
  • Extended maximum video duration to 30 minutes.
  Note

  Effective July 8, 2024, Pegasus 1.0 is no longer supported. All existing indexes created with Pegasus 1.0 will be automatically upgraded to Pegasus 1.1. No manual intervention is required for this migration process, and all indexes will utilize Pegasus 1.1 upon completion.

June 18, 2024

New features

• Image-to-Video Search API: TwelveLabs is proud to introduce the Image-to-Video Search API. This new API allows you to find semantically related video segments by providing an image as a query. The platform identifies similar content within videos. To get started with the Image-to-Video Search API, refer to the Searchpage.

May 15, 2024

New features

• Embed API: TwelveLabs is proud to introduce the Embed API. You can use this new API to create multimodal embeddings that are contextual vector representations for your videos and texts. You can utilize multimodal embeddings in various downstream tasks, including but not limited to training custom multimodal models for applications such as clustering, classification, search, recommendation, and anomaly detection. See the Create embeddings page for details.

March 12, 2024

New features

• TwelveLabs is proud to introduce the new versions of its video understanding models:
  • Marengo 2.6: A new state-of-the-art (SOTA) multimodal foundation model capable of performing any-to-any search tasks, including Text-To-Video, Text-To-Image, Text-To-Audio, Audio-To-Video, Image-To-Video, and more. Note that the platform currently supports text-to-video search and classification features. Other modalities will be supported in a future release. This model represents a significant leap in video understanding technology, enabling more intuitive and comprehensive search capabilities across various media types. For an overview of the new features and improvements in this version, refer to this blog post: Introducing Marengo 2.6: A New State-of-the-Art Video Foundation Model for Any-to-Any Search.
  • Pegasus 1.0 beta: This version of the model provides fine-grained video descriptions, summaries, and question-answering capabilities. For an overview of the new features and improvements in this version, refer to this blog post: Pegasus-1 Open Beta: Setting New Standards in Video-Language Modeling.
• The platform now supports search queries in multiple languages. For a complete list of supported languages, refer to the Supported languages page.

Updates

• You can now enable the Pegasus and Marengo video understanding engines on the same index.

February 15, 2024

New features

• You can now tune the temperature to control the randomness of the text output generated by the /summarize and /generate endpoints. See the Tune the temperature page for details.

October 30, 2023

New features

Version 1.2 of the TwelveLabs Video Understanding Platform introduces the following new features:

• The alpha version of the Pegasus video understanding engine has been released. You can now use it to analyze videos.
• You can now upload videos from external providers. Currently, only YouTube is supported as an external provider, but we will add support for additional providers in the future.

Updates

This section lists the differences between version 1.1.2 and version 1.2 of the TwelveLabs Video Understanding API.

• When you make an API call, make sure that you specify the 1.2 version in the URL. The URL should look similar to the following one: https://api.twelvelabs.io/v1.2/{resource}/{path_parameters}?{query_parameters}. For more details, see the Call an endpoint section.
• To enable the utilization of multiple engines for an index, the following changes have been made:
  • POST /indexes: The engine_id and indexing_options parameters of the request have been deprecated. Instead, you can now define the engine configuration as a list of objects. See the Create an index page for details.
  • GET /indexes/{index_id}: The engine_id field in the response has been superseded by an array of objects named engines. See the Retrieve an index page for details.
  • GET /indexes:
    • The engine_id field in the response has been superseded by an array of objects named engines. See the List indexes page for details.
    • The engine_family query parameter has been introduced, allowing you to filter by engine family.
    • The index_options query parameter has been marked for deprecation. You can still use it in this version of the API, but it will be deprecated in a future release. Instead, use engine_options or engine_family.
  • GET /engines: The allowed_index_option field in the response has been renamed to allowed_engine_options.
  • GET/engines/{engine-id} The allowed_index_option field in the response has been renamed to allowed_engine_options. See the Retrieve an engine page for details.
• The /search and /search/{page-token} endpoints no longer return the conversation_option, search_options, and query fields.

Version 1.1.2

If you have used the 1.1.1 version of the API, please refer to the following section for important information regarding the changes.

Improvements

To further improve the usability of the /classify endpoint, the following changes have been made:

• The endpoint now allows you to classify a set of videos. The video_id parameter has been deprecated and now you must pass an array of strings named video_ids instead. Each element of the array represents the unique identifier of a video you want to classify.

• The threshold field in the request is now an object, and you can use it to filter based on the following criteria:

  • The confidence level that a video matches the specified class
  • The confidence level that a clip matches the specified class.
  • The duration ratio, which is the sum of the lengths of the matching video clips inside a video divided by the total length of the video.

  For details, see the Filtering > Content classification page.

• The endpoint now supports pagination.

• The duration-weighted score has been deprecated. When setting the show_detailed_score parameter to true, the platform now returns the maximum, average, and normalized scores.

Version 1.1.1

If you have used the 1.1 version of the API, please refer to the following sections for important information regarding the changes.

New features

Version 1.1.1 of the TwelveLabs Video Understanding Platform introduces the following new features:

• Version 2.5 of the Marengo video understanding engine has been released. For details, see the Video understanding engines page.
• The /indexes, /search, /combined-search, and /classify endpoints now support the ability to integrate with the Playground, a sandbox environment that allows you to try out the features of the TwelveLabs Video Understanding Platform through an intuitive web page.
• The platform now supports the ability to store the video you’re uploading. For details, see the Create a video indexing task page.

Improvements

To further improve flexibility, usability, and clarity, the following changes have been made:

• Combined queries:
  • You can now define global values for the search_options and conversation_option parameters for the entire request instead of per-query basis. For details, see the Use combined queries page.
  • The /beta/search endpoint has been renamed to /combined-search.
• Logo detection: The logo add-on has been deprecated. To enable logo detection for an index, you must now use the logo indexing option.
• Conversation option: The transcription conversation option has been renamed to exact_match.
• Classifying videos:
  • The labels parameter has been renamed to classes.
  • The threshold field you can use to narrow down a response obtained from the platform is now of type int. For details, see the API Reference > Classify a video page.

Version 1.1

The introduction of new features and improvements in the 1.1 version of the TwelveLabs Video Understanding Platform has required changes to some endpoints. If you have used the 1.0 version of the API, please refer to the following sections for important information regarding the changes.

New features

Version 1.1 of the TwelveLabs Video Understanding Platform introduces the following new features:

• Classification of content: You can now define a list of labels that you want to classify your videos into, and the new classification API endpoint will return the duration for which the specified labels have appeared in your videos and the confidence that each of the matches represents the label you’ve specified.
• Combined Queries: The 1.1 version of the API introduces a new format of search queries named combined queries. A combined query includes any number of subqueries linked with any number of logical operators. Combined queries are executed in one API request. Combined queries support the following additional features:
  • Negating a condition: In addition to the existing AND operator, the platform now allows you to use the NOT operator to negate a condition. For example, this allows you to write a query that retrieves all the video clips in which someone is cooking but neither spaghetti nor lasagna is mentioned in the conversation.
  • The THEN operator: The platform now supports the THEN operator that allows you to specify that the platform must return only the results for which the order of the matching video clips is the same as the order of your queries.
  • Time-proximity search: The TwelveLabs Video Understanding API now allows you to use the proximity parameter to extend the lower and upper boundaries of each subquery. For example, this allows you to write a query that finds all car accidents that happened within a specific interval of time before someone wins a race. For details, see the Use combined queries page.
• Logo detection: The platform can now detect brand logos.

Updates

This section lists the differences between version 1 and version 1.1 of the TwelveLabs Video Understanding API.

• When you make an API call, make sure that you specify the 1.1 version in the URL. The URL should look similar to the following one: https://api.twelvelabs.io./v1.1/{resource}/{path_parameters}?{query_parameters}. For more details, see the Call an endpoint section.

• The following methods now return a 200 OK status code when the response is empty:

  • [GET] /indexes
  • [GET] /tasks
  • [GET] /indexes/{index_id}/videos

• The /tasks endpoint is now a separate endpoint and is no longer part of the /indexes endpoint. The table below shows the changes made to each method of the /tasks endpoint:

  1.0	1.1
  GET /indexes/tasks	GET /tasks
  POST /indexes/tasks	POST /tasks
  GET /indexes/tasks/{task_id}	GET /tasks/{task_id}
  DELETE /indexes/tasks/{task_id}	DELETE /tasks/{task_id}
  POST /indexes/tasks/transfers	POST /tasks/transfers
  GET /indexes/tasks/status	GET /tasks/status

• The /indexes/tasks/{task_id}/video_id endpoint has been deprecated. You can now retrieve the unique identifier of a video by invoking the GET method of the /tasks/{task_id} endpoint. The response will contain a field named video_id.

• When an error occurs, the platform now follows the recommendations of the RFC 9110 standard. Instead of numeric codes, the platform now returns string values containing human-readable descriptions of the errors. The format of the error messages is as follows:

  • code: A string representing the error code.
  • message: A human-readable string describing the error, intended to be suitable for display in a user interface.
  • (Optional) docs_url: The URL of the relevant documentation page. For example, if you tried to list all the videos in an index and the unique identifier of the index you specified didn’t exist, the 1.0 version of the API returned an error similar to the following one:

  1
  {
  2
    "error_code": 201,
  3
    "message": "ID 234234 does not exist"
  4
  }

  Now, when using the 1.1 version of the API, the error should look similar to the following one:

  1
  {
  2
    "code": "parameter_not_provided",
  3
    "message": "The index_id parameter is required but was not provided."
  4
  }

  For a list of error messages, see the API Reference > Error codes page.

• The next_page_id and prev_page_id fields of the page_info object have been renamed to next_page_token and prev_page_id.

• The type field has been removed from all the responses.

• When performing searches specifying multiple search options, the platform returns an object containing the confidence level that a specific video clip matched your search terms for each type of search. In version v1.0, this field was a dictionary named module_confidence. In version v1.1, this field is now named module and is of type array.

• The POST method of the /search/{page-token} endpoint has been deprecated. To retrieve the subsequent pages, you must now call the GET method of the /search/{page-token} endpoint, passing it the unique identifier of the page you want to retrieve.