> This page is for version v1.3 (default). > For other versions, use one of these documentation indexes: > - v1.3 (default): https://docs.twelvelabs.io/v1.3/llms.txt > For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.twelvelabs.io/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.twelvelabs.io/_mcp/server. # Release notes > New features, enhancements, and changes to the platform. Updated regularly. The sections below list all new features, enhancements, and changes to the platform, in chronological order. All dates and times are in the PT timezone, unless stated otherwise. # Version 1.3 ## October 7, 2026 ### Introducing item metadata for knowledge store items Knowledge store items now support item metadata: custom metadata stored on the item alone, separate from the user-defined metadata of its source asset. Send the `item_metadata` field when you create an item. To use this feature, upgrade to the latest version of the Python or Node.js SDK: **`Python SDK`** ```sh Python SDK pip install --upgrade twelvelabs ``` **`Node.js SDK`** ```sh Node.js SDK npm install twelvelabs-js@latest ``` For details, see the [Create a knowledge store item](/v1.3/api-reference/knowledge-store-items/create) and [The knowledge store item object](/v1.3/api-reference/knowledge-store-items/the-knowledge-store-item-object) pages in the API Reference section, or the SDK reference for [Python](/v1.3/sdk-reference/python/knowledge-store-items) and [Node.js](/v1.3/sdk-reference/node-js/knowledge-store-items). ### Deprecating the `metadata` field when creating a knowledge store item The `metadata` field of the request to create a knowledge store item is deprecated. Use `item_metadata` instead. The old name still works: the item stores the pairs you send as `metadata` in its `item_metadata` field. If your code reads `metadata` from the item, read `item_metadata` instead. For details, see the [Create a knowledge store item](/v1.3/api-reference/knowledge-store-items/create) page in the API Reference section, or the SDK reference for [Python](/v1.3/sdk-reference/python/knowledge-store-items) and [Node.js](/v1.3/sdk-reference/node-js/knowledge-store-items). ## October 6, 2026 ### Introducing Pegasus 1.6 Pegasus 1.6, a new video understanding model, analyzes videos and images to generate text. Key capabilities: * **Egocentric video understanding**: Analyzes first-person footage from sources such as wearable cameras and teleoperated systems. * **Image analysis**: Analyzes one to twenty images per request. * **Improved entity recognition**: Identifies and names people, objects, and characters in a scene more consistently than Pegasus 1.5. * **Improved metadata extraction**: Extracts metadata that is less dependent on the segment definitions, enabling richer fields in a single analysis. * **In-segment events**: Extracts a list of timestamped events inside each segment when you use video segmentation. * **Segmentation without a token limit**: Segments long videos and returns the segments it produced, even when the output is incomplete. To use these features, upgrade to the latest version of the Python or Node.js SDK: **`Python SDK`** ```sh Python SDK pip install --upgrade twelvelabs ``` **`Node.js SDK`** ```sh Node.js SDK npm install twelvelabs-js@latest ``` For a how-to guide, see the [Analyze videos and images](/v1.3/docs/guides/analyze-videos-and-images) page. For complete details about the model, see the [Pegasus 1.6](/v1.3/docs/concepts/models/pegasus/pegasus-1-6) page. ## September 30, 2026 ### Configurable embedding dimensions for Marengo 3.5 Choose how many dimensions Marengo 3.5 uses for each embedding by setting the `embedding_dimension` parameter to `128`, `256`, or `512`. The default is `512`, so existing integrations are unaffected. Use the same value across an index. A shorter embedding consists of the first values of the full-length embedding. Shorter embeddings reduce the size of your index and speed up similarity search; longer embeddings improve retrieval quality. To use this feature, upgrade to the latest version of the Python or Node.js SDK: **`Python SDK`** ```sh Python SDK pip install --upgrade twelvelabs ``` **`Node.js SDK`** ```sh Node.js SDK npm install twelvelabs-js@latest ``` For details, see the [Create sync embeddings](/v1.3/api-reference/create-embeddings-v2/create-embeddings) and [Create an async embedding task](/v1.3/api-reference/create-embeddings-v2/create-async-embedding-task) pages in the API Reference section, or the SDK reference for [Python](/v1.3/sdk-reference/python/create-embeddings-v-2) and [Node.js](/v1.3/sdk-reference/node-js/embed-v2). ### New segmentation options for asynchronous document embeddings Control how the platform divides a document before it creates embeddings by setting the `segmentation` parameter. The options depend on the file type: * **PDF files**: Divide each page into a 2×2 grid, producing five embeddings per page: one for the whole page, and one for each quarter. * **Plain text and Markdown files**: Divide the file into chunks of whole sentences, with a chunk size you control. To use this feature, upgrade to the latest version of the Python or Node.js SDK: **`Python SDK`** ```sh Python SDK pip install --upgrade twelvelabs ``` **`Node.js SDK`** ```sh Node.js SDK npm install twelvelabs-js@latest ``` For details, see the [Create an async embedding task](/v1.3/api-reference/create-embeddings-v2/create-async-embedding-task) page in the API Reference section, or the SDK reference for [Python](/v1.3/sdk-reference/python/create-embeddings-v-2) and [Node.js](/v1.3/sdk-reference/node-js/embed-v2). ## September 29, 2026 ### Cancel asynchronous analysis tasks You can now cancel an asynchronous analysis task in your account while it is queued, pending, or processing. Tasks created as part of a batch cannot be canceled individually; cancel the batch instead. Add handling for the `canceled` status in the code that branches on task status. The other values remain `queued`, `pending`, `processing`, `ready`, and `failed`. Canceling a task also produces a new `analyze.task.canceled` webhook notification. If your webhook endpoint is subscribed to all events, it now receives this notification. Update your handler to recognize `analyze.task.canceled`, or to ignore event types it does not support. **`Python SDK`** ```sh Python SDK pip install --upgrade twelvelabs ``` **`Node.js SDK`** ```sh Node.js SDK npm install twelvelabs-js@latest ``` For details, see the [Cancel an async analysis task](/v1.3/api-reference/analyze-videos/cancel-an-async-analysis-task) page in the API Reference section, or the SDK reference for [Python](/v1.3/sdk-reference/python/analyze-videos/async-analysis#cancel-an-async-analysis-task) and [Node.js](/v1.3/sdk-reference/node-js/analyze-videos/async-analysis#cancel-an-async-analysis-task). For the payload and delivery details of the notification, see the [Notification schema](/v1.3/docs/advanced/webhooks/response-schema) page. ## September 22, 2026 ### Pegasus now supports files up to 10 GB To analyze a video up to 10 GB, upload it with multipart uploads. For instructions, see the [Multipart uploads](/v1.3/api-reference/upload-content/multipart-uploads) page in the API Reference section, or the SDK reference for [Python](/v1.3/sdk-reference/python/upload-content/multipart-uploads) and [Node.js](/v1.3/sdk-reference/node-js/upload-content/multipart-uploads). Direct uploads remain limited to 4 GB, and duration limits are unchanged. ## September 10, 2026 ### Video segmentation tasks now fail when the output is truncated In video segmentation, an analysis task fails when its reaches the maximum response length or the [context window](/v1.3/docs/concepts/models/pegasus/pegasus-1-5#context-window) before completing. The `error.message` field states the limit the analysis reached. Previously, the task completed with `status` set to `ready`, `result.finish_reason` set to `length`, and an empty or partial segment list in `result.data`. A failed task returns no `result` object. If your code checks `finish_reason` to detect truncation, check `status` instead. General analysis is unchanged. A truncated response still completes with `status` set to `ready`, the partial output in `result.data`, and a warning in the `error` field. For details, see the [Segment videos](/v1.3/docs/guides/segment-videos) page. ### The duration limit now applies to the window you analyze For asynchronous analysis endpoints, the duration limit applies to the window you analyze, not the whole video. The window must stay within 2 hours. HLS and base64 sources are limited to 2 hours. The [`POST`](/v1.3/api-reference/analyze-videos/create-async-analysis-task) method of the `/analyze/tasks` endpoint accepts a video of up to 4 hours when you analyze only a portion of it. The same limits apply to each analysis request in the [`POST`](/v1.3/api-reference/analyze-videos/batch-analysis/create-batch) method of the `/analyze/batches` endpoint. When you omit `start_time` and `end_time`, you analyze the whole video, so the video must still be within 2 hours. Synchronous analysis with the [`POST`](/v1.3/api-reference/analyze-videos/sync-analysis) method of the `/analyze` endpoint keeps its 1-hour limit. File size limits are unchanged. ### The minimum analyzed duration is now 1 second The minimum analysis duration in Pegasus 1.5 is 1 second, down from 4 seconds. This minimum applies to the analyzed portion of a video, not the whole video. For example, you can analyze one second of a 60-second video. ## August 31, 2026 ### Introducing Marengo 3.5 > **Note** > > Marengo 3.5 creates embeddings only. For search, use Marengo 3.0. Marengo 3.5 is a new embedding model for video understanding. It analyzes video, audio, images, and documents, and produces 512-dimensional embeddings. Key features: * **Composed queries across modalities**: Combine text with images, video, or audio into a single query vector when you create embeddings synchronously. Reference each media source by name in the query text. * **Documents as an input type**: Create embeddings from PDF files, with one embedding per page. * **Video and audio duration**: No limit on the duration of video and audio files. Marengo 3.0 supports up to 4 hours. * **Audio track**: A unified encoder processes speech, music, and non-dialog audio. Marengo 3.0 uses a separate transcription embedding. * **Embedding uncertainty**: Request a per-dimension uncertainty vector alongside each embedding. A higher value shows lower confidence. * **Time-based metadata fusion**: Fold your own time-aligned text, such as a stats feed, into the fused embeddings of the segments it overlaps in time. * **Token usage**: See how much a composed query leans on each modality, and track the tokens each request uses. * **Automatic truncation**: Choose to truncate text longer than 2,000 tokens instead of receiving an error. To use these features, upgrade to the latest version of the Python or Node.js SDK: **`Python SDK`** ```sh Python SDK pip install --upgrade twelvelabs ``` **`Node.js SDK`** ```sh Node.js SDK npm install twelvelabs-js@latest ``` For details about creating embeddings with Marengo 3.5, see the [Create sync embeddings](/v1.3/api-reference/create-embeddings-v2/create-embeddings) and [Create an async embedding task](/v1.3/api-reference/create-embeddings-v2/create-async-embedding-task) API reference pages, or the SDK reference for [Python](/v1.3/sdk-reference/python/create-embeddings-v-2) and [Node.js](/v1.3/sdk-reference/node-js/embed-v2). For an overview of Marengo 3.5 and its input requirements, see the [Marengo 3.5](/v1.3/docs/concepts/models/marengo/marengo-3-5) page. For step-by-step migration instructions, see the [Migrate from Marengo 3.0 to Marengo 3.5](/v1.3/docs/get-started/migration-guides/marengo-3-0-to-3-5) page. Existing integrations that use Marengo 3.0 continue to work without changes. #### Rate limits The platform measures embedding with Marengo 3.5 in input tokens, not duration. Each type of content has its own limit: video, audio, image, document, and text. The platform never adds them into a shared total. Request limits are unchanged, and duration limits still apply to Marengo 3.0. For the per-tier values, see the [Rate limits](/v1.3/docs/get-started/rate-limits) page. #### SDK type change In the Python SDK, the `AsyncVideoInputRequest` and `AsyncAudioInputRequest` classes replace `VideoInputRequest` and `AudioInputRequest` on the [`embed.v_2.tasks.create`](/v1.3/sdk-reference/python/create-embeddings-v-2/create-async-embeddings#create-an-async-embedding-task) method. The new classes add the fields that Marengo 3.5 supports: `time_based_metadata` and the `local` embedding scope. This change affects type-checked Python code only. The SDK accepts both the old and the new classes, so your existing code continues to work after you upgrade. Update it to clear the type-checker errors. Synchronous calls to [`embed.v2.create`](/v1.3/sdk-reference/node-js/embed-v2/sync) and the Node.js SDK require no changes. ### PDF, text, and Markdown uploads You can now upload PDF, text, and Markdown files to the platform as assets. Local files can be up to 200 MB, and public URLs up to 512 MB. To use this feature, upgrade to the latest version of the Python or Node.js SDK: **`Python SDK`** ```sh Python SDK pip install --upgrade twelvelabs ``` **`Node.js SDK`** ```sh Node.js SDK npm install twelvelabs-js@latest ``` For details, see the [Upload content](/v1.3/api-reference/upload-content) page in the API Reference section, or the SDK reference for [Python](/v1.3/sdk-reference/python/upload-content) and [Node.js](/v1.3/sdk-reference/node-js/upload-content). ## August 19, 2026 ### Introducing transcriptions for video and audio assets You can now retrieve the transcription of a video or audio asset directly, without indexing it first. Previously, the platform generated transcriptions only as part of video indexing, and you retrieved them from the indexed video. Use this method when you analyze video or audio without creating an index, or when you want a transcript on its own. If you already index your videos, the transcription returned with the indexed video is unchanged. The platform transcribes video and audio assets asynchronously. Poll this endpoint to monitor the transcription status. When the status is `ready`, you can choose how the transcription is segmented: one entry for each word, for each sentence, or for each speaker turn. To use this feature, upgrade to the latest version of the Python or Node.js SDK: **`Python SDK`** ```sh Python SDK pip install --upgrade twelvelabs ``` **`Node.js SDK`** ```sh Node.js SDK npm install twelvelabs-js@latest ``` For details, see the [Retrieve the transcription of an asset](/v1.3/api-reference/manage-assets/retrieve-transcription) page in the API Reference section, or the SDK reference for [Python](/v1.3/sdk-reference/python/manage-assets#retrieve-the-transcription-of-an-asset) and [Node.js](/v1.3/sdk-reference/node-js/manage-assets#retrieve-the-transcription-of-an-asset). ## August 18, 2026 ### Pegasus 1.2 has been removed Use Pegasus 1.5 for all video analysis. The platform now rejects any request that specifies Pegasus 1.2, and uses Pegasus 1.5 by default when you omit the `model_name` parameter. This change affects the following: * **Analysis**: The `/analyze` and `/analyze/tasks` endpoints accept only Pegasus 1.5. * **Index creation**: You cannot create an index with Pegasus 1.2. * **Existing indexes**: You can no longer add videos to an index that has only Pegasus 1.2 enabled. When you add videos to an index that has both Marengo and Pegasus 1.2 enabled, the platform indexes them with Marengo only. #### Actions required Migrate your integration to Pegasus 1.5. Follow the steps in the [Migration guide](/v1.3/docs/get-started/migration-guide). #### Support For migration assistance, contact our support team at [support@twelvelabs.io](mailto:support@twelvelabs.io). ## August 14, 2026 ### Distinguish new data connector imports from duplicates Some of the files your users import from a connected account may already be in the platform from an earlier import. You can now show accurate counts, such as "3 imported, 2 already added," instead of treating every file as new. Each item returned by the [Import files](/v1.3/api-reference/data-connectors/imports/import-files) and [Retrieve an import](/v1.3/api-reference/data-connectors/imports/retrieve-an-import) methods now includes a field named `action` that indicates the outcome of the import operation. Previously, a file skipped as a duplicate was indistinguishable from a new import while it was processing, because both showed the `processing` status. Imports created before this release do not include the `action` field. For details on its possible values, see the [Item actions](/v1.3/api-reference/data-connectors/imports/the-import-object#item-actions) section on **The import object** page. ## July 22, 2026 ### Audio and image support for multipart uploads [Multipart uploads](/v1.3/api-reference/upload-content/multipart-uploads) now accept audio and images. Previously, this method accepted video only. * **Audio**: Local audio files up to **10 GB**, up from the 200 MB limit that applies to direct uploads. * **Images**: Local images up to **32 MB**, the same limit that applies to direct uploads. Multipart uploads increase the upload limit for local audio files. For images, direct uploads and multipart uploads have the same 32 MB limit. Use multipart uploads when you want one upload method for all content types. To use this feature, upgrade to the latest version of the Python or Node.js SDK: **`Python SDK`** ```sh Python SDK pip install --upgrade twelvelabs ``` **`Node.js SDK`** ```sh Node.js SDK npm install twelvelabs-js@latest ``` These limits apply when you create an asset. Each model still enforces its own file size and duration limits. For details, see the [Upload and processing methods](/v1.3/docs/concepts/upload-methods) page. ## July 20, 2026 ### Introducing the Google Drive data connector You can now import files from a connected Google Drive account into the platform as assets, without asking your users to download and re-upload each file. Each user grants your application access to their account, selects files using the Google Drive Picker, and your application imports them on their behalf. Imported files become assets, the same as files you upload directly. To use this feature, upgrade to the latest version of the Python or Node.js SDK: **`Python SDK`** ```sh Python SDK pip install --upgrade twelvelabs ``` **`Node.js SDK`** ```sh Node.js SDK npm install twelvelabs-js@latest ``` For details, see the **Data connectors** page in the [API Reference](/v1.3/api-reference/data-connectors) section, or the SDK reference for [Python](/v1.3/sdk-reference/python/data-connectors) and [Node.js](/v1.3/sdk-reference/node-js/data-connectors). ## July 15, 2026 ### Introducing Jockey (research preview) Jockey is now available in research preview. Use Jockey for reasoning across entire collections of videos and images. **What you can do** * Summarize the themes, subjects, and patterns across your videos and images * Search your videos and images and see why each result matches * Track a subject across multiple videos * Organize and categorize your videos and images * Assemble matching moments into highlight reels For more use cases, see the [Recipes](/v1.3/agents/recipes) section. **Get started** To use Jockey, upgrade to the latest version of the Python or Node.js SDK: **`Python SDK`** ```sh Python SDK pip install --upgrade twelvelabs ``` **`Node.js SDK`** ```sh Node.js SDK npm install twelvelabs-js@latest ``` For step-by-step instructions, see the [Quickstart](/v1.3/agents/get-started/quickstart) page. **Coming from Models?** For individual videos, Models remain available with dedicated Search, Analyze, and Embed APIs. To compare the two and plan a migration, see the [Migrate from Models](/v1.3/agents/get-started/migrate-from-models) page. ## July 7, 2026 ### Asynchronous upload validation The platform now processes every upload asynchronously. When you create an asset with [direct uploads](/v1.3/api-reference/upload-content/direct-uploads) or [multipart uploads](/v1.3/api-reference/upload-content/multipart-uploads), the platform returns the asset immediately with the `status` field set to `processing`. The asset transitions to the `ready` status when it is available for use, or to `failed` when the file is invalid or corrupt. Poll the [Retrieve an asset](/v1.3/api-reference/upload-content/direct-uploads/retrieve) endpoint until the `status` field is `ready` before you use the asset. **What changed**: * **Polling now applies to every upload.** Previously, only public URL uploads larger than 200 MB returned `processing` and required polling; smaller uploads returned `ready` in the create response. Now, every upload returns `processing`, so you must poll before using any asset. * **Invalid files now fail asynchronously.** Previously, an invalid or corrupt file returned an error at upload time. Now the upload succeeds, and the asset transitions to the `failed` status, which you detect by polling. If your integration already polls the asset status until it's `ready`, as shown on the [Direct uploads](/v1.3/api-reference/upload-content/direct-uploads) and [Multipart uploads](/v1.3/api-reference/upload-content/multipart-uploads) pages or in the guides, no change is required. Otherwise, poll until the status of the asset is `ready` before you use it: #### Python ```python import time asset = client.assets.create(method="url", url="") while asset.status == "processing": time.sleep(5) asset = client.assets.retrieve(asset.id) if asset.status == "failed": raise RuntimeError("Asset processing failed") ``` #### Node.js ```typescript let asset = await client.assets.create({ method: "url", url: "" }); while (asset.status === "processing") { await new Promise((resolve) => setTimeout(resolve, 5000)); asset = await client.assets.retrieve(asset.id); } if (asset.status === "failed") { throw new Error("Asset processing failed"); } ``` This change does not require an SDK upgrade. ### Technical metadata for assets The [Retrieve an asset](/v1.3/api-reference/upload-content/direct-uploads/retrieve) and [List assets](/v1.3/api-reference/upload-content/direct-uploads/list) responses now include an object named `technical_metadata` with details read from the media file: the container format, per-stream video and audio properties, image properties, and derived attributes such as aspect ratio. The platform populates this object asynchronously after the upload completes. To access this feature, upgrade your SDK to the latest version: **`Python SDK`** ```sh Python SDK pip install --upgrade twelvelabs ``` **`Node.js SDK`** ```sh Node.js SDK npm install twelvelabs-js@latest ``` For details, see [Retrieve an asset](/v1.3/api-reference/upload-content/direct-uploads/retrieve) in the API Reference section, or the SDK reference for [Python](/v1.3/sdk-reference/python/manage-assets#retrieve-an-asset) and [Node.js](/v1.3/sdk-reference/node-js/manage-assets#retrieve-an-asset). ### Higher upload limits * **Multipart uploads** now support local videos up to **10 GB** (previously 4 GB). * **Image uploads** now support files up to **32 MB** (previously 5 MB), for both local files and public URLs. These limits apply when you create an asset. Each model still enforces its own file size and duration limits. For details, see the [Upload and processing methods](/v1.3/docs/concepts/upload-methods) page. Public video and audio URL uploads remain limited to 4 GB. This change does not require an SDK upgrade. ## June 18, 2026 ### Introducing batch analysis You can now submit up to 1,000 video analysis requests in a single call with batch analysis, instead of creating and tracking one asynchronous analysis task per video. Batch analysis returns a single batch identifier you use to monitor progress and retrieve per-item results. Batch analysis requires Pegasus 1.5. **When to use batch analysis**: * Run the same model and analysis settings across many videos * Track a single batch instead of many individual analysis tasks To use this feature, upgrade to the latest version of the Python or Node.js SDK: **`Python SDK`** ```sh Python SDK pip install --upgrade twelvelabs ``` **`Node.js SDK`** ```sh Node.js SDK npm install twelvelabs-js@latest ``` For details, see the **Batch analysis** page in the [API Reference](/v1.3/api-reference/analyze-videos/batch-analysis) section, or the SDK reference for [Python](/v1.3/sdk-reference/python/analyze-videos/batch-analysis) and [Node.js](/v1.3/sdk-reference/node-js/analyze-videos/batch-analysis). ## June 5, 2026 ### Failure reason for embedding tasks The response of the [`GET`](/v1.3/api-reference/create-embeddings-v2/retrieve-embeddings) method of the `/embed-v2/tasks/{task_id}` endpoint now includes an object named [`error`](/v1.3/api-reference/create-embeddings-v2/retrieve-embeddings#response.body.error) that contains a human-readable failure reason when an embedding task fails. The platform returns this field only when `status` is `failed`. The field is available in the [Python SDK](/v1.3/sdk-reference/python/create-embeddings-v-2/create-async-embeddings#retrieve-task-status-and-results) and [Node.js SDK](/v1.3/sdk-reference/node-js/create-embeddings-v-2/create-async-embeddings#retrieve-task-status-and-results). To access this feature, upgrade your SDK to the latest version: **`Python SDK`** ```sh Python SDK pip install --upgrade twelvelabs ``` **`Node.js SDK`** ```sh Node.js SDK npm install twelvelabs-js@latest ``` ## June 4, 2026 ### Replace the user-defined metadata of an asset You can now replace the entire user-defined metadata of an asset in a single call with the new [`PUT`](/v1.3/api-reference/upload-content/direct-uploads/replace-user-metadata) method of the `/assets/{asset_id}/user-metadata` endpoint. The method overwrites the stored value in its entirety and removes any keys you omit from the request body. Unlike the existing [`PATCH`](/v1.3/api-reference/upload-content/direct-uploads/update-user-metadata) method, this method does not merge your changes into the existing metadata. The method is available in the [Python SDK](/v1.3/sdk-reference/python/manage-assets#replace-the-user-defined-metadata-of-an-asset) and [Node.js SDK](/v1.3/sdk-reference/node-js/manage-assets#replace-the-user-defined-metadata-of-an-asset). To access this feature, upgrade your SDK to the latest version: **`Python SDK`** ```sh Python SDK pip install --upgrade twelvelabs ``` **`Node.js SDK`** ```sh Node.js SDK npm install twelvelabs-js@latest ``` ## June 2, 2026 ### SDK support for user metadata and timestamp formatting The following features are now available in the Python and Node.js SDKs. * **User-defined metadata on asset upload**: [Announcement](#user-defined-metadata-on-asset-upload). * **Timestamp formatting for analysis responses**: [Announcement](#timestamp-formatting-for-analysis-responses). To access these features, upgrade your SDK to the latest version: **`Python SDK`** ```sh Python SDK pip install --upgrade twelvelabs ``` **`Node.js SDK`** ```sh Node.js SDK npm install twelvelabs-js@latest ``` ## May 28, 2026 ### Context window and longer responses for Pegasus 1.5 Pegasus 1.5 now uses a context window of 261,120 tokens that covers the combined input and output of each request. The maximum response length increases to 98,304 tokens (up from 65,536). These changes apply to [Sync analysis](/v1.3/api-reference/analyze-videos/sync-analysis) and [Create an async analysis task](/v1.3/api-reference/analyze-videos/create-async-analysis-task). For details about what counts toward the context window, see the [Context window](/v1.3/docs/concepts/models/pegasus/pegasus-1-5#context-window) section. Pegasus 1.2 still enforces the prompt limit of 2,000 tokens and supports responses up to 4,096 tokens. #### Truncated responses When a response reaches the maximum response length or the context window, the platform returns the partial output and sets `finish_reason` to `"length"`. A warning appears in the `error` field. Asynchronous tasks with truncated output keep `status` set to `"ready"` and return both `result` and `error`. Pegasus 1.2 async analysis tasks now also return `finish_reason` set to `"length"` when the response reaches the maximum response length. Previously, async analysis tasks always returned `"stop"`. #### Token usage Responses for Pegasus 1.5 now include the `usage.input_tokens` field. Use it with `usage.output_tokens` to track how much of the context window a request uses. For parameter details, see [Sync analysis](/v1.3/api-reference/analyze-videos/sync-analysis) and [Create an async analysis task](/v1.3/api-reference/analyze-videos/create-async-analysis-task) in the **API Reference** section. For truncation messages, see the [Error codes](/v1.3/api-reference/error-codes#the-analyze-endpoint) page. ## May 22, 2026 ### User-defined metadata on asset upload You can now attach user-defined metadata to your assets during upload. Previously, you could only add metadata after indexing a video. With this update, the following methods accept the `user_metadata` parameter: * [Create an asset](/v1.3/api-reference/upload-content/direct-uploads/create) * [Create a multipart upload session](/v1.3/api-reference/upload-content/multipart-uploads/create) * [Index an asset](/v1.3/api-reference/index-content/create) This release also introduces two new methods for managing user-defined metadata on existing assets: * [Update the user-defined metadata of an asset](/v1.3/api-reference/upload-content/direct-uploads/update-user-metadata) * [Delete the user-defined metadata of an asset](/v1.3/api-reference/upload-content/direct-uploads/delete-user-metadata) **Update**: SDK support is now available. ### Timestamp formatting for analysis responses You can now choose the output format for timestamps in both general analysis (prompt-based text generation) and video segmentation. This feature requires Pegasus 1.5. **Update**: SDK support is now available. #### Timestamp formatting in structured responses Declare a field as `{"type": "timestamp", "format": ""}` to control the format of the returned value. Available formats: `seconds`, `hh:mm:ss`, and `hh:mm:ss.fff`. For complete parameter specifications, see the following parameters in the API Reference section: * **Sync analysis**: [`json_schema`](/v1.3/api-reference/analyze-videos/sync-analysis#request.body.response_format.json_schema). * **Async analysis**: [`json_schema`](/v1.3/api-reference/analyze-videos/create-async-analysis-task#request.body.response_format.json_schema) and [`fields`](/v1.3/api-reference/analyze-videos/create-async-analysis-task#request.body.response_format.segment_definitions.fields). #### Format control for segment boundaries Set the `segment_time_format` parameter to control the output format for the automatic `start_time` and `end_time` boundaries on each returned segment. Available formats: `seconds`, `hh:mm:ss`, and `hh:mm:ss.fff`. For complete parameter specifications, see the [`segment_time_format`](/v1.3/api-reference/analyze-videos/create-async-analysis-task#request.body.response_format.segment_time_format) property in the API Reference section. ## May 7, 2026 ### Combined video hours limit on the Free plan The Free plan now includes a single 10-hour limit shared across indexing and video analysis. Previously, only indexing hours counted toward this limit. You can split the 10 hours however you choose between indexing and analysis. For details, see the [Frequently asked questions](/v1.3/docs/resources/frequently-asked-questions#what-are-the-video-hours-and-video-count-limits-per-index) page. ### Video segmentation pricing update for paid plans On paid plans, video segmentation pricing now factors in the number of segment definitions per request. The cost equals the billable video duration multiplied by the number of segment definitions. If you provide the [`start_time`](/v1.3/api-reference/analyze-videos/create-async-analysis-task#request.body.start_time) and [`end_time`](/v1.3/api-reference/analyze-videos/create-async-analysis-task#request.body.end_time) parameters, the billable duration covers that time range only. Otherwise, it covers the full video duration. On the Free plan, the number of segment definitions does not affect your limit. For details and examples, see the [Frequently asked questions](/v1.3/docs/resources/frequently-asked-questions#how-is-video-segmentation-priced) page. ### SDK support for Pegasus 1.5 All Pegasus 1.5 features are now available in the Python and Node.js SDKs. Upgrade to the latest version: **`Python SDK`** ```sh Python SDK pip install --upgrade twelvelabs ``` **`Node.js SDK`** ```sh Node.js SDK npm install twelvelabs-js@latest ``` #### Breaking changes The following change requires updates to your code when you upgrade the SDK: * **ResponseFormat type split**: The `ResponseFormat` type has been split into `SyncResponseFormat` (for sync analysis) and `AsyncResponseFormat` (for async analysis). Update your imports and type references. See the [Pegasus 1.2 to 1.5 migration guide](/v1.3/docs/get-started/migration-guides/pegasus-1-2-to-1-5) for details. #### New features The following features are opt-in. Your existing code continues to work without changes. * **Video segmentation**: Extract structured, timestamped metadata with custom segment definitions. See the [Segment videos](/v1.3/docs/guides/segment-videos) guide. * **Structured prompts with reference images**: Use the `prompt_v2` parameter to include up to 4 reference images in your prompt. * **Video clipping**: Use the `start_time` and `end_time` parameters to analyze a specific portion of the video. * **Per-definition time ranges for segmentation**: Restrict segment extraction to specific time windows within the video. #### Enhancements The following improvements take effect automatically. No code changes required. * **Analysis task response echo**: The `request_params` object in task responses now includes the full set of parameters you submitted when creating the task. * **Longer responses**: Pegasus 1.5 supports responses up to 98,304 tokens (raised from 65,536 on [May 28, 2026](#context-window-and-longer-responses-for-pegasus-15)). * **Custom task identifier**: Attach a custom identifier to async analysis tasks to correlate them across responses. For migration instructions, see the [Migration guides](/v1.3/docs/get-started/migration-guides). For details about each feature, see the original announcements on [April 20](#introducing-pegasus-15-with-video-segmentation), [April 27](#general-analysis-for-pegasus-15), and [May 7](#pegasus-15-now-supports-synchronous-analysis). ## May 6, 2026 ### Pegasus 1.5 now supports synchronous analysis You can now use Pegasus 1.5 with the synchronous analysis endpoint. Set the `model_name` parameter to `pegasus1.5` when calling the [`POST`](/v1.3/api-reference/analyze-videos/sync-analysis) method of the `/analyze` endpoint to analyze videos directly from a URL, asset, or base64 string and receive results in the response. The following Pegasus 1.5 capabilities, previously available only through async analysis, are now available on the synchronous analysis endpoint: * **Structured prompts with reference images**: Use the `prompt_v2` parameter to include up to 4 reference images in your prompt. * **Video clipping**: Use the `start_time` and `end_time` parameters to analyze a specific portion of the video. The clip must be at least 4 seconds long. > **Note** > > Synchronous analysis with Pegasus 1.5 supports general analysis (prompt-based text generation) only. For video segmentation, use the [`POST`](/v1.3/api-reference/analyze-videos/create-async-analysis-task) method of the `/analyze/tasks` endpoint. For complete parameter specifications, see the [Sync analysis](/v1.3/api-reference/analyze-videos/sync-analysis) page in the API Reference section. ## April 27, 2026 TwelveLabs is proud to introduce general analysis (prompt-based text generation) for Pegasus 1.5, along with the following new capabilities. ### General analysis for Pegasus 1.5 Set `model_name` to `pegasus1.5` and `analysis_mode` to `general` to analyze videos with a prompt. Pegasus 1.5 accepts videos directly from a URL, an asset, or a base64 string, with no pre-indexing required. ### Structured prompts with reference images Use the new `prompt_v2` field to include up to 4 reference images in your prompt. Assign a name to each image and insert `<@name>` placeholders in the prompt text (Example: `"Is there a <@tiger-1> in the video?"`). Requires `model_name` set to `pegasus1.5`. ### Video clipping Use the `start_time` and `end_time` fields to analyze only a portion of the video. The clip must be at least 4 seconds long. Requires `model_name` set to `pegasus1.5`. ### Per-definition time ranges for segmentation When using video segmentation (`analysis_mode` set to `time_based_metadata`), add the `time_ranges` to individual segment definitions to restrict segment extraction to specific parts of the video. This gives you finer control over which portions are analyzed for each definition. Requires `model_name` set to `pegasus1.5`. ### Analysis task response echo The `request_params` object in task responses now includes the full set of parameters you submitted when creating the task: `prompt`, `prompt_v2`, `json_schema`, `segment_definitions`, `time_ranges`, `start_time`, and `end_time`. The list endpoint returns compact versions (truncated prompts, omitted schemas), while the retrieve endpoint returns the full data. For complete parameter specifications, see the [Create an async analysis task](/v1.3/api-reference/analyze-videos/create-async-analysis-task) and [Retrieve analysis task status and results](/v1.3/api-reference/analyze-videos/retrieve-analysis-task-status-results) pages in the API Reference section. ## April 24, 2026 ### Track async analysis tasks with a custom identifier You can now attach a custom identifier to each async analysis task. Set the `custom_id` field when you call the [`POST`](/v1.3/api-reference/analyze-videos/create-async-analysis-task) method of the `/analyze/tasks` endpoint, and the platform returns it unchanged in the following responses: * The [`GET`](/v1.3/api-reference/analyze-videos/retrieve-analysis-task-status-results) method of the `/analyze/tasks/{task_id}` endpoint * The [`GET`](/v1.3/api-reference/analyze-videos/list-async-analysis-tasks) method of the `/analyze/tasks` endpoint * The `analyze.task.ready` and `analyze.task.failed` webhook payloads Use this field to correlate tasks, for example, to distinguish tasks by type or environment. For complete parameter specifications, see the [Create an async analysis task](/v1.3/api-reference/analyze-videos/create-async-analysis-task) page in the API Reference section. ## April 20, 2026 ### Introducing Pegasus 1.5 with video segmentation Pegasus 1.5, a new video understanding model, transforms raw videos into structured, timestamped data. Define the types of segments you want to detect, such as editorial segments, sports plays, speaker changes, or brand appearances, specify custom fields for each segment, and receive structured results in JSON format. Instead of manually reviewing footage or relying on brittle heuristics, you can define what matters to your application and automatically extract these events across entire video libraries. For general analysis (prompt-based text generation) support, see the [April 27 announcement](#pegasus-15-now-supports-general-analysis). Video segmentation is available through the asynchronous analysis endpoint. To use this feature, upgrade to the latest version of the Python or Node.js SDK: **`Python SDK`** ```sh Python SDK pip install --upgrade twelvelabs ``` **`Node.js SDK`** ```sh Node.js SDK npm install twelvelabs-js@latest ``` #### SDK type changes (breaking change) The `ResponseFormat` type has been split into two separate types: #### Python * **`AsyncResponseFormat`**: Used by the asynchronous analysis endpoint (`analyze_async.tasks.create`). Supports both `"json_schema"` and `"segment_definitions"` response types. * **`SyncResponseFormat`**: Used by the synchronous analysis endpoints (`analyze` and `analyze_stream`). Supports `"json_schema"` only. Update your imports when upgrading: ```python # Before from twelvelabs.types import ResponseFormat # After — for async analysis / video segmentation from twelvelabs.types import AsyncResponseFormat # After — for sync analysis from twelvelabs.types import SyncResponseFormat ``` #### Node.js * **`AsyncResponseFormat`**: Used by the asynchronous analysis endpoint (`analyzeAsync.tasks.create`). Supports both `"json_schema"` and `"segment_definitions"` response types. * **`SyncResponseFormat`**: Used by the synchronous analysis endpoints (`analyze` and `analyzeStream`). Supports `"json_schema"` only. Update your imports when upgrading: ```typescript // Before const responseFormat: TwelvelabsApi.ResponseFormat = { ... }; // After — for async analysis / video segmentation const responseFormat: TwelvelabsApi.AsyncResponseFormat = { ... }; // After — for sync analysis const responseFormat: TwelvelabsApi.SyncResponseFormat = { ... }; ``` For a how-to guide, see the [Segment videos](/v1.3/docs/guides/segment-videos) page. For complete parameter specifications, see the [Create an async analysis task](/v1.3/api-reference/analyze-videos/create-async-analysis-task) page in the API Reference section. ## April 15, 2026 ### Introducing asset management This release introduces new asset management capabilities. It also includes the first set of changes for the [deletion safeguards](#deleting-referenced-assets-will-be-denied-by-default-effective-april-26-2026) announced on April 8. #### HLS streaming and thumbnail generation You can now request HLS playlists and thumbnail images when creating an asset. Set the `enable_hls` or `enable_thumbnail` field to `true` in your request to the [`POST`](/v1.3/api-reference/upload-content/direct-uploads/create) method of the `/assets` endpoint or the [`POST`](/v1.3/api-reference/upload-content/multipart-uploads/create) method of the `/assets/multipart-uploads` endpoint. The platform generates these in the background. To check progress, call the [`GET`](/v1.3/api-reference/upload-content/direct-uploads/retrieve) method of the `/assets/{asset_id}` endpoint and inspect the `hls.status` and `thumbnail.status` fields. #### New response fields The following response schemas now include new fields: * **Asset**: The `size` field contains the file size in bytes. The `duration` field contains the length in seconds for video and audio assets. The `method` field can now also return `multipart` for assets uploaded through the multipart upload flow. * **Indexed asset**: The `asset_id` field identifies the source asset, so you can trace an indexed asset back to the asset it was created from. * **Entity**: The `entity_collection_id` field identifies the collection the entity belongs to. #### Filename filtering The [`GET`](/v1.3/api-reference/upload-content/direct-uploads/list) method of the `/assets` endpoint now accepts the `filename` query parameter to filter assets by filename. The match is case-insensitive and supports partial matches. #### Look up indexed assets and entities by asset As [announced on April 8](#deleting-referenced-assets-will-be-denied-by-default-effective-april-26-2026), the following are now available: * The [`GET`](/v1.3/api-reference/index-content/list-indexed-assets-by-asset) method of the `/assets/{asset_id}/indexed-assets` endpoint lists the indexed assets that reference a given asset. * The [`GET`](/v1.3/api-reference/entities/list-entities-by-asset) method of the `/assets/{asset_id}/entities` endpoint lists the entities that reference a given asset. * The `force` query parameter on the [`DELETE`](/v1.3/api-reference/upload-content/direct-uploads/delete) method of the `/assets/{asset_id}` endpoint lets you delete the asset even if indexed assets reference it. On **April 26, 2026**, delete requests for referenced assets will be denied by default. See the [April 8 announcement](#deleting-referenced-assets-will-be-denied-by-default-effective-april-26-2026) for required actions. ## April 8, 2026 ### Deleting referenced assets will be denied by default (Effective April 26, 2026) Currently, you can delete any asset, even if an indexed asset references it. Starting on April 26, 2026, the platform will deny delete requests for referenced assets by default, returning a `409 Conflict` error. These changes will roll out in two phases: * **April 15, 2026**: The following become available, and you can begin updating your deletion workflow: * The `GET` method of the `/assets/{asset_id}/indexed-assets` endpoint * The `force` query parameter on the [`DELETE`](/v1.3/api-reference/upload-content/direct-uploads/delete) method of the `/assets/{asset_id}` endpoint * **April 26, 2026** (breaking change): Delete requests for referenced assets are denied by default. #### Actions required Complete these steps before April 26, 2026: Review your code for calls to the [`DELETE`](/v1.3/api-reference/upload-content/direct-uploads/delete) method of the `/assets/{asset_id}` endpoint. Before deleting, call the `GET` method of the `/assets/{asset_id}/indexed-assets` endpoint to check whether any indexed assets reference the asset. If references exist, either remove them before deleting the asset or add `force=true` as a query parameter to delete it anyway. Use `force=true` only when your application intends to bypass the reference check and delete the asset regardless of existing references. Update your error handling to account for `409 Conflict` responses on delete requests. Test your changes before April 26, 2026. #### Support For migration assistance, contact our support team at [support@twelvelabs.io](mailto:support@twelvelabs.io). ## March 31, 2026 ### Introducing async analysis endpoints for longer videos You can now analyze videos up to 2 hours using the new asynchronous analysis endpoints. Previously, the Analyze API only supported synchronous analysis for videos up to 1 hour long. **When to use async analysis**: * Videos longer than one hour (up to 2 hours) * Background processing without blocking your application **When to use sync analysis**: * Videos up to one hour * Immediate results or real-time streaming For a how-to guide, see the [Analyze videos and images](/v1.3/docs/guides/analyze-videos-and-images) page. For complete parameter specifications, see the API reference for [async analysis](/v1.3/api-reference/analyze-videos/create-async-analysis-task) or the SDK reference for [Python](/v1.3/sdk-reference/python/analyze-videos/async-analysis) and [Node.js](/v1.3/sdk-reference/node-js/analyze-videos/async-analysis). ## March 30, 2026 On March 30th, 2026 (7PM PT), Marengo 2.7 has been sunset. You no longer can index new content, perform search requests, or retrieve any embeddings from previously indexed content. Update your application to ensure continuous service. ## March 11, 2026 TwelveLabs is proud to introduce multiple-image search, multiple-image embeddings, and fused embeddings. To use these features, upgrade to version 1.2.1 or later of the Python or Node.js SDK: **`Python SDK`** ```sh Python SDK pip install twelvelabs==1.2.1 ``` **`Node.js SDK`** ```sh Node.js SDK npm install twelvelabs-js@1.2.1 ``` ### Multiple-image search You can now use up to 10 images, alone or combined with a text query, in a single search request. For a how-to guide, see the [Search](/v1.3/docs/guides/search) page. For complete parameter specifications, see the [Make any-to-video search requests](/v1.3/api-reference/any-to-video-search/make-search-request) page in the API Reference section. ### Multiple-image embeddings The Embed API v2 now creates a single embedding from up to 10 images. You can also include text for context. To reference a specific image in your text, assign a name to that image and use a placeholder in the format `<@name>`. For example: "A person wearing \<@outfit> and holding \<@accessory>." For a how-to guide, see the [Embed a query](/v1.3/docs/guides/create-embeddings/query) page. For complete parameter specifications, see the [Create sync embeddings](/v1.3/api-reference/create-embeddings-v2/create-embeddings) page in the API Reference section. ### Fused embeddings The Embed API v2 now lets you control how the platform returns embeddings for audio and video content. You can request separate embeddings per modality, a single combined embedding that integrates all modalities, or both in the same response. For how-to guides, see the [Audio embeddings](/v1.3/docs/guides/create-embeddings/at-scale/audio) and [Video embeddings](/v1.3/docs/guides/create-embeddings/at-scale/video) pages. For complete parameter specifications, see the `embedding_type` parameter in the API Reference section: * [Audio embeddings](/v1.3/api-reference/create-embeddings-v2/create-embeddings#request.body.audio.embedding_type) * [Video embeddings](/v1.3/api-reference/create-embeddings-v2/create-embeddings#request.body.video.embedding_type) ## February 28, 2026 ### Marengo 2.7 will be deprecated > **Note** > > The timeline and actions on this page apply to the first-party API. If you use Marengo through Amazon Bedrock, see the [Amazon Bedrock migration guide](/v1.3/docs/cloud-partner-integrations/amazon-bedrock/migration-guide) instead. On March 30th, 2026 (7PM PT), Marengo 2.7 has been sunset. You no longer can index new content, perform search requests, or retrieve any embeddings from previously indexed content. Update your application to ensure continuous service. #### Actions required * **Search operations**: Update your application logic for Marengo 3.0 compatibility. Deprecated parameters are now ignored, and audio processing has changed. Your code will not return errors, but results may differ. * **Embeddings**: Regenerate all embeddings. Marengo 2.7 embeddings are not compatible with Marengo 3.0. #### Support For migration assistance, contact our support team at [support@twelvelabs.io](mailto:support@twelvelabs.io). ## February 15, 2026 ### Predefined formats for video analysis have been sunset and removed The `/gist` and `/summarize` endpoints, which generate text based on predefined formats, have been sunset and removed on February 15, 2026. Instead, use the [`/analyze`](/api-reference/analyze-videos/analyze) endpoint, which provides structured JSON responses. If you have not migrated yet, follow the migration steps in the [January 7, 2026](/docs/get-started/release-notes#january-7-2026) entry below. ## January 26, 2026 ### Introducing custom API key expiration periods API keys previously expired after 90 days. You can now choose from 3 months, 6 months, 12 months (default), a custom date, or never expire. This change applies only to new API keys created after January 26, 2026. The expiration dates for existing API keys remain unchanged. ## January 7, 2026 ### Predefined formats for video analysis will be sunset and removed **Update**: This deprecation was completed on February 15, 2026. See the February 15, 2026 entry for details. The `/gist` and `/summarize` endpoints, which generate text based on predefined formats, will be sunset and removed on February 15, 2026. Instead, use the [`/analyze`](/api-reference/analyze-videos/analyze) endpoint, which provides structured JSON responses. This consolidation provides the same capabilities with the following benefits: * **Reduced API surface area**: A single endpoint simplifies integration and maintenance * **Fully customizable schemas**: Define custom fields, naming conventions, and schema variations to match your requirements * **Improved chapterization quality**: Significantly better chapter detection and segmentation * **Better adherence to rules**: Enhanced accuracy when applying duration-based chapter rules * **More stable output**: Predictable, consistent responses across requests #### Actions required Complete these migration steps before February 15, 2026: Update your code to use the `/analyze` endpoint instead of `/gist` and `/summarize`. Define a schema for the `/analyze` endpoint that outlines the expected structure for the response. The prompt in the example below requests the title of a video, and the schema defines a field named `title` for the response. **`Python SDK`** ```python Python SDK import json from twelvelabs import TwelveLabs from twelvelabs.types import ResponseFormat client = TwelveLabs(api_key="") text = client.analyze( video_id="", prompt="Provide the title of this video", response_format=ResponseFormat( type="json_schema", json_schema={ "type": "object", "properties": { "title": {"type": "string"} }, "required": ["title"], }, ), ) data = json.loads(text.data) if text.data else {} print(f"Title: {data.get('title', 'N/A')}") ``` **`Node.js SDK`** ```typescript Node.js SDK import { TwelveLabs } from "twelvelabs-js"; const client = new TwelveLabs({ apiKey: "" }); const result = await client.analyze({ videoId: "", prompt: "Provide the title of this video", responseFormat: { type: "json_schema", jsonSchema: { type: "object", properties: { title: { type: "string" }, }, required: ["title"], }, }, }); const data = result.data ? JSON.parse(result.data) : {}; console.log(`Title: ${data.title ?? "N/A"}`); ``` **`cURL`** ```shell cURL curl -X POST https://api.twelvelabs.io/v1.3/analyze \ -H "x-api-key: " \ -H "Content-Type: application/json" \ -d '{ "video_id": "", "prompt": "Provide the title of this video", "response_format": { "type": "json_schema", "json_schema": { "type": "object", "properties": { "title": {"type": "string"} }, "required": ["title"] } }, "stream": false }' ``` For more examples demonstrating how to use structured JSON responses instead of the `/gist` and `/summarize` endpoints, see the [Structured responses](/v1.3/docs/guides/analyze-videos-and-images/structured-responses) page. Test your changes by comparing outputs and finalizing your schemas before February 15, 2026. #### Support For migration assistance, contact our support team at [support@twelvelabs.io](mailto:support@twelvelabs.io). ## December 12, 2025 ### Enhanced rate limits (Effective January 12, 2026) TwelveLabs is introducing an enhanced rate-limiting system that provides better visibility and more granular control over your usage: * **Multi-dimensional rate limiting**: Rate limits will measure usage across multiple dimensions: * Request count * Video and audio duration processed * Token usage for text generation * **Tiered Developer plan**: The Developer plan will include three tiers that automatically upgrade based on monthly spending. * **Modality-based limits**: Separate rate limits for video, audio, image, and text processing. * **Enhanced response headers**: New multi-dimensional headers will show your remaining capacity for each dimension. * **Automatic tier management**: The platform will automatically upgrade your tier when you reach spending thresholds. Downgrades include a one-month grace period. #### Actions required Review the [new rate limits](/v1.3/docs/get-started/rate-limits) to understand how they affect your usage. Update your applications to handle the new multi-dimensional rate limit headers. Monitor your usage. ## December 5, 2025 ### Pegasus 1.2 is now available in 23 new AWS regions via global cross-region inference Amazon Bedrock introduces global cross-region inference for Pegasus 1.2, expanding the model's availability to 23 additional regions. You can now also access the model in all EU regions using geographic cross-region inference. Use geographic cross-region inference for data residency or compliance requirements. Use global cross-region inference for high availability, performance, and cost efficiency. For a complete list of supported regions and detailed information about cross-region inference, see the [Analyze videos](/docs/cloud-partner-integrations/amazon-bedrock/analyze-videos) page. ## November 17, 2025 ### Marengo 3.0 is now generally available TwelveLabs is proud to announce that Marengo 3.0 is now generally available through the first-party API. This version improves accuracy and performance in the following areas: * **Composed text and image search**: Combine text descriptions with images in a single search query for more precise results. * **Improved cinematography understanding**: Enhanced search performance for cinematography terms like zoom, pan, and tracking shot. * **Sports intelligence**: Improved recognition of soccer and basketball actions. Support for baseball, ice hockey, and American football. * **Faster indexing**: Significant performance improvement with the new indexing technology. * **Extended text processing**: Maximum text length increased from 77 to 500 tokens for both search queries and text embeddings. * **Optimized embeddings**: 512-dimensional embeddings for faster processing and reduced storage. * **Long content support**: Process up to four hours of video and audio content while maintaining context. * **Expanded language support**: Query videos in 36 languages plus English (up from 12 plus English). This release includes the following additional features: * **Entity search**: Find specific people performing actions in your videos, such as locating players in sports footage, tracking characters in entertainment content, and analyzing individuals in surveillance videos. For details, see the [Entity search](/v1.3/docs/guides/search/entity-search) page. * **Composed text and image search**: Search using text descriptions and images in a single query. For concepts, see the [Composed text and image queries](/v1.3/docs/guides/search/search-with-text-and-image-queries#composed-text-and-image-queries) page. For a how-to guide, see the [Search](/v1.3/docs/guides/search) page. * **New methods for uploading content**: The platform now provides two new methods for uploading your videos, images, and audio files. These methods create reusable assets that you can use across different workflows, including search, analysis, and embedding creation. * **Direct uploads**: Upload whole files without splitting them. Use this method for a simple upload process. Direct uploads support files up to 4 GB. * **Multipart uploads**: Upload large local files by dividing them into smaller chunks. This method enables reliable uploads of large files and supports parallel processing and resuming interrupted transfers. TwelveLabs recommends this method for local files larger than 200 MB, with a maximum file size of 4 GB. For details, see the [Upload content](/v1.3/api-reference/upload-content) page. * **Embed API v2**: Create embeddings for text, images, audio, and video content using the new Embed API v2.This API provides two endpoints: * **Synchronous endpoint**: Use for text, images, or audio and video content under 10 minutes. * **Asynchronous endpoint**: Use for audio and video content up to 4 hours. For details, see the [Create embeddings v2](/v1.3/api-reference/create-embeddings-v2) page. ## October 30, 2025 ### Pegasus 1.2 expands to three additional AWS regions Support for Pegasus 1.2 now includes US East (Ohio), US West (N. California), and Europe (Frankfurt), bringing the total number of supported regions to seven. Deploy your applications in regions near your data and users to improve response times and streamline your infrastructure setup. **Regional availability**: Pegasus 1.2 is now available in the following regions: US East (N. Virginia), US West (Oregon), US East (Ohio), US West (N. California), Europe (Ireland), Europe (Frankfurt), Asia Pacific (Seoul) For details on using the model, see the [Analyze videos](/docs/cloud-partner-integrations/amazon-bedrock/analyze-videos) page. ## October 29, 2025 ### Marengo 3.0 is now available in Amazon Bedrock Amazon Bedrock now supports the Marengo 3.0 video understanding model, which unifies videos, images, audio, and text into a single representation space. Use it to build any-to-any search, recommendation systems, and content analysis applications. > **Info** > > Marengo 3.0 is available on Amazon Bedrock. First-party API support will be added in a future release. Marengo 3.0 introduces the following enhancements: * **Extended content processing**: Process up to 4 hours of video or audio content and files up to 6 GB. Ideal for analyzing sporting events, training videos, and film productions. * **Enhanced sports analysis**: Improved understanding of gameplay dynamics, player movements, and event detection. Better recognition of soccer and basketball actions with new support for baseball, ice hockey, and American football. * **Global multilingual support**: Language support expanded from 12 to 36 languages (plus English), enabling unified search across diverse regions and markets. * **Multimodal search precision**: Combine images and text in a single embedding request to merge visual similarity with semantic understanding for more accurate results. * **Expanded text processing**: Maximum text length increased from 77 to 500 tokens for text embeddings. Supports a more detailed and richer context. * **Optimized embeddings**: 512-dimensional embeddings deliver faster processing and reduced storage requirements. **Regional availability**: US East (N. Virginia), Europe (Ireland), and Asia Pacific (Seoul) To get started, visit the [Amazon Bedrock console](https://console.aws.amazon.com/bedrock/) page. For details on pricing, see the [Amazon Bedrock pricing](https://aws.amazon.com/bedrock/pricing/) page. ## October 14, 2025 TwelveLabs is proud to announce that Marengo 3.0 R2 is now available with the following enhancements: ### Composed text and image search Search using text descriptions and images in a single query. This feature allows you to specify both visual references and textual context for more precise results. For concepts, see the [Composed text and image queries](/v1.3/docs/guides/search/search-with-text-and-image-queries#composed-text-and-image-queries) page. For a how-to guide, see the [Search](/v1.3/docs/guides/search) page. For complete parameter specifications, see the [Make any-to-video search requests](/v1.3/api-reference/any-to-video-search/make-search-request) page. ### Improved cinematography understanding The platform now provides enhanced search performance for cinematography terms like zoom, pan, and tracking shot. ### Enhanced control over spoken word searches Search spoken words independently from other audio using the new `transcription` value in the `search_options` parameter. The `transcription_options` parameter offers two matching methods: * **Lexical matching**: Finds exact words and phrases (ideal for product names and technical terms) * **Semantic matching**: Finds concepts expressed differently (ideal for general topics) * **Combined**: Uses both methods for comprehensive results For more details and example use cases, see the [Transcription options](/v1.3/docs/concepts/modalities#transcription-options) section. For complete parameter specifications, see the [Make any-to-video search requests](/v1.3/api-reference/any-to-video-search/make-search-request) page. ### Expanded language support for video search The Marengo 3.0 video understanding model now supports 36 languages, in addition to English, for video search operations (up from 12 languages). Language support applies to your queries, regardless of the language used in the content. * **Find visual content**: Query in any language to locate objects, actions, on-screen text, and brand logos. * **Find sounds and music**: Query in any language to locate music, sound effects, and ambient audio. * **Find spoken words**: Query in any language to locate dialogue and conversations. Use semantic search to query in one language and find content in another. Use a lexical search to find exact phrases in the same language as the content. **Supported languages**: Arabic, Bengali, Chinese (Simplified), Croatian, Cusco, Czech, Danish, Dutch, English, Farsi, Filipino, Finnish, French, German, Greek, Hebrew, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Maori, Norwegian, Polish, Portuguese, Romanian, Russian, Spanish, Swahili, Swedish, Telugu, Thai, Turkish, Ukrainian, and Vietnamese. ### Enhanced sports intelligence The Marengo 3.0 video understanding model enhances the recognition of soccer and basketball actions, and provides support for baseball, ice hockey, and American football. ### Extended audio processing Process up to four hours of audio content while maintaining context. ## September 17, 2025 ### The cloud-to-cloud integrations feature will be deprecated After **October 31, 2025**, the cloud-to-cloud integrations feature will be deprecated, and files in your S3 buckets will no longer be uploaded to the platform automatically. #### Actions required This change affects your file processing workflow. Complete the migration steps before October 31, 2025. Update your code to iterate through your S3 objects and create presigned URLs. While you can use simple public URLs by making your S3 objects publicly accessible, TwelveLabs recommends S3 presigned URLs because they provide time-limited access with built-in security controls without making your entire bucket public. For details on creating presigned URLs, see the [Sharing objects with presigned URLs](https://docs.aws.amazon.com/AmazonS3/latest/userguide/ShareObjectPreSignedURL.html) page of the official Amazon documentation. Implement a new upload process using the [`POST`](/v1.3/api-reference/upload-content/tasks/create) method of the `/tasks` endpoint for each file. **`Python SDK`** ```python Python SDK task = client.tasks.create(index_id="", video_url="") ``` **`Node.js SDK`** ```typescript Node.js SDK const task = await client.tasks.create({ indexId: "", videoUrl:"", }); ``` **`cURL`** ```curl cURL curl -X POST https://api.twelvelabs.io/v1.3/tasks \ -H "x-api-key: " \ -H "Content-Type: multipart/form-data" \ -F index_id="" \ -F video_url="" ``` Test your updated workflow thoroughly. #### Support For migration assistance, contact our support team at [support@twelvelabs.io](mailto:support@twelvelabs.io). ## September 16, 2025 ### Introducing structured JSON responses for video analysis You can now request structured JSON responses when analyzing video content to generate open-ended text and summaries. This feature allows you to receive predictable, machine-readable outputs. To learn how to use structured responses effectively with examples and best practices, refer to the [Structured responses](/v1.3/docs/guides/analyze-videos-and-images/structured-responses) guide. For complete parameter specifications, see the [Sync analysis](/v1.3/api-reference/analyze-videos/sync-analysis) and [Summaries, chapters, and highlights](/v1.3/api-reference/analyze-videos/summarize) pages in the API Reference section. ## September 12, 2025 TwelveLabs is proud to introduce the following new features and improvements: ### Marengo 3 video understanding model > **Info** > > Marengo 3.0 is available as a limited, private preview. To request access to the preview version, contact us at [sales@twelvelabs.io](mailto:sales@twelvelabs.io). * **Sports intelligence**: Improved recognition of soccer and basketball actions. * **Faster indexing**: Significant performance improvement with the new indexing technology. * **Extended text processing**: Maximum text length increased from 77 to 500 tokens for both search queries and text embeddings. * **Optimized embeddings**: 512-dimensional embeddings for faster processing and reduced storage. * **Long content support**: Process up to four hours of video content while maintaining context. **API Changes**: When using Marengo 3, this update affects the following existing endpoints: * [`POST`](/v1.3/api-reference/create-embeddings-v1/text-image-audio-embeddings/create-text-image-audio-embeddings) `/embed` * The `text_truncate` parameter is deprecated and the maximum length for text embeddings is now 500 tokens. * [`GET`](/v1.3/api-reference/videos/retrieve) `/indexes/:index-id/videos/:video-id` and [`GET`](/v1.3/api-reference/create-embeddings-v1/video-embeddings/retrieve-video-embeddings) `/embed/tasks/:task_id` * The `embedding_option` parameter can now take the following values: `visual`, `audio`, `transcription`. For details, see the [Embedding options](/v1.3/docs/concepts/modalities#embedding-options) section. * [`POST`](/v1.3/api-reference/any-to-video-search/make-search-request) `/search` * The `audio` search option now excludes human speech. A new search option named `transcription` for finding specific spoken words and phrases will be available in a future release. * The maximum query length is now 500 tokens. * The platform no longer returns the `score` and `confidence` fields. Use the new `rank` field instead to determine the relevance of a result. ### Entity Search API The Entity Search API enables you to search for specific people performing actions. Use cases include finding players in sports footage, tracking characters in entertainment content, and analyzing individuals in surveillance videos. To get started, see the [Entity Search](/v1.3/docs/guides/search/entity-search) page. ## September 9, 2025 ### Amazon Bedrock adds synchronous inference for Marengo 2.7 Amazon Bedrock now supports synchronous inference for Marengo 2.7. You can now retrieve low-latency text and image embeddings directly in the API response. This update enables you to build more responsive and interactive search and retrieval experiences. The synchronous inference maintains the same video understanding capabilities that Marengo 2.7 provides. To generate embeddings from video, audio, and large-scale images, continue using asynchronous inference for optimal performance. **Regional availability** Marengo 2.7 with synchronous inference is available in these regions: * US East (N. Virginia) * Europe (Ireland) * Asia Pacific (Seoul) **Next steps** Review the [Create embeddings](/v1.3/docs/cloud-partner-integrations/amazon-bedrock/create-embeddings) guide to get started. ## August 28, 2025 ### The PUT method of the `/indexes/:index-id/videos/:video-id` endpoint has been deprecated The `PUT` method is now deprecated. Use the [`PATCH`](/v1.3/api-reference/videos/update) method instead, which provides identical functionality. ## August 5, 2025 ### Version 1.0.0 of the Python and Node.js SDKs is now the stable version TwelveLabs has released version 1.0.0 of the Python and Node.js SDKs as the stable version. You can still use versions 0.4.x and earlier, but they are no longer actively maintained. #### Action required Upgrade to version 1.0.0 for continued support and updates. ## July 18, 2025 ### The pre-release version 1.0.0 of the Python and Node.js SDKs is now available TwelveLabs is proud to introduce redesigned SDKs that incorporate modern development practices to enhance the developer experience. > **Tip** > > Version 1.0.0 is a pre-release version. Version 0.4.x remains the current stable version. #### Key Improvements * **Enhanced type safety and development experience**: Complete typing support, full type hints, auto-completion, and Pydantic v1/v2 compatibility for superior IDE integration, safer code, and modern tooling. * **Auto-generated architecture**: Built directly from the OpenAPI specifications, ensuring API consistency and automatic updates with new features. * **Improved pagination**: Automatic page fetching, allowing easier handling of large datasets. * **Native async support**: The Python SDK now fully supports `async`/`await` with a dedicated asynchronous client. * **Improved documentation**: Detailed docstrings/JSDoc comments and thorough SDK Reference sections that outline all parameters and response fields. #### Actions required The pre-release version includes breaking changes from the stable 0.4.x release. Before you use version 1.0.0 in your application: * Update your code to work with the new version * Test all changes thoroughly * Verify that everything functions as expected before you deploy to production. > **Note** > > If you encounter any issues with this pre-release version, please report them on the Issues page of the respective GitHub repository: > > * [twelvelabs-python](https://github.com/twelvelabs-io/twelvelabs-python/issues). > * [twelvelabs-js](https://github.com/twelvelabs-io/twelvelabs-js/issues) #### Version support TwelveLabs will stop maintaining versions up to 0.4.x after the stable 1.0.0 version is officially released. #### Documentation The documentation has been updated with current examples, detailed guides, and comprehensive SDK Reference sections. ## July 16, 2025 ### Marengo 2.7 and Pegasus 1.2 are now available in Amazon Bedrock Amazon Bedrock now supports the Marengo 2.7 and Pegasus 1.2 video understanding models. **Regional availability** * **Marengo 2.7**: Available in US East (N. Virginia), Europe (Ireland), and Asia Pacific (Seoul) * **Pegasus 1.2**: Available in US West (Oregon) and Europe (Ireland) through cross-region inference For details on accessing these models, see the [Amazon Bedrock](/docs/cloud-partner-integrations/amazon-bedrock) page. For details on pricing, see the [Amazon Bedrock pricing](https://aws.amazon.com/bedrock/pricing/) page. ## July 11, 2025 ### Enhanced user metadata capabilities The platform now provides advanced capabilities for managing user metadata, offering improved flexibility and precision: * **Upload videos with metadata**: When you upload a video, you can now include user-defined metadata in the request body. This metadata will be attached to the video, helping you categorize it effectively. * **Update or delete metadata fields**: After a video has been uploaded to the platform, you can update specific metadata fields, such as the file name, or delete them by setting the value to `null`. * **Controls metadata in search results**: When you make search requests, you can specify whether user-defined metadata appears in the search results. > **Note** > > To use these new capabilities, you must invoke the API directly. These changes will be supported in a future version of the official SDKs. For more details on these enhancements, refer to the updated documentation: * [Create a video indexing task](/v1.3/api-reference/upload-content/tasks/create) * [Partial update video information](/v1.3/api-reference/videos/partial-update-video-information) * [Make any-to-video search requests](/v1.3/api-reference/any-to-video-search/make-search-request) * [Retrieve a specific page of search results](/v1.3/api-reference/any-to-video-search/retrieve-page). ## June 24, 2025 ### Organizations now support self-service creation and are available to all users The Organizations feature has been improved to offer better accessibility and more control. This update includes several key changes: * **Self-service creation**: You can now create and manage your organization directly from the [Playground](https://playground.twelvelabs.io). * **Available to all users**: The Organizations feature is no longer restricted to paid plans. * **Enhanced administrative features**: Administrators now have the ability to upgrade or downgrade their organization's plan. For detailed instructions on setting up and managing your organization, see the updated [Organizations](/docs/advanced/organizations) section. If you previously had an organization set up through our sales team, your existing organization remains unchanged. ## June 13, 2025 ### Marengo 2.6 and API v1.2 have been deprecated The Marengo video understanding model version 2.6 has been deprecated. To ensure your applications remain compatible, please update them to use Marengo version 2.7. Additionally, the API version 1.2 has also been deprecated. You must update your applications to utilize API version 1.3. ## June 4, 2025 ### The Generate API has been renamed to the Analyze API The Generate API has been renamed to the Analyze API to more accurately reflect its purpose of analyzing videos to generate text. This update includes changes to specific API endpoints and SDK methods, outlined below. You can continue using the Generate API until July 30, 2025. After this date, the Generate API will be deprecated, and you must transition to the Analyze API. **Changes to the API**: * The `/generate` endpoint is now the `/analyze` endpoint. * The `/gist` endpoint remains unchanged. * The `/summarize` endpoint remains unchanged. **Changes to SDKs**: The `generate` prefix has been removed from method names, and the methods below have been renamed as follows: * `generate.gist` is now `gist` * `generate.summarize` is now `summarize` * `generate.text` is now `analyze` * `generate.text_stream` is now `analyze_stream` (Python) * `generate.textStream` is now `analyzeStream` (Node.js) To maintain compatibility, update your API calls and SDK methods to the new names before July 30, 2025. For additional details, refer to the following resources: * [API Reference](/api-reference/analyze-videos) * [Python SDK Reference](/sdk-reference/python/analyze-videos) * [Node.js SDK Reference](/sdk-reference/node-js/analyze-videos) ## May 16, 2025 ### Increased prompt length The maximum prompt length for the [`/analyze`](/api-reference/analyze-videos/sync-analysis) and [`/summarize`](/api-reference/analyze-videos/summarize) endpoints is now 2,000 tokens. ## May 9, 2025 ### Support for retrieving transcriptions The platform now allows you to retrieve transcriptions for your videos. This update adds new functionality without breaking existing code and affects the following endpoints: * `/indexes/:index-id/videos/:video-id`: Set the new `transcription` query parameter to `true` to retrieve transcriptions. The platform will include a field named `transcription` in the response. It's an array of objects, each consisting of the time range and spoken words for that segment. For more details, see the [Retrieve video information](/api-reference/videos/retrieve) page. * `/search`: The response now includes a field named `transcription` for each match found. It's a string that contains the transcription of the spoken words in the video. For more details, see the [Make any-to-any search requests](/api-reference/any-to-video-search/make-search-request) page. ## April 15, 2025 ### New embedding retrieval options The Embed API now provides more control over the types of embeddings you retrieve. This is a breaking change that requires updating your code and affects the following endpoints: * `/indexes/:index-id/videos/:video-id`: The `embed` parameter has been deprecated and replaced with the new `embedding_option` parameter, allowing you to retrieve specific types of embeddings. For details, see the [Retrieve video information](/api-reference/videos/retrieve) page. * `/embed/tasks/:task_id`: You can now use the optional `embedding_option` parameter to specify which types of embeddings to retrieve. For details, see the [Retrieve video embeddings](/v1.3/api-reference/create-embeddings-v1/video-embeddings/retrieve-video-embeddings) page. In the responses from both endpoints, each segment now includes a new field named `embedding_option` located at `video_embedding.segments[].embedding_option`, which identifies the type of embedding as either "visual-text" or "audio." > **Note** > > If you use the TwelveLabs SDKs, ensure you have updated to the latest version. ## March 18, 2025 ### User-provided transcriptions have been deprecated The [`POST`](/v1.3/api-reference/upload-content/tasks/create) method of the `/tasks` endpoint no longer processes transcription data submitted through the following parameters: * `provide_transcription` * `transcription_file` * `transcription_url` Note that the platform will not return an error if you include these parameters in a request. ## March 3, 2025 ### Introducing the Organizations feature TwelveLabs is excited to announce the launch of the Organizations feature, available exclusively for Enterprise customers. This feature enables the sharing of indexes, videos, and S3 integrations across the organization. Enterprise customers interested in setting up an organization can do so by contacting our sales team at [sales@twelvelabs.io](mailto:sales@twelvelabs.io). Once the organization is configured, administrators will be able to invite team members and start collaborating. For more details about this feature, see the [Organizations](/docs/advanced/organizations) page. ## February 11, 2025 ### Pegasus 1.2 has been released TwelveLabs announces the official release of the Pegasus 1.2 video understanding model. For details on the new features and improvements in this version, refer to this blog post: [Introducing Pegasus 1.2: An Industry-Grade Video Language Model for Scalable Applications](https://www.twelvelabs.io/blog/introducing-pegasus-1-2). Note that you can no longer use Pegasus 1.1 to create new indexes, and this version will be discontinued on February 25, 2025. All existing Pegasus 1.1 indexes will automatically be upgraded to Pegasus 1.2 on a rolling basis. No manual intervention is required for this migration process, and all indexes will utilize Pegasus 1.2 upon completion. ## January 13, 2025 ### Pegasus 1.2 public preview TwelveLabs announces the public preview release of Pegasus 1.2, our latest video understanding model. **Key improvements**: The new model offers significant improvements over Pegasus 1.1: * Extended video processing capacity from 30 minutes to 1 hour per video. * Enhanced performance across video-language tasks compared to Pegasus 1.1 and other models of the same size. * More granular visual comprehension of objects, on-screen text, and numerical content. * More accurate temporal grounding and timestamp identification. For example, you can ask questions about the timestamps of certain events. During the preview phase, the model is available only for new and sample indexes. Existing Pegasus 1.1 indexes remain fully supported. All current indexes will be automatically migrated to Pegasus 1.2 at no cost during the official release (date to be announced). Note that the model may produce occasional errors or hallucinations. For support or feedback, contact [support@twelvelabs.io](mailto:support@twelvelabs.io). ## December 2, 2024 TwelveLabs is proud to introduce the following new features and improvements: * **Marengo 2.7**: This new version of the Marengo video understanding engine improves accuracy and performance in the following areas: * Multimodal processing that combines visual, audio, and text elements. * Fine-grained image-to-video search: detect brand logos, text, and small objects (as small as 10% of the video frame). * Improvement in motion search capability. * Counting capabilities. * More nuanced audio comprehension: music, lyrics, sound, and silence. For more details on the new features and improvements in this version, refer to this blog post: [Introducing Marengo 2.7: Pioneering Multi-Vector Embeddings for Advanced Video Understanding](https://www.twelvelabs.io/blog/introducing-marengo-2-7). * Simplified modalities: * `visual`: includes objects, actions, text OCR, logos. * `audio`: includes speech, music, and ambient sounds. * `conversation` has been deprecated. * `text_in_video` and `logo` are now part of `visual`. * Streamlined endpoint structure: Several endpoints and parameters have been deprecated, removed, or renamed. > **Notes** > > * The 1.3 version of the API version only supports Marengo 2.7. > * Marengo 2.7 generates embeddings that are not backward compatible. You must reindex all your videos and regenerate all your embeddings with Marengo 2.7. > * The audio search feature in Marengo 2.7 works best with full-sentence queries. Short queries may yield suboptimal results. This limitation is temporary and will be addressed in a future release. # Version 1.2 If you have used the 1.1.2 version of the API, please refer to the following section for important information regarding the changes. ## November 12, 2024 ### Improvements * **Cloud-to-cloud Integrations API**: The API has been updated to provide a more intuitive experience. The `/tasks/transfers` endpoint will be deprecated. Use the following endpoints instead: * Import videos * Retrieve import status * Retrieve import logs > **Note** > > Cloud-to-cloud integrations now require a paid plan. If you're on the Free plan, you can find information on upgrading your plan in the  [Upgrade your plan](/docs/get-started/manage-your-plan#upgrade-your-plan) section. ## November 5, 2024 ### Improvements * **Embed API**: The structure of the responses has been streamlined across all endpoints to provide a more consistent and intuitive experience: * **Standardized object naming**: * The `video_embeddings` field has been renamed to `video_embedding`. * The `video_embedding` object now encapsulates the embeddings, related metadata, and additional information. * **Enhanced response structure**: * The embedding vectors are now nested under an array named `segments`. * The `metadata` objects have been moved under their respective parent embedding objects. * The `is_success` boolean has been removed. * **Affected endpoints**: * [All video embedding endpoints](/api-reference/create-embeddings-v2) the endpoint for creating video embedding tasks. * [Create embeddings for text, image, and audio](/api-reference/create-embeddings-v1). * [Retrieve video information](/api-reference/videos/retrieve). * **Embed API**: You can now retrieve vector embeddings for any indexed video by setting `embed=true` in your [`GET`](/api-reference/videos/retrieve) `/indexes/{index-id}/videos/{video-id}` requests. ## October 24, 2024 ### New features * **Embed API**: * You can create image and audio embeddings in addition to its existing video and text capabilities. See the [Create embeddings](/docs/guides/create-embeddings) page for details. * You can now retrieve a list of the video embedding tasks in your account by invoking the [`GET`](/v1.3/api-reference/create-embeddings-v1/video-embeddings/list-video-embedding-tasks) method of the `/embed/tasks` endpoint. ## July 7, 2024 ### New features * **Pegasus 1.1**: The 1.1 version of the Pegasus video understanding engine has been released, introducing the following enhancements: * Improved model accuracy for video description and question-answering. * Fine-grained visual understanding and instruction following. * Streaming support when generating open-ended text. For details, refer to the [Streaming responses](/docs/guides/analyze-videos-and-images/videos) section. * Increased maximum prompt length to 375 tokens. * Extended maximum video duration to 30 minutes. > **Note** > > Effective July 8, 2024, Pegasus 1.0 is no longer supported. All existing indexes created with Pegasus 1.0 will be automatically upgraded to Pegasus 1.1. No manual intervention is required for this migration process, and all indexes will utilize Pegasus 1.1 upon completion. ## June 18, 2024 ### New features * **Image-to-Video Search API**: TwelveLabs is proud to introduce the Image-to-Video Search API. This new API allows you to find semantically related video segments by providing an image as a query. The platform identifies similar content within videos. To get started with the Image-to-Video Search API, refer to the [Search](/docs/guides/search)page. ## May 15, 2024 ### New features * **Embed API**: TwelveLabs is proud to introduce the Embed API. You can use this new API to create multimodal embeddings that are contextual vector representations for your videos and text. You can utilize multimodal embeddings in various downstream tasks, including but not limited to training custom multimodal models for applications such as clustering, classification, search, recommendation, and anomaly detection. See the [Create embeddings](/docs/guides/create-embeddings) page for details. ## March 12, 2024 ### New features * TwelveLabs is proud to introduce the new versions of its video understanding models: * **Marengo 2.6**: A new state-of-the-art (SOTA) multimodal foundation model capable of performing any-to-any search tasks, including Text-To-Video, Text-To-Image, Text-To-Audio, Audio-To-Video, Image-To-Video, and more. Note that the platform currently supports text-to-video search and classification features. Other modalities will be supported in a future release. This model represents a significant leap in video understanding technology, enabling more intuitive and comprehensive search capabilities across various media types. For an overview of the new features and improvements in this version, refer to this blog post: [Introducing Marengo 2.6: A New State-of-the-Art Video Foundation Model for Any-to-Any Search](https://www.twelvelabs.io/blog/introducing-marengo-2-6). * **Pegasus 1.0 beta**: This version of the model provides fine-grained video descriptions, summaries, and question-answering capabilities. For an overview of the new features and improvements in this version, refer to this blog post: [Pegasus-1 Open Beta: Setting New Standards in Video-Language Modeling](https://www.twelvelabs.io/blog/upgrading-pegasus-1). * The platform now supports search queries in multiple languages. For a complete list of supported languages, refer to the [Supported languages](/docs/supported-languages) page. ### Updates * You can now enable the Pegasus and Marengo video understanding engines on the same index. ## February 15, 2024 ### New features * You can now tune the temperature to control the randomness of the text output generated by the [`/summarize`](/api-reference/analyze-videos/summarize) and [`/generate`](/api-reference/analyze-videos/sync-analysis) endpoints. See the [Tune the temperature](/docs/guides/analyze-videos-and-images/tune-the-temperature) page for details. ## October 30, 2023 ### New features Version 1.2 of the TwelveLabs Video Understanding Platform introduces the following new features: * The alpha version of the Pegasus video understanding engine has been released. You can now use it to [analyze videos](/docs/guides/analyze-videos-and-images). * You can now upload videos from external providers. Currently, only YouTube is supported as an external provider, but we will add support for additional providers in the future. ### Updates This section lists the differences between version 1.1.2 and version 1.2 of the TwelveLabs Video Understanding API. * When you make an API call, make sure that you specify the `1.2` version in the URL. The URL should look similar to the following one: `https://api.twelvelabs.io/v1.2/{resource}/{path_parameters}?{query_parameters}`. For more details, see the [Call an endpoint](/reference/api-reference#call-an-endpoint) section. * To enable the utilization of multiple engines for an index, the following changes have been made: * **POST** `/indexes`: The `engine_id` and `indexing_options` parameters of the request have been deprecated. Instead, you can now define the engine configuration as a list of objects. See the [Create an index](/reference/create-index) page for details. * **GET** `/indexes/{index_id}`: The `engine_id` field in the response has been superseded by an array of objects named `engines`. See the [Retrieve an index](/api-reference/indexes/retrieve) page for details. * **GET** `/indexes`: * The `engine_id` field in the response has been superseded by an array of objects named `engines`. See the [List indexes](/api-reference/indexes/list) page for details. * The `engine_family` query parameter has been introduced, allowing you to filter by engine family. * The `index_options` query parameter has been marked for deprecation. You can still use it in this version of the API, but it will be deprecated in a future release. Instead, use `engine_options` or `engine_family`. * **GET** `/engines`: The `allowed_index_option` field in the response has been renamed to `allowed_engine_options`. * **GET**`/engines/{engine-id}` The `allowed_index_option` field in the response has been renamed to `allowed_engine_options`. See the `Retrieve an engine page` for details. * The `/search` and `/search/{page-token}` endpoints no longer return the `conversation_option`, `search_options`, and `query` fields. # Version 1.1.2 If you have used the 1.1.1 version of the API, please refer to the following section for important information regarding the changes. ## Improvements To further improve the usability of the `/classify` endpoint, the following changes have been made: * The endpoint now allows you to classify a set of videos. The `video_id` parameter has been deprecated and now you must pass an array of strings named `video_ids` instead. Each element of the array represents the unique identifier of a video you want to classify. * The `threshold` field in the request is now an object, and you can use it to filter based on the following criteria: * The confidence level that a video matches the specified class * The confidence level that a clip matches the specified class. * The duration ratio, which is the sum of the lengths of the matching video clips inside a video divided by the total length of the video. For details, see the `Filtering > Content classification` page. * The endpoint now supports pagination. * The duration-weighted score has been deprecated. When setting the `show_detailed_score` parameter to `true`, the platform now returns the maximum, average, and normalized scores. # Version 1.1.1 If you have used the 1.1 version of the API, please refer to the following sections for important information regarding the changes. ## New features Version 1.1.1 of the TwelveLabs Video Understanding Platform introduces the following new features: * Version 2.5 of the Marengo video understanding engine has been released. For details, see the [Video understanding engines](/docs/concepts/models) page. * The `/indexes`, `/search`, `/combined-search`, and `/classify` endpoints now support the ability to integrate with the [Playground](https://playground.twelvelabs.io), a sandbox environment that allows you to try out the features of the TwelveLabs Video Understanding Platform through an intuitive web page. * The platform now supports the ability to store the video you're uploading. For details, see the [Create a video indexing task](/reference/create-video-indexing-task) page. ## Improvements To further improve flexibility, usability, and clarity, the following changes have been made: * **Combined queries**: * You can now define global values for the `search_options` and `conversation_option` parameters for the entire request instead of per-query basis. For details, see the **Use combined queries** page. * The `/beta/search` endpoint has been renamed to `/combined-search`. * **Logo detection**: The `logo` add-on has been deprecated. To enable logo detection for an index, you must now use the `logo` indexing option. * **Conversation option**: The `transcription` conversation option has been renamed to `exact_match`. * **Classifying videos**: * The `labels` parameter has been renamed to `classes`. * The `threshold` field you can use to narrow down a response obtained from the platform is now of type `int`. For details, see the **API Reference > Classify a video** page. # Version 1.1 The introduction of new features and improvements in the 1.1 version of the TwelveLabs Video Understanding Platform has required changes to some endpoints. If you have used the 1.0 version of the API, please refer to the following sections for important information regarding the changes. ## New features Version 1.1 of the TwelveLabs Video Understanding Platform introduces the following new features: * **Classification of content:** You can now define a list of labels that you want to classify your videos into, and the new classification API endpoint will return the duration for which the specified labels have appeared in your videos and the confidence that each of the matches represents the label you've specified. * **Combined Queries:** The `1.1` version of the API introduces a new format of search queries named combined queries. A combined query includes any number of subqueries linked with any number of logical operators. Combined queries are executed in one API request. Combined queries support the following additional features: * **Negating a condition**: In addition to the existing `AND` operator, the platform now allows you to use the `NOT` operator to negate a condition. For example, this allows you to write a query that retrieves all the video clips in which someone is cooking but neither spaghetti nor lasagna is mentioned in the conversation. * **The THEN operator**: The platform now supports the `THEN` operator that allows you to specify that the platform must return only the results for which the order of the matching video clips is the same as the order of your queries. * **Time-proximity search**: The TwelveLabs Video Understanding API now allows you to use the `proximity` parameter to extend the lower and upper boundaries of each subquery. For example, this allows you to write a query that finds all car accidents that happened within a specific interval of time before someone wins a race. For details, see the **Use combined queries** page. * **Logo detection**: The platform can now detect brand logos. ## Updates This section lists the differences between version 1 and version 1.1 of the TwelveLabs Video Understanding API. * When you make an API call, make sure that you specify the `1.1` version in the URL. The URL should look similar to the following one: `https://api.twelvelabs.io./v1.1/{resource}/{path_parameters}?{query_parameters}`. For more details, see the [Call an endpoint](/reference/api-reference#call-an-endpoint) section. * The following methods now return a `200 OK` status code when the response is empty: * `[GET] /indexes` * `[GET] /tasks` * `[GET] /indexes/{index_id}/videos` * The `/tasks` endpoint is now a separate endpoint and is no longer part of the `/indexes` endpoint. The table below shows the changes made to each method of the `/tasks` endpoint: | 1.0 | 1.1 | | --------------------------------- | ------------------------- | | GET `/indexes/tasks` | GET `/tasks` | | POST `/indexes/tasks` | POST `/tasks` | | GET `/indexes/tasks/{task_id}` | GET `/tasks/{task_id}` | | DELETE `/indexes/tasks/{task_id}` | DELETE `/tasks/{task_id}` | | POST `/indexes/tasks/transfers` | POST `/tasks/transfers` | | GET `/indexes/tasks/status` | GET `/tasks/status` | * The `/indexes/tasks/{task_id}/video_id` endpoint has been deprecated. You can now retrieve the unique identifier of a video by invoking the GET method of the `/tasks/{task_id}` endpoint. The response will contain a field named `video_id`. * When an error occurs, the platform now follows the recommendations of the [RFC 9110](https://www.rfc-editor.org/rfc/rfc9110.html) standard. Instead of numeric codes, the platform now returns string values containing human-readable descriptions of the errors. The format of the error messages is as follows: * `code`: A string representing the error code. * `message`: A human-readable string describing the error, intended to be suitable for display in a user interface. * *(Optional)* `docs_url`: The URL of the relevant documentation page. For example, if you tried to list all the videos in an index and the unique identifier of the index you specified didn't exist, the `1.0` version of the API returned an error similar to the following one: ```json { "error_code": 201, "message": "ID 234234 does not exist" } ``` Now, when using the `1.1` version of the API, the error should look similar to the following one: ```json { "code": "parameter_not_provided", "message": "The index_id parameter is required but was not provided." } ``` For a list of error messages, see the [API Reference > Error codes](/reference/error-codes) page. * The `next_page_id` and `prev_page_id` fields of the `page_info` object have been renamed to `next_page_token` and `prev_page_id.` * The `type` field has been removed from all the responses. * When performing searches specifying multiple search options, the platform returns an object containing the confidence level that a specific video clip matched your search terms for each type of search. In version `v1.0`, this field was a dictionary named `module_confidence`. In version `v1.1`, this field is now named `module` and is of type `array`. * The POST method of the `/search/{page-token}` endpoint has been deprecated. To retrieve the subsequent pages, you must now call the GET method of the `/search/{page-token}` endpoint, passing it the unique identifier of the page you want to retrieve.