Migration guide

This guide shows how to migrate your applications to the 1.3 version of the API, which introduces significant improvements to the video understanding capabilities of the platform, simplified modalities, and a streamlined endpoint structure.

SDK version compatibility with API versions:

  • SDK 0.4.x and later supports API 1.3
  • SDK 0.3.x and earlier supports API 1.2
Note

All changes described in this guide apply to the 1.3 version of the API. Features marked as deprecated, renamed, or removed are unavailable in v1.3 but remain functional in v1.2 until further notice. The only exception is the Classify API, which has been deprecated in v1.2 on Feb 28, 2025.

What’s new in v1.3?

  • Marengo 2.7: A new version of the Marengo video understanding model has been released. The 1.3 version of the API version only supports Marengo 2.7. This new version improves accuracy and performance in the following areas:
    • Multimodal processing that combines visual, audio, and text elements.
    • Fine-grained image-to-video search: detect brand logos, text, and small objects (as small as 10% of the video frame).
    • Improvement in motion search capability.
    • Counting capabilities.
    • More nuanced audio comprehension: music, lyrics, sound, and silence.
  • Simplified modalities:
    • visual: includes objects, actions, text OCR, logos.
    • audio: includes speech, music, and ambient sounds.
    • conversation has been deprecated.
    • text_in_video and logo are now part of visual.
  • Streamlined endpoint structure: Several endpoints and parameters have been deprecated, removed, or renamed.
Note

This guide presents the changes to the API. Since the SDKs reflect the structure of the API, review the Migration examples section below and the relevant SDK reference sections to understand how these changes have been implemented:

Breaking changes

This section presents the changes that require updates to your code and includes the following subsections:

  • Global changes that affect multiple endpoints
  • Changes organized by endpoint and functionality (example: upload videos, manage indexes, etc.)

In the sections below, see the Required Action column for each change, then use the corresponding example in the Migration examples section to update your code.

Global changes

DescriptionRequired action
Marengo 2.7 generates embeddings that are not backward compatible.Reindex all your videos and regenerate all your embeddings with Marengo 2.7.
The 1.3 version of the API version only supports Marengo 2.7.Update the version strings in your code. For indexes, use “marengo2.7” instead of “marengo2.6”. For embeddings, use “marengo-retrieval-2.7” instead of “marengo-retrieval-2.6”. For examples of using Marengo 2.7, see the Create indexes and Create emeddings sections below.
The following parameters and fields in the responses have been renamed:

- engines → models
- engine_name → model_name
- engine_options → model_options		Update parameter names and response parsing in your code. For an example of using these parameters, see the Create indexes section below.
The previous modalities of [visual, conversation, text_in_video, logo] have been simplified to [visual, audio].Update parameter names and response parsing in your code:

- conversation -> audio
- Remove logo and text_in_video.

Note that, while logo and text_in_video have been deprecated, no functionality has been removed. To achieve the same outcome, see the Detect logos and Search for text shown in videos sections below.

NOTE: For indexes created with Marengo v2.6, the /indexes endpoints continue to return the legacy modality values (text_in_video, logo, and conversation). You cannot perform any downstream tasks on these indexes.

For examples of using the new values, see the Create indexes and Perform a search requests sections below.
The /search-v2 endpoint becomes the new /search endpoint.If you’re using an HTTP client to interact with the API, update all search requests to use the new /search endpoint, which now supports both text and image queries.
No action is required if you’re using one of the official SDKs.

Deprecated endpoints

EndpointRequired action
/classifyYou can no longer use this endpoint. TwelveLabs recommends classifying videos using the text generation capabilities of the platform. For details, see the Use Pegasus to classify videos section.
- /engines
- /engines/{engine-id}		Update your code, removing any calls to these endpoints.
- /indexes/{index-id}/videos/{video-id}/text-in-video
- /indexes/{index-id}/videos/{video-id}/logo
- /indexes/{index_id}/videos/{video_id}/thumbnail
- /indexes/{index-id}/videos/{video-id}/transcription		You can no longer use these endpoints. TwelveLabs recommends updating your code by replacing any calls to the specified endpoints with the alternative options provided below:

- Text in video: Search using text queries. For details, see the Search for text shown in videos section.
- Logos: Search using text or image queries to detect logos. For details see the Detect logos section.
- Thumbnails: The /search endpoint returns thumbnails for each results. You can use them in your application.
- Transcriptions: Search using text queries. For details, see the Search for text shown in videos section.
/search/combinedYou can no longer use this endpoint. TwelveLabs recommends updating your code to use the new /search endpoint instead of /search/combined.

Upload videos

MethodDescriptionRequired action
POST /tasksThe disable_video_stream parameter has been renamed to enable_video_stream. The default value is true.- Find and replace all occurences of disable_video_stream with enable_video_stream
- Review your logic - no need to set the enable_video_stream parameter if you want to enable streaming.
GET /tasksThe following parameters have been deprecated:

- id
- estimated_time		Update your code - you no longer can filter tasks based on these parameters.

Manage indexes

MethodDescriptionRequired action
GET /indexes/{index-id}The _id parameter has been deprecated.Update your code - you no longer can filter videos based on this parameter.

Manage videos

MethodDescriptionRequired action
GET /indexes/{index-id}/videosThe metadata parameter has been renamed to user_metadataUpdate your code to use user_metadata instead of metadata.
GET /indexes/{index-id}/videosThe metadata field in the response has been renamed to system_metadataUpdate your code to use system_metadata instead of metadata.
GET /indexes/{index-id}/videos/{video-id}The metadata field in the response has been renamed to system_metadataUpdate your code to use system_metadata instead of metadata.

MethodDescriptionRequired action
POST /searchThe conversation_option parameter has been deprecated.Remove conversation_option from your  search requests .
- POST /search
- GET /search/{page-token}		The page_info.page_expired_at field in the response has been renamed to page_info.page_expires_at.Update your code to use page_expires_at instead of page_expired_at.
- POST /search
- GET /search/{page-token}		The metadata and modules fields in the response have been deprecated.Update your code - the platform no longer returns these fields.

Generate text from video


MethodDescriptionRequired action
POST /generateThe stream parameter now defaults to true.Review your logic. Set the stream parameter to false to turn off streaming responses. Otherwise, the platform will stream responses.

Non-breaking changes

These changes add new functionality while maintaining backward compatibility.

Upload videos

EndpointDescription
- POST /tasks
- GET /tasks
- GET /tasks/{task-id		These endpoints now return a field named video_id with the unique identifier of your video.
GET /tasksThis method now accepts a query parameter named status. You can use it to filter your video indexing tasks by their status.

Migration steps

Migrating to v1.3 involves two main steps:

  1. Update your integration
  2. Update your code. Refer to the Migration Examples setion for details.

1. Update your integration

Choose the appropriate method based on how you interact with the TwelveLabs API:

  • Official SDKs: Install version 0.4.x or later.
  • HTTP client: Update your base URL.
$pip3 install twelvelabs --upgrade

2. Migration examples

Below are examples showing how to update your code for key breaking changes. Choose the examples matching your integration type.

Create indexes

Creating an index in version 1.3 includes the following key changes:

  • Renamed parameters: The parameters that previously began with engine* have now been renamed to model*.
  • Simplified modalities: The previous modalities of [visual, conversation, text_in_video, logo] have been simplified to [visual, audio].
  • Marengo version update: Use “marengo2.7” instead of “marengo2.6”.
1models = [
2        {
3          "name": "marengo2.7",
4          "options": ["visual", "audio"]
5        },
6        {
7            "name": "pegasus1.2",
8            "options": ["visual", "audio"]
9        }
10    ]
11index = client.index.create(
12    name="<YOUR_INDEX_NAME>",
13    models=models,
14    addons=["thumbnail"] # Optional
15)

Perform a search request

Performing a search request includes the following key changes:

  • Simplified modalities: The previous modalities of [visual, conversation, text_in_video, logo] have been simplified to [visual, audio].
  • Deprecated parameter: The conversation_option parameter has been deprecated.
  • Streamlined response: The metadata and modules fields in the response have been deprecated.
1search_results = client.search.query(
2  index_id="<YOUR_INDEX_ID>", 
3  query_text="<YOUR_QUERY>", 
4  options=["visual", "audio"]
5)

Create embeddings

Creating embeddings includes the following key changes:

  • Marengo version update: Use “Marengo-retrieval-2.7” instead of “Marengo-retrieval-2.6”.
  • Renamed parameter: The parameters that previously began with engine* have now been renamed to model*.

The following example creates a text embedding, but the principles demonstrated are similar for image, audio, and video embeddings:

1res = client.embed.create(
2  model_name="Marengo-retrieval-2.7",
3  text="<YOUR_TEXT>",
4)

Use Pegasus to classify videos

The Pegasus video understanding model offers flexible video classification through its text generation capabilities. You can use established category systems like YouTube video categories or IAB Tech Lab Content Taxonomy . You can also define custom categories for your specific needs.

The example below classifies a video based on YouTube’s video categories:

1res = client.generate.text(
2  video_id="<YOUR_VIDEO_ID>",
3  prompt="Classify this video using up to five labels from YouTube standard content categories. Provide the results in the JSON format."
4)

Detect logos

You can search for logos using text or image queries:

  • Text queries: For logos that include text (example: Nike)
  • Image queries: For logos without text (example: Apple’s apple symbol).

The following example searches for the Nike logo using a text query:

1search_results = client.search.query(
2  index_id="<YOUR_INDEX_ID>", 
3  query_text="Nike", 
4  options=["visual"]
5)

The following example searches for the Apple logo using an image query:

1search_results = client.search.query(
2    index_id="<YOUR_INDEX_ID>",
3    query_media_type="image",
4    query_media_url="https://logodownload.org/wp-content/uploads/2013/12/apple-logo-16.png,
5    options=["visual"]
6)

Search for text shown in videos

To search for text in videos, use text queries that target either on-screen text or spoken words in transcriptions rather than objects or concepts. The platform searches across both:

  • Text shown on screen (such as titles, captions, or signs)
  • Spoken words from audio transcriptions

Note that the platform may return both textual and visual matches. For example, searching for the word “smartphone” might return:

  • Segments where “smartphone” appears as on-screen text.
  • Segments where “smartphone” is spoken.
  • Segments where smartphones are visible as objects.

The example below finds all the segments where the word “innovation” appears as on-screen text or as a spoken word in transcriptions:

1search_results = client.search.query(
2  index_id="<YOUR_INDEX_ID>", 
3  query_text="Innovation", 
4  options=["visual"]
5)

Additional resources

Python SDK Reference
Node.js SDK Reference
API Reference
Discord
Built with