Typical workflows

This page provides an overview of common workflows for using the TwelveLabs Video Understanding Platform. Each workflow consists of a series of steps, with links to detailed documentation for each step.

All workflows involving uploading video content to the platform require asynchronous processing. You must wait for the video processing to complete before proceeding with the subsequent steps.

Prerequisites

  • To use the platform, you need an API key.
    • If you already have an account:
      1. Go to the API Key page.
      2. Click the Copy icon next to your key.
    • If you don’t have an account:
      1. Sign up for a free account.
      2. Follow the steps above to retrieve your API key.

Search

Follow the steps in this section to search through your video content and find specific moments, scenes, or information. For an interactive implementation using the Python SDK, see the Quickstart Search Jupyter notebook.

Steps:

  1. Create an index, enabling the Marengo video understanding engine.
  2. Upload videos and monitor the processing.
  3. Perform a search request, using text or images as queries.
Notes
  • The search scope is an individual index.
  • Results support pagination, filtering, sorting, and grouping.

Generate text from videos

Follow the steps in this section to generate texts based on your videos. For an interactive implementation using the Python SDK, see the Quickstart Generate Jupyter notebook.

Steps:

  1. Create an index, enabling the Pegasus video understanding engine.
  2. Upload videos and monitor the processing.
  3. Depending on your use case, generate one of the following:
Note

When generating open-ended texts, the platform supports streaming responses.

Classify

Deprecation notice

The Classify API has been deprecated on Feb 28, 2025.

Recommended alternative: Update to the 1.3 version of the API and use the Pegasus video understanding model to classify videos.

Resources: Migration guide > Use Pegasus to classify videos.

Create text, image, and audio embeddings

This workflow guides you through creating embeddings for text.

Steps:

  1. Create text, image, and audio embeddings.
Note

Creating text embeddings is a synchronous process.

Create video embeddings

This workflow guides you through creating embeddings for videos. For an interactive implementation using the Python SDK, see the Quickstart Embed Jupyter notebook.

Steps:

  1. Upload a video and monitor the processing.
  2. Retrieve the embeddings.

Extract video data

Follow the steps in this guide to extract various types of data from your videos, such as transcriptions, on-screen text, logos, and thumbnails.

Steps:

  1. Create an index, enabling the Marengo video understanding engine.
  2. Upload videos and monitor the processing.
  3. Depending on your use case:
Built with