Summaries, chapters, and highlights
This guide shows how you can generate summaries, chapters, and highlights from videos using pre-defined formats and optional prompts for customization.
- Summaries are concise overviews capturing the key points, adaptable into formats like paragraphs, emails, or bullet points based on your prompt.
- Chapters offer a chronological breakdown of the video, with timestamps, headlines, and summaries for each section.
- Highlights list the most significant events chronologically, including timestamps and brief descriptions.
Below are some examples of how to guide the platform in generating content tailored to your needs.
Prerequisites
-
To use the platform, you need an API key:
-
Ensure the TwelveLabs SDK is installed on your computer:
-
The videos you wish to use must meet the following requirements:
- Video resolution: Must be at least 360x360 and must not exceed 3840x2160.
- Aspect ratio: Must be one of 1:1, 4:3, 4:5, 5:4, 16:9, or 9:16.
- Video and audio formats: Your video files must be encoded in the video and audio formats listed on the FFmpeg Formats Documentation page. For videos in other formats, contact us at support@twelvelabs.io.
- Duration: Must be between 4 seconds and 60 minutes (3600s). In a future release, the maximum duration will be 2 hours (7,200 seconds).
- File size: Must not exceed 2 GB.
If you require different options, contact us at support@twelvelabs.io.
Complete example
This complete example shows how to create an index, upload a video, and generate summaries chapters and highlights. Ensure you replace the placeholders surrounded by <>
with your values.
Step-by-step guide
Python
Node.js
Import the SDK and initialize the client
Create a client instance to interact with the TwelveLabs Video Understanding platform.
Function call: You call the constructor of the TwelveLabs
class.
Parameters:
api_key
: The API key to authenticate your requests to the platform.
Return value: An object of type TwelveLabs
configured for making API calls.
Specify the index containing your videos
Indexes help you organize and search through related videos efficiently. This example creates a new index, but you can also use an existing index by specifying its unique identifier. See the Indexes page for more details on creating an index.
Function call: You call the index.create
function.
Parameters:
name
: The name of the index.models
: An object specifying your model configuration. This example enables the Pegasus video understanding model and thevisual
andaudio
model options.
Return value: An object containing, among other information, a field named id
representing the unique identifier of the newly created index.
Upload videos
To perform any downstream tasks, you must first upload your videos, and the platform must finish processing them.
Function call: You call the task.create
function. This starts a video indexing task, which is an object of type Task
that tracks the status of your video upload and indexing process.
Parameters:
index_id
: The unique identifier of your index.file
orurl
: The path or the publicly accessible URL of your video file.
Return value: An object of type Task
containing, among other information, the following fields:
video_id
: The unique identifier of your videostatus
: The status of your video indexing task.
Note
You can also upload multiple videos in a single API call. For details, see the Cloud-to-cloud integrations page.
Monitor the indexing process
The platform requires some time to index videos. Check the status of the video indexing task until it’s completed.
Function call: You call the task.wait_for_done
function.
Parameters:
sleep_interval
: The time interval, in seconds, between successive status checks. In this example, the method checks the status every five seconds.callback
: A callback function that the SDK executes each time it checks the status. Note that the callback function takes a parameter of typeTask
representig the video indexing task you’ve created in the previous step. Use it to display the status of your video indexing task.
Return value: An object containing, among other information, a field named status
representing the status of your task. Wait until the value of this field is ready
.
Generate summaries, chapters, and highlights
Function call: You call the generate.summarize
method.
Parameters:
video_id
: The unique identifier of the video for which you want to generate text.type
: The type of text you want to generate. It can take one of the following values: “summary”, “chapter”, or “highlight”.- (Optional)
prompt
: A string you can use to provide context for the summarization task. The maximum length of a prompt is 2,000 tokens. Example: “Generate chapters using casual and conversational language to match the vlogging style of the video.” - (Optional)
temperature
: A number that controls the randomness of the text. A higher value generates more creative text, while a lower value produces more deterministic text.
Return value: A string for summaries and a list of objects for chapters and highlights.