Classify a set of videos
This guide shows how you can classify a set of videos.
You can interact with the platform using one of the available SDKs or an HTTP client like requests
or axios
. This guide demonstrates how to use the SDKs, the recommended approach for most scenarios. If you need to make direct HTTP requests, refer to the API Reference > Classify a set of videos page for details.
Prerequisites
The examples in this guide assume the following:
- You’re familiar with the concepts that are described on the Platform overview page.
- You've already created an index, and the Marengo video understanding engine is enabled for this index.
- You've uploaded one or more videos, and the platform has finished indexing them. The unique identifiers of your videos are stored in a set of variables named
VIDEO_ID_1
toVIDEO_ID_X
. For details, see the Upload videos page.
Examples
This section shows several examples of classifying videos and specifying various parameters. For a description of each field in the request and response, see the API Reference > Classify a set of videos page.
Classify based on visual cues
To classify a set of videos based on visual cues, invoke the videos
method of the classify
object with the following parameters:
video_ids
: An array of strings representing the unique identifiers of the videos you wish to classify.classes
: An array of objects containing the names and definitions of the entities or actions that the platform must identify.options
: An array of strings specifying the sources of information the platform uses when categorizing a video. For details, see the search options page. This example uses["visual"]
.
from twelvelabs import TwelveLabs
VIDEO_IDS= [VIDEO_ID_1, VIDEO_ID_2, VIDEO_ID_2, VIDEO_ID_3]
CLASSES = [
{
"name": "DanceTok",
"prompts": [
"Dance tutorial",
"Dance group",
"Dance competition"
]
},
{
"name": "CookTok",
"prompts": [
"Cooking tutorial",
"Cooking ustensils review"
]
}
]
client = TwelveLabs("<YOUR_API_KEY">)
# Utility function to print a specific page
def print_page(page):
for data in page:
print(f"video_id={data.video_id}")
for cl in data.classes:
print(
f" name={cl.name} score={cl.score} duration_ratio={cl.duration_ratio}"
)
res = client.classify.videos(
video_ids=VIDEO_IDS,
options=["visual"],
classes=CLASSES,
)
print_page(res.data)
while True:
try:
next_page_data = next(res)
print_page(next_page_data)
except StopIteration:
break
import { TwelveLabs } from 'twelvelabs-js';
const VIDEO_IDS= [VIDEO_ID_1, VIDEO_ID_2, VIDEO_ID_3]
const CLASSES = [
{
name: 'DanceTok',
prompts: ['Dance tutorial', 'Dance group', 'Dance competition'],
},
{
name: 'CookTok',
prompts: ['Cooking tutorial', 'Cooking ustensils review'],
},
];
const client = new TwelveLabs({ apiKey: '<YOUR_API_KEY>' });
const res = await client.classify.videos({
videoIds: VIDEO_IDS,
options: ['visual'],
classes: CLASSES,
});
printPage(res);
let nextIndexPage = await res.next();
while (nextIndexPage !== null) {
printPage(nextIndexPage);
nextIndexPage = await res.next();
}
// Utility function to print a specific page
function printPage(result: any): void {
result.data.forEach((data: any) => {
console.log(`video_id=${data.videoId}`);
data.classes.forEach((cl: any) => {
console.log(
` name=${cl.name} score=${cl.score} durationRatio=${cl.durationRatio}`
);
});
});
}
The output should look similar to the following one:
video_id=665dbf08d22b3a3c97bf0bbe
name=DanceTok score=96.4 duration_ratio=0.99
video_id=665dbf30d22b3a3c97bf0bbf
name=DanceTok score=96.03 duration_ratio=0.99
video_id=665dbe2cd22b3a3c97bf0bbb
name=DanceTok score=95.75 duration_ratio=0.84
In this example, note that the data
array has three elements, and each element contains information about a specific video.
Retrieve information about each matching video clip
The following example code sets the include_clips
parameter to True
to specify that the platform should retrieve detailed information about each matching video clip:
# Utility function to print a specific page
def print_page(page):
for data in page:
print(f"video_id={data.video_id}")
for cl in data.classes:
print(
f" name={cl.name} score={cl.score} duration_ratio={cl.duration_ratio} clips={cl.clips.model_dump_json(indent=2)}"
)
res = client.classify.videos(
video_ids=VIDEO_IDS,
options=["visual"],
classes=CLASSES,
include_clips=True
)
const res = await client.classify.videos({
videoIds: VIDEO_IDS,
options: ['visual'],
classes: CLASSES,
includeClips: true
});
// Utility function to print a specific page
function printPage(result: any): void {
result.data.forEach((data: any) => {
console.log(`video_id=${data.videoId}`);
data.classes.forEach((cl: any) => {
console.log(
` name=${cl.name} score=${cl.score} durationRatio=${cl.durationRatio} clips=${JSON.stringify(cl.clips, null, 2)}`
);
});
});
}
The following example output has been truncated for brevity:
video_id=665dbf08d22b3a3c97bf0bbe
name=DanceTok score=96.4 duration_ratio=0.99 clips=[
{
"start": 148.84999999999997,
"end": 306.84999999999997,
"score": 85.95,
"option": "visual",
"prompt": "Dance competition",
"thumbnail_url": "https://project-one-thumbnail.s3.us-west-2.amazonaws.com/665dbf08d22b3a3c97bf0bbe/149.jpeg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAYRWJPOVHW6XGLKFV%2F20240619%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20240619T041830Z&X-Amz-Expires=604799&X-Amz-SignedHeaders=host&x-id=GetObject&X-Amz-Signature=c8d382e4249b31037e4b653926b988112edb15ddf90dd7dfc305ec9e4dd61f45"
},
{
"start": 476.84999999999997,
"end": 494.84999999999997,
"score": 85.91,
"option": "visual",
"prompt": "Dance competition",
"thumbnail_url": "https://project-one-thumbnail.s3.us-west-2.amazonaws.com/665dbf08d22b3a3c97bf0bbe/477.jpeg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAYRWJPOVHW6XGLKFV%2F20240619%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20240619T041830Z&X-Amz-Expires=604799&X-Amz-SignedHeaders=host&x-id=GetObject&X-Amz-Signature=9ae347c85f9d3b31cb2afd81fddcdffdd82e45e6f8dac0746a1a6220c84e0605"
},
video_id=665dbf30d22b3a3c97bf0bbf
name=DanceTok score=96.03 duration_ratio=0.99 clips=[
{
"start": 771.4333333332886,
"end": 873.5666666665386,
"score": 85.54,
"option": "visual",
"prompt": "Dance competition",
"thumbnail_url": "https://project-one-thumbnail.s3.us-west-2.amazonaws.com/665dbf30d22b3a3c97bf0bbf/772.jpeg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAYRWJPOVHW6XGLKFV%2F20240619%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20240619T041830Z&X-Amz-Expires=604799&X-Amz-SignedHeaders=host&x-id=GetObject&X-Amz-Signature=df7cff3781f5f83f585c9c32f392bba0fa71e01e4bc2d8339a84e5699e574225"
},
In this example, note that, for each video, the platform returns an array named clips
. Each element of these arrays contains detailed information about a single video clip.
Retrieve a detailed score for each class
The following example code sets the show_detailed_score
parameter to True
to specify that the platform must retrieve the maximum, average, and normalized scores for each class:
# Utility function to print a specific page
def print_page(page):
for data in page:
print(f"video_id={data.video_id}")
for cl in data.classes:
print(
f" name={cl.name} score={cl.score} duration_ratio={cl.duration_ratio} detailed_scores={cl.detailed_scores.model_dump_json(indent=2)}"
)
res = client.classify.videos(
video_ids=VIDEO_IDS,
options=["visual"],
classes=CLASSES,
show_detailed_score=True
)
const res = await client.classify.videos({
videoIds: VIDEO_IDS,
options: ['visual'],
classes: CLASSES,
showDetailedScore: true
});
// Utility function to print a specific page
function printPage(result: any): void {
result.data.forEach((data: any) => {
console.log(`video_id=${data.videoId}`);
data.classes.forEach((cl: any) => {
console.log(
` name=${cl.name} score=${cl.score} durationRatio=${cl.durationRatio} detailedScores=${JSON.stringify(cl.detailedScores, null, 2)}`
);
});
});
}
The output should look similar to the following one:
video_id=665dbf08d22b3a3c97bf0bbe
name=DanceTok score=96.4 duration_ratio=0.99 detailed_scores={
"max_score": 85.95,
"avg_score": 83.94,
"normalized_score": 100.0
}
video_id=665dbf30d22b3a3c97bf0bbf
name=DanceTok score=96.03 duration_ratio=0.99 detailed_scores={
"max_score": 85.54,
"avg_score": 82.9,
"normalized_score": 100.0
}
video_id=665dbd90d22b3a3c97bf0bb9
name=DanceTok score=95.94 duration_ratio=0.99 detailed_scores={
"max_score": 85.55,
"avg_score": 80.99,
"normalized_score": 100.0
}
Filtering and pagination
Understanding and utilizing filtering and pagination enhances your ability to find and organize information. See the sections below for more details:
Updated 19 days ago