Classify all the videos within an index
This guide shows how you can classify the content of all the videos within an index.
You can interact with the platform using one of the available SDKs or an HTTP client like requests or axios. This guide demonstrates how to use the SDKs, the recommended approach for most scenarios. If you need to make direct HTTP requests, refer to the API Reference > Classify all the videos within an index page for details.
Prerequisites
- You’re familiar with the concepts that are described on the Platform overview page.
- You've already created an index, and the Marengo video understanding engine is enabled for this index.
- You've uploaded one or more videos, and the platform has finished indexing them. For details, see the Upload videos page.
- The unique identifier of the index to which you've uploaded your videos is stored in a variable named
INDEX_ID.
Examples
This section shows several examples of classifying videos and specifying various parameters. For a description of each field in the request and response, see the API Reference > Classify all the videos in an index page.
Classify based on visual cues
To classify all the videos within an index, invoke the index method of the classify object with the following parameters:
index_id: A string representing the unique identifier of the index containing the videos you wish to classify.classes: An array of objects containing the names and definitions of the entities or actions that the platform must identify.options: An array of strings specifying the sources of information the platform uses when categorizing a video. For details, see the search options page. This example uses["visual"].
from twelvelabs import TwelveLabs
CLASSES = [
{
"name": "DanceTok",
"prompts": [
"Dance tutorial",
"Dance group",
"Dance competition"
]
},
{
"name": "CookTok",
"prompts": [
"Cooking tutorial",
"Cooking ustensils review"
]
}
]
client = TwelveLabs(api_key="<YOUR_API_KEY>")
# Utility function to print a specific page
def print_page(page):
for data in page:
print(f"video_id={data.video_id}")
for cl in data.classes:
print(
f" name={cl.name} score={cl.score} duration_ratio={cl.duration_ratio}"
)
res = client.classify.index(
index_id=INDEX_ID,
options=["visual"],
classes=CLASSES,
)
print_page(res.data)
while True:
try:
next_page_data = next(res)
print_page(next_page_data)
except StopIteration:
break
const res = await client.classify.index({
indexId: INDEX_ID,
options: ['visual', 'conversation'],
classes: CLASSES,
showDetailedScore: true
});
The output should look similar to the following one:
video_id=665dbf08d22b3a3c97bf0bbe
name=DanceTok score=96.4 duration_ratio=0.99
video_id=665dbf30d22b3a3c97bf0bbf
name=DanceTok score=96.03 duration_ratio=0.99
video_id=665dbe2cd22b3a3c97bf0bbb
name=DanceTok score=95.75 duration_ratio=0.84
In the example output above, note that the data array has three elements, and each element contains information about a specific video.
Retrieve information about each matching video clip
The following example code sets the include_clips parameter to True to specify that the platform should retrieve detailed information about each matching video clip:
# Utility function to print a specific page
def print_page(page):
for data in page:
print(f"video_id={data.video_id}")
for cl in data.classes:
print(
f" name={cl.name} score={cl.score} duration_ratio={cl.duration_ratio} clips={cl.clips.model_dump_json(indent=2)}"
)
res = client.classify.index(
index_id=INDEX_ID,
options=["visual"],
classes=CLASSES,
include_clips=True
)
const res = await client.classify.index({
indexId: INDEX_ID,
options: ['visual'],
classes: CLASSES,
includeClips: true
});
// Utility function to print a specific page
function printPage(result: any): void {
result.data.forEach((data: any) => {
console.log(`video_id=${data.videoId}`);
data.classes.forEach((cl: any) => {
console.log(
` name=${cl.name} score=${cl.score} durationRatio=${cl.durationRatio} clips=${JSON.stringify(cl.clips, null, 2)}`
);
});
});
}
The following example output has been truncated for brevity:
video_id=665dbf08d22b3a3c97bf0bbe
name=DanceTok score=96.4 duration_ratio=0.99 clips=[
{
"start": 148.84999999999997,
"end": 306.84999999999997,
"score": 85.95,
"option": "visual",
"prompt": "Dance competition",
"thumbnail_url": "https://project-one-thumbnail.s3.us-west-2.amazonaws.com/665dbf08d22b3a3c97bf0bbe/149.jpeg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAYRWJPOVHW6XGLKFV%2F20240619%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20240619T041830Z&X-Amz-Expires=604799&X-Amz-SignedHeaders=host&x-id=GetObject&X-Amz-Signature=c8d382e4249b31037e4b653926b988112edb15ddf90dd7dfc305ec9e4dd61f45"
},
{
"start": 476.84999999999997,
"end": 494.84999999999997,
"score": 85.91,
"option": "visual",
"prompt": "Dance competition",
"thumbnail_url": "https://project-one-thumbnail.s3.us-west-2.amazonaws.com/665dbf08d22b3a3c97bf0bbe/477.jpeg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAYRWJPOVHW6XGLKFV%2F20240619%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20240619T041830Z&X-Amz-Expires=604799&X-Amz-SignedHeaders=host&x-id=GetObject&X-Amz-Signature=9ae347c85f9d3b31cb2afd81fddcdffdd82e45e6f8dac0746a1a6220c84e0605"
},
video_id=665dbf30d22b3a3c97bf0bbf
name=DanceTok score=96.03 duration_ratio=0.99 clips=[
{
"start": 771.4333333332886,
"end": 873.5666666665386,
"score": 85.54,
"option": "visual",
"prompt": "Dance competition",
"thumbnail_url": "https://project-one-thumbnail.s3.us-west-2.amazonaws.com/665dbf30d22b3a3c97bf0bbf/772.jpeg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAYRWJPOVHW6XGLKFV%2F20240619%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20240619T041830Z&X-Amz-Expires=604799&X-Amz-SignedHeaders=host&x-id=GetObject&X-Amz-Signature=df7cff3781f5f83f585c9c32f392bba0fa71e01e4bc2d8339a84e5699e574225"
},
In the example output above, note that, for each video, the platform returns an array named clips. Each element of these arrays contains detailed information about a single video clip.
Retrieve a detailed score for each class
The following example code sets the show_detailed_score parameter to True to specify that the platform must retrieve the maximum, average, and normalized scores for each class:
# Utility function to print a specific page
def print_page(page):
for data in page:
print(f"video_id={data.video_id}")
for cl in data.classes:
print(
f" name={cl.name} score={cl.score} duration_ratio={cl.duration_ratio} detailed_scores={cl.detailed_scores.model_dump_json(indent=2)}"
)
res = client.classify.index(
index_id=INDEX_ID,
options=["visual"],
classes=CLASSES,
show_detailed_score=True
)
const res = await client.classify.index({
indexId: INDEX_ID,
options: ['visual'],
classes: CLASSES,
showDetailedScore: true
});
// Utility function to print a specific page
function printPage(result: any): void {
result.data.forEach((data: any) => {
console.log(`video_id=${data.videoId}`);
data.classes.forEach((cl: any) => {
console.log(
` name=${cl.name} score=${cl.score} durationRatio=${cl.durationRatio} detailedScores=${JSON.stringify(cl.detailedScores, null, 2)}`
);
});
});
}
The output should look similar to the following one:
video_id=665dbf08d22b3a3c97bf0bbe
name=DanceTok score=96.4 duration_ratio=0.99 detailed_scores={
"max_score": 85.95,
"avg_score": 83.94,
"normalized_score": 100.0
}
video_id=665dbf30d22b3a3c97bf0bbf
name=DanceTok score=96.03 duration_ratio=0.99 detailed_scores={
"max_score": 85.54,
"avg_score": 82.9,
"normalized_score": 100.0
}
video_id=665dbd90d22b3a3c97bf0bb9
name=DanceTok score=95.94 duration_ratio=0.99 detailed_scores={
"max_score": 85.55,
"avg_score": 80.99,
"normalized_score": 100.0
}
Filtering and pagination
Understanding and utilizing filtering and pagination enhances your ability to find and organize information. See the sections below for more details:
Updated about 1 month ago