Search results
When you perform a search, the platform returns all the relevant matches. Filtering narrows the scope of your query. The platform allows you to filter your search results based on metadata or the level of confidence that the search results match your query.
Note
The examples on this page are specific to using simple queries. However, the principles demonstrated are similar when using combined queries.
Filtering search results based on metadata
To filter your search results based on metadata, use the filter
parameter.
The filter
parameter is of type Object
and can contain one or more of the following fields:
id
: An array of strings that filters your search results based on the specified video IDs.duration
: A field of typeObject
that filters your search results based on the duration of the video containing the segment that matches your query.width
: A numeric field that filters your search results based on width.height
: A numeric field that filters your search results based on height.size
: A numeric field that filters your search results based on the size of the video containing the segment that matches your query, expressed in bytes.title
: A string field that filters your search results based on the title of the video.
To indicate the relationship between a field and its value, you can use the exact match or comparison operators.
Exact match operator
The exact match operator matches only the results that equal the value you specify. The syntax is as follows: <field>: <value>
.
Comparison operators
Use the comparison operators (lte
and gte
) to match based on the arithmetic comparison. The syntax is as follows: <field>:{"gte": <value>, "lte": <value}
.
Filter composition
You can filter on multiple fields by adding the fields you want to filter on to the filter
parameter as shown in the Filter on size, width, and height section below.
Filtering on the level of confidence
The level of confidence represents the degree of similarity between your query and the search results. You can use the threshold
parameter to filter on the level of confidence. This allows you to narrow down a response obtained from the platform by retrieving only the most relevant results. The threshold
parameter can take one of the following values:
low
(this is the default value)medium
high
For a description of each field in the request and response, see the API Reference > Make any-to-video search request page.
Prerequisites
- You're familiar with the concepts that are described on the Platform overview page.
- You've already created an index, and the Marengo video understanding engine is enabled for this index.
- You've already uploaded a video, and the platform has finished indexing it.
Examples
Filter on a specific video ID
The following example code uses the id
field of the filter
parameter to filter on a specific video ID:
from twelvelabs import TwelveLabs
client = TwelveLabs(api_key="<YOUR_API_KEY>")
search_results = client.search.query(
index_id="<YOUR_INDEX_ID>",
query_text= "<YOUR_QUERY>",
options=["conversation"],
filter={"id":["<YOUR_VIDEO_ID>"]}
)
# Utility function to print a specific page
def print_page(page):
for clip in page:
print(
f"video_id={clip.video_id} score={clip.score} start={clip.start} end={clip.end} confidence={clip.confidence} metadata={clip.metadata}"
)
print_page(search_results.data)
while True:
try:
print_page(next(search_results))
except StopIteration:
break
import { TwelveLabs, SearchData } from 'twelvelabs-js';
const client = new TwelveLabs({ apiKey: '<YOUR_API_KEY>'});
let searchResults = await client.search.query({
indexId: '<YOUR_INDEX_ID>',
queryText: '<YOUR_QUERY>',
options: ['conversation'],
filter: { id: ['<YOUR_VIDEO_ID>'] },
});
printPage(searchResults.data);
while (true) {
const page = await searchResults.next();
if (page === null) break;
else printPage(page);
}
// Utility function to print a specific page
function printPage(searchData) {
(searchData as SearchData[]).forEach((clip) => {
console.log(
`video_id= ${clip.videoId} score=${clip.score} start=${clip.start} end=${clip.end} confidence=${clip.confidence} metadata=${JSON.stringify(clip.metadata)}`,
);
});
}
The following output has been truncated for brevity:
video_id=629deb6ff05c91527bc5c1c9 score=72.95 start=416.65 end=433.42 confidence=low metadata=[{'text': "Leave it alone, I have a gun, I'm walking backwards on the trail back to the camp, I'm walking down the trail and mom and two cubs so I am continually to walk backwards.", 'type': 'conversation'}]
video_id=629deb6ff05c91527bc5c1c9 score=68.81 start=201.74 end=279.86 confidence=low metadata=[{'text': "Mhm Well that puts an end to my tarpon fishing, get back now when you're playing with lions, it's important to never show fear chuck.", 'type': 'conversation'}]
video_id=629deb6ff05c91527bc5c1c9 score=68.74 start=436.88 end=490.25 confidence=low metadata=[{'text': 'The mom and the cubs keep following me and or walking at least as fast as I am. People get it. Oh, you, you, you, oh, oh no!', 'type': 'conversation'}]
Filter on multiple video IDs
The following example code uses the id
field of the filter
query parameter to filter on multiple video IDs:
from twelvelabs import TwelveLabs
client = TwelveLabs(api_key="<YOUR_API_KEY>")
search_results = client.search.query(
index_id="<YOUR_INDEX_ID>",
query_text= "<YOUR_QUERY>",
options=["visual"],
filter={"id":["<FIRST_VIDEO_ID>", "<SECOND_VIDEO_ID"]}
)
# Utility function to print a specific page
def print_page(page):
for clip in page:
print(
f"video_id={clip.video_id} score={clip.score} start={clip.start} end={clip.end} confidence={clip.confidence} metadata={clip.metadata}"
)
print_page(search_results.data)
while True:
try:
print_page(next(search_results))
except StopIteration:
break
import { TwelveLabs, SearchData } from 'twelvelabs-js';
const client = new TwelveLabs({ apiKey: '<YOUR_API_KEY>'});
let searchResults = await client.search.query({
indexId: '<YOUR_INDEX_ID>',
queryText: '<YOUR_QUERY>',
options: ['conversation'],
filter: { id: ['<FIRST_VIDEO_ID>','<SECOND_VIDEO_ID>'] },
});
printPage(searchResults.data);
while (true) {
const page = await searchResults.next();
if (page === null) break;
else printPage(page);
}
// Utility function to print a specific page
function printPage(searchData) {
(searchData as SearchData[]).forEach((clip) => {
console.log(
`video_id= ${clip.videoId} score=${clip.score} start=${clip.start} end=${clip.end} confidence=${clip.confidence} metadata=${JSON.stringify(clip.metadata)}`,
);
});
}
The following output has been truncated for brevity:
video_id=629deb6ff05c91527bc5c1c9 score=72.95 start=416.65 end=433.42 confidence=low metadata=[{'text': "Leave it alone, I have a gun, I'm walking backwards on the trail back to the camp, I'm walking down the trail and mom and two cubs so I am continually to walk backwards.", 'type': 'conversation'}]
video_id=629deb6ff05c91527bc5c1c9 score=68.81 start=201.74 end=279.86 confidence=low metadata=[{'text': "Mhm Well that puts an end to my tarpon fishing, get back now when you're playing with lions, it's important to never show fear chuck.", 'type': 'conversation'}]
video_id=629deb9df05c91527bc5c1cb score=68.74 start=436.88 end=490.25 confidence=low metadata=[{'text': 'The mom and the cubs keep following me and or walking at least as fast as I am. People get it. Oh, you, you, you, oh, oh no!', 'type': 'conversation'}]
Filter on size, width, and height
The example code below uses the size
, width
, and height
fields of the filter
parameter to return only the matches found in videos that meet all the following criteria:
- Size is greater than or equal to
50000000
bytes and less and equal to53000000
bytes. - Width is greater than or equal to
850
. - Height is greater than or equal to
400
and less and equal to500.
from twelvelabs import TwelveLabs
client = TwelveLabs(api_key="<YOUR_API_KEY>")
search_results = client.search.query(
index_id="<YOUR_INDEX_ID>",
query_text= "<YOUR_QUERY>",
options=["conversation"],
filter={
"size": {
"gte": 50000000, "lte": 53000000
},
"width":
{
"gte": 850
},
"height":
{
"gte": 400, "lte": 500
}
}
)
# Utility function to print a specific page
def print_page(page):
for clip in page:
print(
f"video_id={clip.video_id} score={clip.score} start={clip.start} end={clip.end} confidence={clip.confidence} metadata={clip.metadata}"
)
print_page(search_results.data)
while True:
try:
print_page(next(search_results))
except StopIteration:
break
import { TwelveLabs, SearchData } from 'twelvelabs-js';
const client = new TwelveLabs({ apiKey: '<YOUR_API_KEY>'});
let searchResults = await client.search.query({
indexId: '<YOUR_INDEX_ID>',
queryText: '<YOUR_QUERY>',
options: ['conversation'],
filter: {
size: {
gte: 50000000,
lte: 53000000,
},
width: {
gte: 850,
},
height: {
gte: 400,
lte: 500,
},
},
});
printPage(searchResults.data);
while (true) {
const page = await searchResults.next();
if (page === null) break;
else printPage(page);
}
// Utility function to print a specific page
function printPage(searchData) {
(searchData as SearchData[]).forEach((clip) => {
console.log(
`video_id= ${clip.videoId} score=${clip.score} start=${clip.start} end=${clip.end} confidence=${clip.confidence} metadata=${JSON.stringify(clip.metadata)}`,
);
});
}
The following output was truncated for brevity:
video_id=629deb9df05c91527bc5c1cb score=88.4 start=930.44 end=937.75 confidence=high metadata=Bear encounters can be scary, exciting. And even funny. It all really depends on the circumstances you're in.
video_id=629deb9df05c91527bc5c1cb score=84.74 start=937.79 end=947.56 confidence=high metadata=While bears aren't necessarily always aggressive creatures, it's probably best to keep your distance. Should you ever run into one in the wild?
video_id=629deb9df05c91527bc5c1cb score=84.73 start=788.34 end=805.3 confidence=high metadata= If you were driving through the forest and saw a bear up ahead, what would you do? Surely you'd feel pretty safe knowing you were inside your car. But what if that bear came up to you and began peeking through your window. That's exactly the situation.
Filter on custom metadata
The example code below filters on a custom field named views
of type integer
. The platform will return only the results found in the videos for which the value of the views
field equals 120000
. For details about specifying custom metadata, see the Provide custom metadata page.
from twelvelabs import TwelveLabs
client = TwelveLabs(api_key="<YOUR_API_KEY>")
search_results = client.search.query(
index_id="<YOUR_INDEX_ID>",
query_text= "<YOUR_QUERY>",
options=["conversation"],
filter = {
"views": 120000
}
)
# Utility function to print a specific page
def print_page(page):
for clip in page:
print(
f"video_id={clip.video_id} score={clip.score} start={clip.start} end={clip.end} confidence={clip.confidence} metadata={clip.metadata}"
)
print_page(search_results.data)
while True:
try:
print_page(next(search_results))
except StopIteration:
break
import { TwelveLabs, SearchData } from 'twelvelabs-js';
const client = new TwelveLabs({ apiKey: '<YOUR_API_KEY>'});
let searchResults = await client.search.query({
indexId: '<YOUR_INDEX_ID>',
queryText: '<YOUR_QUERY>',
options: ['conversation'],
filter: {
views: 120000,
},
});
printPage(searchResults.data);
while (true) {
const page = await searchResults.next();
if (page === null) break;
else printPage(page);
}
// Utility function to print a specific page
function printPage(searchData) {
(searchData as SearchData[]).forEach((clip) => {
console.log(
`video_id= ${clip.videoId} score=${clip.score} start=${clip.start} end=${clip.end} confidence=${clip.confidence} metadata=${JSON.stringify(clip.metadata)}`,
);
});
}
The following output was truncated for brevity:
video_id=65d5fbad48db9fa780cb415c score=65.0 start=347.04 end=363.929 confidence=low metadata=[{'type': 'conversation', 'text': "And with that goal, Lionel Messi becomes Argentina's all time leading scorer. "}]
Filtering on the level of confidence
The following example code specifies that the minimum level of confidence shouldn't be lower than medium
:
from twelvelabs import TwelveLabs
client = TwelveLabs(api_key="<YOUR_API_KEY>")
search_results = client.search.query(
index_id="<YOUR_INDEX_ID>",
query_text= "<YOUR_QUERY>",
options=["conversation"],
threshold='medium'
)
# Utility function to print a specific page
def print_page(page):
for clip in page:
print(
f"video_id={clip.video_id} score={clip.score} start={clip.start} end={clip.end} confidence={clip.confidence} metadata={clip.metadata}"
)
print_page(search_results.data)
while True:
try:
print_page(next(search_results))
except StopIteration:
break
import { TwelveLabs, SearchData } from 'twelvelabs-js';
const client = new TwelveLabs({ apiKey: '<YOUR_API_KEY>'});
let searchResults = await client.search.query({
indexId: '<YOUR_INDEX_ID>,
queryText: '<YOUR_QUERY>',
options: ['conversation'],
threshold: 'medium',
});
printPage(searchResults.data);
while (true) {
const page = await searchResults.next();
if (page === null) break;
else printPage(page);
}
// Utility function to print a specific page
function printPage(searchData) {
(searchData as SearchData[]).forEach((clip) => {
console.log(
`video_id= ${clip.videoId} score=${clip.score} start=${clip.start} end=${clip.end} confidence=${clip.confidence} metadata=${JSON.stringify(clip.metadata)}`,
);
});
The following example output was truncated for brevity:
video_id=63b2c707ce36463e0199c906 score=80.28 start=25.421875 end=31.375 confidence=medium metadata=[{'type': 'visual'}]
video_id=639963a1ce36463e0199c8c7 score=75.25 start=266.765625 end=270.5 confidence=medium metadata=[{'type': 'visual'}]
video_id=639963a1ce36463e0199c8c7 score=75.09 start=119.796875 end=125.71875 confidence=high metadata=[{'type': 'visual'}]
video_id=639963a1ce36463e0199c8c7 score=73.96 start=300.475 end=304.15625 confidence=high metadata=[{'type': 'visual'}]
In this example, note that the data
array is composed of a list of objects. Each object corresponds to a matching video clip and has a field named confidence
whose value is either medium
or high
.
Updated 17 days ago