When you perform a search, the platform returns all the relevant matches. Filtering narrows the scope of your query. The platform allows you to filter your search results based on metadata or the level of confidence that the search results match your query.

📘
Note
The examples on this page are specific to using simple queries. However, the principles demonstrated are similar when using combined queries.

Filtering search results based on metadata

To filter your search results based on metadata, use the filter parameter.

The filter parameter is of type Object and can contain one or more of the following fields:

id: An array of strings that filters your search results based on the specified video IDs.
duration: A field of type Object that filters your search results based on the duration of the video containing the segment that matches your query.
width: A numeric field that filters your search results based on width.
height: A numeric field that filters your search results based on height.
size: A numeric field that filters your search results based on the size of the video containing the segment that matches your query, expressed in bytes.
title: A string field that filters your search results based on the title of the video.

To indicate the relationship between a field and its value, you can use the exact match or comparison operators.

Exact match operator

The exact match operator matches only the results that equal the value you specify. The syntax is as follows: <field>: <value>.

Comparison operators

Use the comparison operators (lte and gte) to match based on the arithmetic comparison. The syntax is as follows: <field>:{"gte": <value>, "lte": <value}.

Filter composition

You can filter on multiple fields by adding the fields you want to filter on to the filter parameter as shown in the Filter on size, width, and height section below.

Filtering on the level of confidence

The level of confidence represents the degree of similarity between your query and the search results. You can use the threshold parameter to filter on the level of confidence. This allows you to narrow down a response obtained from the platform by retrieving only the most relevant results. The threshold parameter can take one of the following values:

low (this is the default value)
medium
high

For a description of each field in the request and response, see the API Reference > Make any-to-video search request page.

Prerequisites

You're familiar with the concepts that are described on the Platform overview page.
You've already created an index, and the Marengo video understanding engine is enabled for this index.
You've already uploaded a video, and the platform has finished indexing it.

Examples

Filter on a specific video ID

The following example code uses the id field of the filter parameter to filter on a specific video ID:

from twelvelabs import TwelveLabs

client = TwelveLabs(api_key="<YOUR_API_KEY>")

search_results = client.search.query(
  index_id="<YOUR_INDEX_ID>",
  query_text= "<YOUR_QUERY>",
  options=["conversation"],
  filter={"id":["<YOUR_VIDEO_ID>"]}
)
# Utility function to print a specific page
def print_page(page):
  for clip in page:
    print(
        f"video_id={clip.video_id} score={clip.score} start={clip.start} end={clip.end} confidence={clip.confidence} metadata={clip.metadata}"
    )

print_page(search_results.data)

while True:
    try:
        print_page(next(search_results))
    except StopIteration:
        break

import { TwelveLabs, SearchData } from 'twelvelabs-js';

const client = new TwelveLabs({ apiKey: '<YOUR_API_KEY>'});

let searchResults = await client.search.query({
  indexId: '<YOUR_INDEX_ID>',
  queryText: '<YOUR_QUERY>',
  options: ['conversation'],
  filter: { id: ['<YOUR_VIDEO_ID>'] },
});
printPage(searchResults.data);
while (true) {
  const page = await searchResults.next();
  if (page === null) break;
  else printPage(page);
}
// Utility function to print a specific page
function printPage(searchData) {
  (searchData as SearchData[]).forEach((clip) => {
    console.log(
      `video_id= ${clip.videoId} score=${clip.score} start=${clip.start} end=${clip.end} confidence=${clip.confidence} metadata=${JSON.stringify(clip.metadata)}`,
    );
  });
}

The following output has been truncated for brevity:

video_id=629deb6ff05c91527bc5c1c9 score=72.95 start=416.65 end=433.42 confidence=low metadata=[{'text': "Leave it alone, I have a gun, I'm walking backwards on the trail back to the camp, I'm walking down the trail and mom and two cubs so I am continually to walk backwards.", 'type': 'conversation'}]
video_id=629deb6ff05c91527bc5c1c9 score=68.81 start=201.74 end=279.86 confidence=low metadata=[{'text': "Mhm Well that puts an end to my tarpon fishing, get back now when you're playing with lions, it's important to never show fear chuck.", 'type': 'conversation'}]
video_id=629deb6ff05c91527bc5c1c9 score=68.74 start=436.88 end=490.25 confidence=low metadata=[{'text': 'The mom and the cubs keep following me and or walking at least as fast as I am. People get it. Oh, you, you, you, oh, oh no!', 'type': 'conversation'}]

Filter on multiple video IDs

The following example code uses the id field of the filter query parameter to filter on multiple video IDs:

from twelvelabs import TwelveLabs

client = TwelveLabs(api_key="<YOUR_API_KEY>")

search_results = client.search.query(
  index_id="<YOUR_INDEX_ID>",
  query_text= "<YOUR_QUERY>",
  options=["visual"],
  filter={"id":["<FIRST_VIDEO_ID>", "<SECOND_VIDEO_ID"]}
)
# Utility function to print a specific page
def print_page(page):
  for clip in page:
    print(
        f"video_id={clip.video_id} score={clip.score} start={clip.start} end={clip.end} confidence={clip.confidence} metadata={clip.metadata}"
    )

print_page(search_results.data)

while True:
    try:
        print_page(next(search_results))
    except StopIteration:
        break

import { TwelveLabs, SearchData } from 'twelvelabs-js';

const client = new TwelveLabs({ apiKey: '<YOUR_API_KEY>'});

let searchResults = await client.search.query({
  indexId: '<YOUR_INDEX_ID>',
  queryText: '<YOUR_QUERY>',
  options: ['conversation'],
  filter: { id: ['<FIRST_VIDEO_ID>','<SECOND_VIDEO_ID>'] },
});
printPage(searchResults.data);
while (true) {
  const page = await searchResults.next();
  if (page === null) break;
  else printPage(page);
}
// Utility function to print a specific page
function printPage(searchData) {
  (searchData as SearchData[]).forEach((clip) => {
    console.log(
      `video_id= ${clip.videoId} score=${clip.score} start=${clip.start} end=${clip.end} confidence=${clip.confidence} metadata=${JSON.stringify(clip.metadata)}`,
    );
  });
}

The following output has been truncated for brevity:

video_id=629deb6ff05c91527bc5c1c9 score=72.95 start=416.65 end=433.42 confidence=low metadata=[{'text': "Leave it alone, I have a gun, I'm walking backwards on the trail back to the camp, I'm walking down the trail and mom and two cubs so I am continually to walk backwards.", 'type': 'conversation'}]
video_id=629deb6ff05c91527bc5c1c9 score=68.81 start=201.74 end=279.86 confidence=low metadata=[{'text': "Mhm Well that puts an end to my tarpon fishing, get back now when you're playing with lions, it's important to never show fear chuck.", 'type': 'conversation'}]
video_id=629deb9df05c91527bc5c1cb score=68.74 start=436.88 end=490.25 confidence=low metadata=[{'text': 'The mom and the cubs keep following me and or walking at least as fast as I am. People get it. Oh, you, you, you, oh, oh no!', 'type': 'conversation'}]

Filter on size, width, and height

The example code below uses the size, width, and height fields of the filter parameter to return only the matches found in videos that meet all the following criteria:

Size is greater than or equal to 50000000 bytes and less and equal to 53000000 bytes.
Width is greater than or equal to 850.
Height is greater than or equal to 400 and less and equal to 500.


from twelvelabs import TwelveLabs

client = TwelveLabs(api_key="<YOUR_API_KEY>")

search_results = client.search.query(
  index_id="<YOUR_INDEX_ID>",
  query_text= "<YOUR_QUERY>",
  options=["conversation"],
  filter={
    "size": {
      "gte": 50000000, "lte": 53000000
    },
    "width":
      {
        "gte": 850
      },
    "height":
      {
        "gte": 400, "lte": 500
      }
  }
)
# Utility function to print a specific page
def print_page(page):
  for clip in page:
    print(
        f"video_id={clip.video_id} score={clip.score} start={clip.start} end={clip.end} confidence={clip.confidence} metadata={clip.metadata}"
    )

print_page(search_results.data)

while True:
    try:
        print_page(next(search_results))
    except StopIteration:
        break

import { TwelveLabs, SearchData } from 'twelvelabs-js';

const client = new TwelveLabs({ apiKey: '<YOUR_API_KEY>'});

let searchResults = await client.search.query({
  indexId: '<YOUR_INDEX_ID>',
  queryText: '<YOUR_QUERY>',
  options: ['conversation'],
  filter: {
    size: {
      gte: 50000000,
      lte: 53000000,
    },
    width: {
      gte: 850,
    },
    height: {
      gte: 400,
      lte: 500,
    },
  },
});
printPage(searchResults.data);
while (true) {
  const page = await searchResults.next();
  if (page === null) break;
  else printPage(page);
}
// Utility function to print a specific page
function printPage(searchData) {
  (searchData as SearchData[]).forEach((clip) => {
    console.log(
      `video_id= ${clip.videoId} score=${clip.score} start=${clip.start} end=${clip.end} confidence=${clip.confidence} metadata=${JSON.stringify(clip.metadata)}`,
    );
  });
}

The following output was truncated for brevity:

video_id=629deb9df05c91527bc5c1cb score=88.4 start=930.44 end=937.75 confidence=high metadata=Bear encounters can be scary, exciting. And even funny. It all really depends on the circumstances you're in.
video_id=629deb9df05c91527bc5c1cb score=84.74 start=937.79 end=947.56 confidence=high metadata=While bears aren't necessarily always aggressive creatures, it's probably best to keep your distance. Should you ever run into one in the wild?
video_id=629deb9df05c91527bc5c1cb score=84.73 start=788.34 end=805.3 confidence=high metadata= If you were driving through the forest and saw a bear up ahead, what would you do? Surely you'd feel pretty safe knowing you were inside your car. But what if that bear came up to you and began peeking through your window. That's exactly the situation.

Filter on custom metadata

The example code below filters on a custom field named views of type integer. The platform will return only the results found in the videos for which the value of the views field equals 120000. For details about specifying custom metadata, see the Provide custom metadata page.

from twelvelabs import TwelveLabs

client = TwelveLabs(api_key="<YOUR_API_KEY>")

search_results = client.search.query(
  index_id="<YOUR_INDEX_ID>",
  query_text= "<YOUR_QUERY>",
  options=["conversation"],
  filter = {
    "views": 120000
  }
)
# Utility function to print a specific page
def print_page(page):
  for clip in page:
    print(
        f"video_id={clip.video_id} score={clip.score} start={clip.start} end={clip.end} confidence={clip.confidence} metadata={clip.metadata}"
    )

print_page(search_results.data)

while True:
    try:
        print_page(next(search_results))
    except StopIteration:
        break

import { TwelveLabs, SearchData } from 'twelvelabs-js';

const client = new TwelveLabs({ apiKey: '<YOUR_API_KEY>'});

let searchResults = await client.search.query({
  indexId: '<YOUR_INDEX_ID>',
  queryText: '<YOUR_QUERY>',
  options: ['conversation'],
  filter: {
    views: 120000,
  },
});
printPage(searchResults.data);
while (true) {
  const page = await searchResults.next();
  if (page === null) break;
  else printPage(page);
}
// Utility function to print a specific page
function printPage(searchData) {
  (searchData as SearchData[]).forEach((clip) => {
    console.log(
      `video_id= ${clip.videoId} score=${clip.score} start=${clip.start} end=${clip.end} confidence=${clip.confidence} metadata=${JSON.stringify(clip.metadata)}`,
    );
  });
}

The following output was truncated for brevity:

video_id=65d5fbad48db9fa780cb415c score=65.0 start=347.04 end=363.929 confidence=low  metadata=[{'type': 'conversation', 'text': "And with that goal, Lionel Messi becomes Argentina's all time leading scorer. "}]

Filtering on the level of confidence

The following example code specifies that the minimum level of confidence shouldn't be lower than medium:

from twelvelabs import TwelveLabs

client = TwelveLabs(api_key="<YOUR_API_KEY>")

search_results = client.search.query(
  index_id="<YOUR_INDEX_ID>",
  query_text= "<YOUR_QUERY>",
  options=["conversation"],
  threshold='medium'
)
# Utility function to print a specific page
def print_page(page):
  for clip in page:
    print(
        f"video_id={clip.video_id} score={clip.score} start={clip.start} end={clip.end} confidence={clip.confidence} metadata={clip.metadata}"
    )

print_page(search_results.data)

while True:
    try:
        print_page(next(search_results))
    except StopIteration:
        break

import { TwelveLabs, SearchData } from 'twelvelabs-js';

const client = new TwelveLabs({ apiKey: '<YOUR_API_KEY>'});

let searchResults = await client.search.query({
  indexId: '<YOUR_INDEX_ID>,
  queryText: '<YOUR_QUERY>',
  options: ['conversation'],
  threshold: 'medium',
});
printPage(searchResults.data);
while (true) {
  const page = await searchResults.next();
  if (page === null) break;
  else printPage(page);
}
// Utility function to print a specific page
function printPage(searchData) {
  (searchData as SearchData[]).forEach((clip) => {
    console.log(
      `video_id= ${clip.videoId} score=${clip.score} start=${clip.start} end=${clip.end} confidence=${clip.confidence} metadata=${JSON.stringify(clip.metadata)}`,
    );
  });

The following example output was truncated for brevity:

video_id=63b2c707ce36463e0199c906 score=80.28 start=25.421875 end=31.375 confidence=medium metadata=[{'type': 'visual'}]
video_id=639963a1ce36463e0199c8c7 score=75.25 start=266.765625 end=270.5 confidence=medium metadata=[{'type': 'visual'}]
video_id=639963a1ce36463e0199c8c7 score=75.09 start=119.796875 end=125.71875 confidence=high metadata=[{'type': 'visual'}]
video_id=639963a1ce36463e0199c8c7 score=73.96 start=300.475 end=304.15625 confidence=high metadata=[{'type': 'visual'}]

In this example, note that the data array is composed of a list of objects. Each object corresponds to a matching video clip and has a field named confidence whose value is either medium or high.

Search results

📘
Note

Filtering search results based on metadata

Exact match operator

Comparison operators

Filter composition

Filtering on the level of confidence

Prerequisites

Examples

Filter on a specific video ID

Filter on multiple video IDs

Filter on size, width, and height

Filter on custom metadata

Filtering on the level of confidence

📘Note

Filtering search results based on metadata

Exact match operator

Comparison operators

Filter composition

Filtering on the level of confidence

Prerequisites

Examples

Filter on a specific video ID

Filter on multiple video IDs

Filter on size, width, and height

Filter on custom metadata

Filtering on the level of confidence

📘
Note