When you perform a search, the platform returns all the relevant matches. Filtering narrows the scope of your query. The platform allows you to filter your search results based on metadata or the level of confidence that the search results match your query.

Filtering search results based on metadata

To filter your search results based on metadata, use the filter parameter.

The filter parameter is of type Object and can contain one or more of the following fields:

id: An array of strings that filters your search results based on the specified video IDs.
duration: A field of type Object that filters your search results based on the duration of the video containing the segment that matches your query.
width: A numeric field that filters your search results based on width.
height: A numeric field that filters your search results based on height.
size: A numeric field that filters your search results based on the size of the video containing the segment that matches your query, expressed in bytes.
title: A string field that filters your search results based on the title of the video.

To indicate the relationship between a field and its value, you can use the exact match or comparison operators.

Exact match operator

The exact match operator matches only the results that equal the value you specify. The syntax is as follows: <field>: <value>.

Comparison operators

Use the comparison operators (lte and gte) to match based on the arithmetic comparison. The syntax is as follows: <field>:{"gte": <value>, "lte": <value}.

Filter composition

You can filter on multiple fields by adding the fields you want to filter on to the filter parameter as shown in the Filter on size, width, and height section below.

Filtering on the level of confidence

The level of confidence represents the degree of similarity between your query and the search results. You can use the threshold parameter to filter on the level of confidence. This allows you to narrow down a response obtained from the platform by retrieving only the most relevant results. The threshold parameter can take one of the following values:

low (this is the default value)
medium
high

For a description of each field in the request and response, see the API Reference > Make any-to-video search request page.

Prerequisites

You're familiar with the concepts that are described on the Platform overview page.
You've already created an index, and the Marengo video understanding engine is enabled for this index.
You've already uploaded a video, and the platform has finished indexing it.

Examples

Filter on a specific video ID

The following example code uses the id field of the filter parameter to filter on a specific video ID:

from twelvelabs import TwelveLabs

client = TwelveLabs(api_key="<YOUR_API_KEY>")

search_results = client.search.query(
  index_id="<YOUR_INDEX_ID>",
  query_text= "<YOUR_QUERY>",
  options=["visual"],
  filter={"id":["<YOUR_VIDEO_ID>"]}
)
# Utility function to print a specific page
def print_page(page):
  for clip in page:
    print(
        f"video_id={clip.video_id} score={clip.score} start={clip.start} end={clip.end} confidence={clip.confidence}"
    )

print_page(search_results.data)

while True:
    try:
        print_page(next(search_results))
    except StopIteration:
        break

import { TwelveLabs, SearchData } from 'twelvelabs-js';

const client = new TwelveLabs({ apiKey: '<YOUR_API_KEY>'});

let searchResults = await client.search.query({
  indexId: '<YOUR_INDEX_ID>',
  queryText: '<YOUR_QUERY>',
  options: ['visual'],
  filter: { id: ['<YOUR_VIDEO_ID>'] },
});
printPage(searchResults.data);
while (true) {
  const page = await searchResults.next();
  if (page === null) break;
  else printPage(page);
}
// Utility function to print a specific page
function printPage(searchData) {
  (searchData as SearchData[]).forEach((clip) => {
    console.log(
      `video_id= ${clip.videoId} score=${clip.score} start=${clip.start} end=${clip.end} confidence=${clip.confidence}`,
    );
  });
}

The following output has been truncated for brevity:

video_id=629deb6ff05c91527bc5c1c9 score=72.95 start=416.65 end=433.42 confidence=low 
video_id=629deb6ff05c91527bc5c1c9 score=68.81 start=201.74 end=279.86 confidence=low 
video_id=629deb6ff05c91527bc5c1c9 score=68.74 start=436.88 end=490.25 confidence=low

Filter on multiple video IDs

The following example code uses the id field of the filter query parameter to filter on multiple video IDs:

from twelvelabs import TwelveLabs

client = TwelveLabs(api_key="<YOUR_API_KEY>")

search_results = client.search.query(
  index_id="<YOUR_INDEX_ID>",
  query_text= "<YOUR_QUERY>",
  options=["visual"],
  filter={"id":["<FIRST_VIDEO_ID>", "<SECOND_VIDEO_ID>"]}
)
# Utility function to print a specific page
def print_page(page):
  for clip in page:
    print(
        f"video_id={clip.video_id} score={clip.score} start={clip.start} end={clip.end} confidence={clip.confidence}"
    )

print_page(search_results.data)

while True:
    try:
        print_page(next(search_results))
    except StopIteration:
        break

import { TwelveLabs, SearchData } from 'twelvelabs-js';

const client = new TwelveLabs({ apiKey: '<YOUR_API_KEY>'});

let searchResults = await client.search.query({
  indexId: '<YOUR_INDEX_ID>',
  queryText: '<YOUR_QUERY>',
  options: ['visual'],
  filter: { id: ['<FIRST_VIDEO_ID>','<SECOND_VIDEO_ID>'] },
});
printPage(searchResults.data);
while (true) {
  const page = await searchResults.next();
  if (page === null) break;
  else printPage(page);
}
// Utility function to print a specific page
function printPage(searchData) {
  (searchData as SearchData[]).forEach((clip) => {
    console.log(
      `video_id= ${clip.videoId} score=${clip.score} start=${clip.start} end=${clip.end} confidence=${clip.confidence}`,
    );
  });
}

The following output has been truncated for brevity:

video_id=629deb6ff05c91527bc5c1c9 score=72.95 start=416.65 end=433.42 confidence=low
video_id=629deb6ff05c91527bc5c1c9 score=68.81 start=201.74 end=279.86 confidence=low
video_id=629deb9df05c91527bc5c1cb score=68.74 start=436.88 end=490.25 confidence=low

Filter on size, width, and height

The example code below uses the size, width, and height fields of the filter parameter to return only the matches found in videos that meet all the following criteria:

Size is greater than or equal to 50000000 bytes and less and equal to 53000000 bytes.
Width is greater than or equal to 850.
Height is greater than or equal to 400 and less and equal to 500.


from twelvelabs import TwelveLabs

client = TwelveLabs(api_key="<YOUR_API_KEY>")

search_results = client.search.query(
  index_id="<YOUR_INDEX_ID>",
  query_text= "<YOUR_QUERY>",
  options=["visual"],
  filter={
    "size": {
      "gte": 50000000, "lte": 53000000
    },
    "width":
      {
        "gte": 850
      },
    "height":
      {
        "gte": 400, "lte": 500
      }
  }
)
# Utility function to print a specific page
def print_page(page):
  for clip in page:
    print(
        f"video_id={clip.video_id} score={clip.score} start={clip.start} end={clip.end} confidence={clip.confidence}"
    )

print_page(search_results.data)

while True:
    try:
        print_page(next(search_results))
    except StopIteration:
        break

import { TwelveLabs, SearchData } from 'twelvelabs-js';

const client = new TwelveLabs({ apiKey: '<YOUR_API_KEY>'});

let searchResults = await client.search.query({
  indexId: '<YOUR_INDEX_ID>',
  queryText: '<YOUR_QUERY>',
  options: ['conversation'],
  filter: {
    size: {
      gte: 50000000,
      lte: 53000000,
    },
    width: {
      gte: 850,
    },
    height: {
      gte: 400,
      lte: 500,
    },
  },
});
printPage(searchResults.data);
while (true) {
  const page = await searchResults.next();
  if (page === null) break;
  else printPage(page);
}
// Utility function to print a specific page
function printPage(searchData) {
  (searchData as SearchData[]).forEach((clip) => {
    console.log(
      `video_id= ${clip.videoId} score=${clip.score} start=${clip.start} end=${clip.end} confidence=${clip.confidence}`,
    );
  });
}

The following output was truncated for brevity:

video_id=629deb9df05c91527bc5c1cb score=88.4 start=930.44 end=937.75 confidence=high video_id=629deb9df05c91527bc5c1cb score=84.74 start=937.79 end=947.56 confidence=high video_id=629deb9df05c91527bc5c1cb score=84.73 start=788.34 end=805.3 confidence=high

Filter on custom metadata

The example code below filters on a custom field named views of type integer. The platform will return only the results found in the videos for which the value of the views field equals 120000. For details about specifying custom metadata, see the Provide custom metadata section.

from twelvelabs import TwelveLabs

client = TwelveLabs(api_key="<YOUR_API_KEY>")

search_results = client.search.query(
  index_id="<YOUR_INDEX_ID>",
  query_text= "<YOUR_QUERY>",
  options=["conversation"],
  filter = {
    "views": 120000
  }
)
# Utility function to print a specific page
def print_page(page):
  for clip in page:
    print(
        f"video_id={clip.video_id} score={clip.score} start={clip.start} end={clip.end} confidence={clip.confidence}"
    )

print_page(search_results.data)

while True:
    try:
        print_page(next(search_results))
    except StopIteration:
        break

import { TwelveLabs, SearchData } from 'twelvelabs-js';

const client = new TwelveLabs({ apiKey: '<YOUR_API_KEY>'});

let searchResults = await client.search.query({
  indexId: '<YOUR_INDEX_ID>',
  queryText: '<YOUR_QUERY>',
  options: ['conversation'],
  filter: {
    views: 120000,
  },
});
printPage(searchResults.data);
while (true) {
  const page = await searchResults.next();
  if (page === null) break;
  else printPage(page);
}
// Utility function to print a specific page
function printPage(searchData) {
  (searchData as SearchData[]).forEach((clip) => {
    console.log(
      `video_id= ${clip.videoId} score=${clip.score} start=${clip.start} end=${clip.end} confidence=${clip.confidence}`,
    );
  });
}

Filtering on the level of confidence

The following example code specifies that the minimum level of confidence shouldn't be lower than medium:

from twelvelabs import TwelveLabs

client = TwelveLabs(api_key="<YOUR_API_KEY>")

search_results = client.search.query(
  index_id="<YOUR_INDEX_ID>",
  query_text= "<YOUR_QUERY>",
  options=["visual"],
  threshold='medium'
)
# Utility function to print a specific page
def print_page(page):
  for clip in page:
    print(
        f"video_id={clip.video_id} score={clip.score} start={clip.start} end={clip.end} confidence={clip.confidence}"
    )

print_page(search_results.data)

while True:
    try:
        print_page(next(search_results))
    except StopIteration:
        break

import { TwelveLabs, SearchData } from 'twelvelabs-js';

const client = new TwelveLabs({ apiKey: '<YOUR_API_KEY>'});

let searchResults = await client.search.query({
  indexId: '<YOUR_INDEX_ID>,
  queryText: '<YOUR_QUERY>',
  options: ['visual'],
  threshold: 'medium',
});
printPage(searchResults.data);
while (true) {
  const page = await searchResults.next();
  if (page === null) break;
  else printPage(page);
}
// Utility function to print a specific page
function printPage(searchData) {
  (searchData as SearchData[]).forEach((clip) => {
    console.log(
      `video_id= ${clip.videoId} score=${clip.score} start=${clip.start} end=${clip.end} confidence=${clip.confidence}`,
    );
  });

The following example output was truncated for brevity:

video_id=63b2c707ce36463e0199c906 score=80.28 start=25.421875 end=31.375 confidence=medium 
video_id=639963a1ce36463e0199c8c7 score=75.25 start=266.765625 end=270.5 confidence=medium video_id=639963a1ce36463e0199c8c7 score=75.09 start=119.796875 end=125.71875 confidence=high video_id=639963a1ce36463e0199c8c7 score=73.96 start=300.475 end=304.15625 confidence=high

In this example, note that the data array is composed of a list of objects. Each object corresponds to a matching video clip and has a field named confidence whose value is either medium or high.