Use single queries

Use single queries to search for matches of the same query across one or multiple sources of information. This section shows how to perform searches using single queries.

This page is divided into two main sections:

  • Search using a single source of information: This section provides detailed instructions on performing searches using data from a specific source of information, such as visual content, conversation, on-screen text, or logos. Each source of information offers unique insights relevant to your query.
  • Search using multiple sources of information: This section shows how to enhance your search queries by combining data from multiple sources of information. It explains the differences in constructing your requests and interpreting the responses.

Prerequisites

The examples in this guide assume the following:

  • You’re familiar with the concepts that are described on the Platform overview page.
  • You've already created an index, and the Marengo video understanding engine is enabled for this index.
  • You've already uploaded a video, and the platform has finished indexing it.

Use a single source of information

When performing a search request using a single source of information, specify at least the following parameters:

  • index_id: The unique identifier of the index containing your videos.
  • query: Your search query. Note that the platform supports full natural language-based search. The following examples are valid queries: "birds flying near a castle" and "crowd cheering in the stadium."
  • options: An array containing the source of information the platform must use.

The response is a SearchResult object, which contains the following fields:

  • pool: A SearchPool object that contains the total number of videos within the index, the total duration of the videos, and the unique identifier of the index that you've searched.
  • data: A list of SearchData objects representing the search results, each of which contains the following fields:
    • score: A quantitative value determined by the AI engine representing the level of confidence that the results match your search terms.
    • start: The start time of the matching video clip, expressed in seconds.
    • end: The end time of the matching video clip, expressed in seconds.
    • video_id: The unique identifier of the video that matched your search terms.
    • metadata: An array of objects where each object contains details about the source of information that matched your search terms and is composed of the following key-value pairs:
      • type: Represents the source of information for which the video clip matched your search terms. It can take one of the following values: visual, conversation, text_in_video, and logo.
      • (Optional) text: For conversation and text_in_video, the platform returns a field named text, representing the text that matches your search terms. When performing a semantic search, the platform determines the meaning of your search query, rather than just matching your search terms to the content of your videos. This means that the value of this field doesn't always match your search terms.
        Example:
      "metadata": [
        {
          "type": "conversation",
          "text": "The safety car has gone, the lights gone green and there's a big crash at the back and that is the Alfa Romeo, certainly Antonio was involved in that skirmish as well."
        }
      ]
      
    • confidence: A qualitative indicator based on the value of the score field. This field can take one of the following values:
      • high
      • medium
      • low
      • extremely_low
    • (Optional) thumbnail_url: If thumbnail generation has been enabled for this index, the platform returns a string representing the URL of the segment's thumbnail. Note that the URL expires in one hour.
  • page_info: A SearchPageInfo object that provides information about pagination. The platform returns the results one page at a time, with a default limit of 10 results per page. For details about retrieving the subsequent pages of results and configuring the pagination behavior, see the Pagination > Search results page.

For a description of each field in the request and response, see the Make a search request page.

Visual

The platform analyzes video content as you would see and hear from it, including actions, objects, sounds, and events, excluding human speech. To search using visual cues, invoke the query method of the search object with the following parameters:

  • index_id: A string representing the unique identifier of the index to search.
  • query: A string representing your search query. Note that the platform supports full natural language-based search. The following examples are valid queries: "birds flying near a castle," "sun shining on the water," "chickens on the road," "an officer holding a child's hand," and "crowd cheering in the stadium."
  • options: An array of strings specifying the sources of information the platform uses when performing a search. This example searches using visual cues.
from twelvelabs import TwelveLabs

client = TwelveLabs(api_key="<YOUR_API_KEY>")

search_results = client.search.query(
  index_id="<YOUR_INDEX_ID>", 
  query="<YOUR_QUERY>", 
  options=["visual"]
)

# Utility function to print a specific page
def print_page(page):
  for clip in page:
    print(
        f" video_id={clip.video_id} score={clip.score} start={clip.start} end={clip.end} confidence={clip.confidence} metadata={clip.metadata}"
    )

print_page(search_results.data)

while True:
    try:
        print_page(next(search_results))
    except StopIteration:
        break
import { TwelveLabs } from 'twelvelabs-js';

const client = new TwelveLabs({ apiKey: '<YOUR_API_KEY>'});

let searchResults = await client.search.query({
  indexId: '<YOUR_INDEX_ID>'
  query: '<YOUR_QUERY>',
  options: ['visual'],
});
printPage(searchResults.data);
while (true) {
  const page = await searchResults.next();
  if (page === null) break;
  else printPage(page);
}
// Utility function to print a specific page
function printPage(searchData) {
  (searchData as SearchData[]).forEach((clip) => {
    console.log(
      `video_id= ${clip.videoId} score=${clip.score} start=${clip.start} end=${clip.end} confidence=${clip.confidence} metadata=${JSON.stringify(clip.metadata)}`,
    );
  });
}

The example output below was truncated for brevity:

video_id=65d60bcf48db9fa780cb415e score=83.73 start=273.96875 end=289.0625 confidence=high metadata=[{'type': 'visual'}]
video_id=65d60bcf48db9fa780cb415e score=83.55 start=397.921875 end=439.84375 confidence=high metadata=[{'type': 'visual'}]
video_id=65d60bcf48db9fa780cb415e score=83.46 start=294.5625 end=311.84375 confidence=high metadata=[{'type': 'visual'}]
video_id=65d60bcf48db9fa780cb415e score=83.45 start=233.0 end=247.78125 confidence=high metadata=[{'type': 'visual'}]

Conversation

The platform analyzes human speech within your videos. The following example sets the value of the options parameter to ["conversation"] to search using human speech as the source of information. Note that you can perform semantic and exact searches by setting the value of the conversation_option parameter. See the Conversation option page for details.

from twelvelabs import TwelveLabs

client = TwelveLabs(api_key="<YOUR_API_KEY>")

search_results = client.search.query(
    index_id="<YOUR_INDEX_ID>", 
    query="<YOUR_QUERY>" 
    options=["conversation"], 
    conversation_option="semantic" # Optional
)

# Utility function to print a specific page
def print_page(page):
  for clip in page:
    print(
        f" video_id={clip.video_id} score={clip.score} start={clip.start} end={clip.end} confidence={clip.confidence} metadata={clip.metadata}"
    )

print_page(search_results.data)

while True:
    try:
        print_page(next(search_results))
    except StopIteration:
        break
import { TwelveLabs, SearchData } from 'twelvelabs-js';

const client = new TwelveLabs({ apiKey: '<YOUR_API_KEY>'});

let searchResults = await client.search.query({
  indexId: '<YOUR_INDEX_ID>',
  query: '<YOUR_QUERY>',
  options: ['conversation'],
  conversationOption: 'semantic', // Optional
});
printPage(searchResults.data);
while (true) {
  const page = await searchResults.next();
  if (page === null) break;
  else printPage(page);
}
// Utility function to print a specific page
function printPage(searchData) {
  (searchData as SearchData[]).forEach((clip) => {
    console.log(
      `video_id= ${clip.videoId} score=${clip.score} start=${clip.start} end=${clip.end} confidence=${clip.confidence} metadata=${JSON.stringify(clip.metadata)}`,
    );
  });
}

The example output below was truncated for brevity:

 video_id=65d5fbad48db9fa780cb415c score=70.05 start=347.04 end=363.929 confidence=low metadata=[{'type': 'conversation', 'text': "And with that goal, Lionel Messi becomes Argentina's all time leading scorer."}]
 video_id=65d5fbad48db9fa780cb415c score=58.75 start=366.869 end=405.589 confidence=low metadata=[{'type': 'conversation', 'text': 'I guess it deserved that goal scorer years ago just like good saves in the second period. Just when you think comes up with something more special.'}]

Text in video

The platform detecst and extracts text (OCR) shown within your videos. The following example sets the value of the options parameter to ["text_in_video"] to search for text that matches your search query:

from twelvelabs import TwelveLabs

client = TwelveLabs(api_key="<YOUR_API_KEY>")

search_results = client.search.query(
  index_id="<YOUR_INDEX_ID>", 
  query="<YOUR_QUERY>", 
  options=["text_in_video"]
)

# Utility function to print a specific page
def print_page(page):
  for clip in page:
    print(
        f" video_id={clip.video_id} score={clip.score} start={clip.start} end={clip.end} confidence={clip.confidence} metadata={clip.metadata}"
    )

print_page(search_results.data)

while True:
    try:
        print_page(next(search_results))
    except StopIteration:
        break
import { TwelveLabs, SearchData } from 'twelvelabs-js';

const client = new TwelveLabs({ apiKey: '<YOUR_API_KEY>'});

let searchResults = await client.search.query({
  indexId: '<YOUR_INDEX_ID>',
  query: '<YOUR_QUERY>',
  options: ['text_in_video'],
});
printPage(searchResults.data);
while (true) {
  const page = await searchResults.next();
  if (page === null) break;
  else printPage(page);
}
// Utility function to print a specific page
function printPage(searchData) {
  (searchData as SearchData[]).forEach((clip) => {
    console.log(
      `video_id= ${clip.videoId} score=${clip.score} start=${clip.start} end=${clip.end} confidence=${clip.confidence} metadata=${JSON.stringify(clip.metadata)}`,
    );
  });
}

The example output below was truncated for brevity:

video_id=65d5fbad48db9fa780cb415c score=28.0 start=347.04 end=363.929 confidence=low metadata=[{'type': 'conversation', 'text': 'bear cubs'}]
video_id=65d5fbad48db9fa780cb415c score=58.75 start=366.869 end=405.589 confidence=low metadata=[{'type': 'conversation', 'text': 'bear cubs'}]

Logo

The platform detects and extracts brand logos as shown within your videos. For this, call the query method of the search object with the following parameters:

  • query: The name of the company. This example uses Starbucks.
  • options: The source of information the platform must use (logo).
  • index_id: The unique identifier of the index containing your videos.

The example code below finds when the Starbucks company logo appears in your videos:

from twelvelabs import TwelveLabs

client = TwelveLabs(api_key="<YOUR_API_KEY>")

search_results = client.search.query(
  index_id="<YOUR_INDEX_ID>", 
  query="Starbucks", 
  options=["logo"]
)

# Utility function to print a specific page
def print_page(page):
  for clip in page:
    print(
        f" video_id={clip.video_id} score={clip.score} start={clip.start} end={clip.end} confidence={clip.confidence} metadata={clip.metadata}"
    )

print_page(search_results.data)

while True:
    try:
        print_page(next(search_results))
    except StopIteration:
        break
import { TwelveLabs, SearchData } from 'twelvelabs-js';

const client = new TwelveLabs({ apiKey: '<YOUR_API_KEY>'});

let searchResults = await client.search.query({
  indexId: '<YOUR_INDEX_ID>',
  query: 'Starbucks',
  options: ['logo'],
});
printPage(searchResults.data);
while (true) {
  const page = await searchResults.next();
  if (page === null) break;
  else printPage(page);
}
// Utility function to print a specific page
function printPage(searchData) {
  (searchData as SearchData[]).forEach((clip) => {
    console.log(
      `video_id= ${clip.videoId} score=${clip.score} start=${clip.start} end=${clip.end} confidence=${clip.confidence} metadata=${JSON.stringify(clip.metadata)}`,
    );
  });
}

The following example output was truncated for brevity:

video_id=63eb67e49678ed7709731d63 score=92.28 start=104 end=106 confidence=high metadata=[{'type': 'logo', 'text': 'Starbucks'}]

In this example output, the Starbucks logo appears at 104 seconds from the start of the video:

Use multiple sources of information

When performing a search request using multiple sources of information, specify at least the following parameters:

  • index_id: The unique identifier of the index containing your videos.

  • query: Your search query. Note that the platform supports full natural language-based search. The following examples are valid queries: "birds flying near a castle" and "crowd cheering in the stadium."

  • options: The sources of information the platform must use.

  • (Optional) operator: The operator the platform must use to combine the sources of information. The following logical operators are supported:

    • or: The platform returns the results for which any source of information matches.
    • and: The platform returns only the results for which all sources of information match.
      Note that the operator parameter is optional. If omitted, the platform will perform a logical OR operation. For clarity, the examples in this section always specify the operator parameter.
      The diagram below explains the difference between specifying the OR and AND logical operators:

The response is a SearchResult object, which contains the following fields:

  • pool: A SearchPool object that contains the total number of videos within the index, the total duration of the videos, and the unique identifier of the index that you've searched.
  • data: A list of SearchData objects, each of which contains the following fields:
    • score: A quantitative value determined by the AI engine representing the level of confidence that the results match your search terms.
    • start: The start time of the matching video clip, expressed in seconds.
    • end: The end time of the matching video clip, expressed in seconds.
    • video_id: The unique identifier of the video that matched your search terms.
    • modules: An array where each element is a dictionary composed of the following key-value pairs:
      • type: The source of information for which the video clip matched your search terms. It can take one of the following values: conversation, visual, text_in_video, and logo.
      • confidence: The confidence level that the result is accurate.
        The following example shows that a video clip matched your search terms using two sources of information - conversation and visual:
      "modules": [
        {
          "type": "conversation",
          "confidence": "medium"
        },
        {
          "type": "visual",
          "confidence": "low"
        }
      ]
      
    • metadata: An array of objects where each object contains details about the source of information that matched your search terms and is composed of the following key-value pairs:
      • type: Represents the source of information for which the video clip matched your search terms. It can take one of the following values: visual, conversation, text_in_video, and logo.
      • (Optional) text: For conversation and text_in_video, the platform returns a field named text, representing the text that matches your search terms. When performing a semantic search, the platform determines the meaning of your search query, rather than just matching your search terms to the content of your videos. This means that the value of this field doesn't always match your search terms.
        Example:
      "metadata": [
        {
          "type": "conversation",
          "text": "The safety car has gone, the lights gone green and there's a big crash at the back and that is the Alfa Romeo, certainly Antonio was involved in that skirmish as well."
        },
        {
          "type": "visual"
        }
      ]
      
    • confidence: A qualitative indicator based on the value of the score field. This field can take one of the following values:
      • high
      • medium
      • low
      • extremely_low
    • thumbnail_url: If thumbnail generation has been enabled for this index, the platform returns a string representing the URL of the thumbnail of the segment. Note that the URL expires in one hour.
  • page_info: A SearchPageInfo object that provides information about pagination. The platform returns the results one page at a time, with a default limit of 10 results per page. For details about retrieving the subsequent pages of results and configuring the pagination behavior, see the Pagination > Search results page.

For a description of each field in the request and response, see the Make a search request page.

Visual and conversation

The following example sets the value options parameter to ["visual", "conversation"] and the value of the operator parameter to or to specify that the platform should return the results for which any of the selected sources of information match:

from twelvelabs import TwelveLabs

client = TwelveLabs(api_key="<YOUR_API_KEY>")

search_results = client.search.query(
  index_id="<YOUR_INDEX_ID>",
  query="<YOUR_QUERY>",
  options=["visual","conversation"],
  operator="or"
)
# Utility function to print a specific page
def print_page(page):
  for clip in page:
    print(
        f" video_id={clip.video_id} score={clip.score} start={clip.start} end={clip.end} confidence={clip.confidence} metadata={clip.metadata} modules={clip.modules}"
    )

print_page(search_results.data)

while True:
    try:
        print_page(next(search_results))
    except StopIteration:
        break
import { TwelveLabs, SearchData } from 'twelvelabs-js';

const client = new TwelveLabs({ apiKey: '<YOUR_API_KEY>'});

let searchResults = await client.search.query({
  indexId: '<YOUR_INDEX_ID>',
  query: '<YOUR_QUERY>',
  options: ['visual', 'conversation'],
  operator: 'or',
});
printPage(searchResults.data);
while (true) {
  const page = await searchResults.next();
  if (page === null) break;
  else printPage(page);
}
// Utility function to print a specific page
function printPage(searchData) {
  (searchData as SearchData[]).forEach((clip) => {
    console.log(
      `video_id= ${clip.videoId} score=${clip.score} start=${clip.start} end=${clip.end} confidence=${clip.confidence} metadata=${JSON.stringify(clip.metadata)} modules=${JSON.stringify(clip.modules)}`,
    );
  });
}

The example output below was truncated for brevity:

video_id=639963a1ce36463e0199c8c7 score=76.79 start=420.66 end=430.23 confidence=medium metadata=[{'type': 'visual'}, {'type': 'conversation'}] modules=[{'type': 'conversation', 'confidence': 'medium'}, {'type': 'visual', 'confidence': 'low'}]
video_id=639963a1ce36463e0199c8c7 score=72.27 start=279.21875 end=289.75 confidence=low metadata=[{'type': 'visual'}] modules=[{'type': 'visual', 'confidence': 'low'}]
video_id=639963a1ce36463e0199c8c7 score=72.2 start=87.38 end=98.73 confidence=low metadata=[{'type': 'conversation'}] modules=[{'type': 'conversation', 'confidence': 'low'}]

The following example sets the value options parameter to ["visual", "conversation"] and the value of the operator parameter to and to specify that the platform should return the results for which all the selected sources of information match:

from twelvelabs import TwelveLabs

client = TwelveLabs(api_key="<YOUR_API_KEY>")

search_results = client.search.query(
  index_id="<YOUR_INDEX_ID>",
  query="<YOUR_QUERY>",
  options=["visual","conversation"],
  operator="and"
)
# Utility function to print a specific page
def print_page(page):
  for clip in page:
    print(
        f" video_id={clip.video_id} score={clip.score} start={clip.start} end={clip.end} confidence={clip.confidence} metadata={clip.metadata} modules={clip.modules}"
    )

print_page(search_results.data)

while True:
    try:
        print_page(next(search_results))
    except StopIteration:
        break
import { TwelveLabs, SearchData } from 'twelvelabs-js';

const client = new TwelveLabs({ apiKey: '<YOUR_API_KEY>'});

let searchResults = await client.search.query({
  indexId: '<YOUR_INDEX_ID>',
  query: '<YOUR_QUERY>',
  options: ['visual', 'conversation'],
  operator: 'and',
});
printPage(searchResults.data);
while (true) {
  const page = await searchResults.next();
  if (page === null) break;
  else printPage(page);
}
// Utility function to print a specific page
function printPage(searchData) {
  (searchData as SearchData[]).forEach((clip) => {
    console.log(
      `video_id= ${clip.videoId} score=${clip.score} start=${clip.start} end=${clip.end} confidence=${clip.confidence} metadata=${JSON.stringify(clip.metadata)} modules=${JSON.stringify(clip.modules)}`,
    );
  });
}

The example output below was truncated for brevity:

video_id=639963a1ce36463e0199c8c7 score=76.79 start=420.66 end=430.23 confidence=medium metadata=[{'type': 'visual'}, {'type': 'conversation'}] modules=[{'type': 'conversation', 'confidence': 'medium'}, {'type': 'visual', 'confidence': 'low'}]
video_id=6399637ace36463e0199c8c6 score=58.99 start=665.745 end=676.795 confidence=low metadata=[{'type': 'visual'}, {'type': 'conversation'}] modules=[{'type': 'conversation', 'confidence': 'low'}, {'type': 'visual', 'confidence': 'low'}]

Conversation and text in video

The following example sets the value options parameter to ["conversation", "text_in_video"] and the value of the operator parameter to or to specify that the platform should return the results for which any of the selected sources of information match:

from twelvelabs import TwelveLabs

client = TwelveLabs(api_key="<YOUR_API_KEY>")

search_results = client.search.query(
  index_id="<YOUR_INDEX_ID>",
  query="<YOUR_QUERY>",
  options=["conversation", "text_in_video"],
  operator="or"
)
# Utility function to print a specific page
def print_page(page):
  for clip in page:
    print(
        f" video_id={clip.video_id} score={clip.score} start={clip.start} end={clip.end} confidence={clip.confidence} metadata={clip.metadata} modules={clip.modules}"
    )

print_page(search_results.data)

while True:
    try:
        print_page(next(search_results))
    except StopIteration:
        break
import { TwelveLabs, SearchData } from 'twelvelabs-js';

const client = new TwelveLabs({ apiKey: '<YOUR_API_KEY>'});

let searchResults = await client.search.query({
  indexId: '<YOUR_INDEX_ID>',
  query: '<YOUR_QUERY>',
  options: ['conversation', 'text_in_video'],
  operator: 'or',
});
printPage(searchResults.data);
while (true) {
  const page = await searchResults.next();
  if (page === null) break;
  else printPage(page);
}
// Utility function to print a specific page
function printPage(searchData) {
  (searchData as SearchData[]).forEach((clip) => {
    console.log(
      `video_id= ${clip.videoId} score=${clip.score} start=${clip.start} end=${clip.end} confidence=${clip.confidence} metadata=${JSON.stringify(clip.metadata)} modules=${JSON.stringify(clip.modules)}`,
    );
  });
}

The example output below was truncated for brevity:

video_id=639963d8ce36463e0199c8ca score=69.72 start=430 end=433 confidence=low metadata=[{'type': 'text_in_video', 'text': "Messi scores a goal similar to the Maradona's "Goal of the Century" ( vs Getafe in 2007)"}] modules=[{'type': 'text_in_video', 'confidence': 'low'}]
video_id=639963d8ce36463e0199c8ca score=69.72 start=597 end=621 confidence=low metadata=[{'type': 'text_in_video', 'text': "But Lucas Moura scores a hat-trick in second-half (last goal at the 96th minute) and qualify Tottenham for the 2019 UCL final"}] modules=[{'type': 'text_in_video', 'confidence': 'low'}]
video_id=639963afce36463e0199c8c8 score=62.33 start=271.1 end=311.64 confidence=low metadata=[{'type': 'conversation', 'text': " Been some something special for his hat trick goal last. Well, I'll tell you what, you're not gonna see. Much better goals than that. What a hat trick. And what a third goal down. But it's the moment of magic that real Madrid had been waiting for his way, way on side."}] modules=[{'type': 'conversation', 'confidence': 'low'}]

The following example sets the value options parameter to ["conversation", "text_in_video"] and the value of the operator parameter to and to specify that the platform should return only the results for which all the selected sources of information match:

from twelvelabs import TwelveLabs

client = TwelveLabs(api_key="<YOUR_API_KEY>")

search_results = client.search.query(
  index_id=INDEX_ID,
  query=QUERY,
  options=["conversation", "text_in_video"],
  operator="and"
)
# Utility function to print a specific page
def print_page(page):
  for clip in page:
    print(
        f" video_id={clip.video_id} score={clip.score} start={clip.start} end={clip.end} confidence={clip.confidence} metadata={clip.metadata}"
    )

print_page(search_results.data)

while True:
    try:
        print_page(next(search_results))
    except StopIteration:
        break
import { TwelveLabs, SearchData } from 'twelvelabs-js';

const client = new TwelveLabs({ apiKey: '<YOUR_API_KEY>'});

let searchResults = await client.search.query({
  indexId: '<YOUR_INDEX_ID>',
  query: '<YOUR_QUERY>',
  options: ['conversation', 'text_in_video'],
  operator: 'and',
});
printPage(searchResults.data);
while (true) {
  const page = await searchResults.next();
  if (page === null) break;
  else printPage(page);
}
// Utility function to print a specific page
function printPage(searchData) {
  (searchData as SearchData[]).forEach((clip) => {
    console.log(
      `video_id= ${clip.videoId} score=${clip.score} start=${clip.start} end=${clip.end} confidence=${clip.confidence} metadata=${JSON.stringify(clip.metadata)} modules=${JSON.stringify(clip.modules)}`,
    );
  });
}

The example output below was truncated for brevity:

video_id=62b3ee6cd01d61be020682f1 confidence=high score=96.26 start=469.58 end=489.06 metadata=[{'type': 'conversation', 'text': "We don't care about the goalkeeper who he is away from 234. Wonderful, wonderful, wonderful. How good is he? A near supernatural goal from Lionel Messi. He is just brilliant."}, {'type': 'text_in_video', 'text': "22 - WONDER GOAL AGAINST BIGGEST COUNTRY RIVAL"}] modules=[{'type': 'conversation', 'confidence': 'high'}, {'type': 'text_in_video', 'confidence': 'high'}]

Conversation, text in video, and visual

The following example sets the value options parameter to ["conversation", "text_in_video", "visual"] and the value of the operator parameter to or to specify that the platform should return the results for which any of the selected sources of information match:

from twelvelabs import TwelveLabs

client = TwelveLabs(api_key="<YOUR_API_KEY>")

search_results = client.search.query(
  index_id="<YOUR_INDEX_ID>",
  query="<YOUR_QUERY>",
  options=["conversation", "text_in_video", "visual"],
  operator="or"
)
# Utility function to print a specific page
def print_page(page):
  for clip in page:
    print(
        f" video_id={clip.video_id} score={clip.score} start={clip.start} end={clip.end} confidence={clip.confidence} metadata={clip.metadata} modules={clip.modules}"
    )

print_page(search_results.data)

while True:
    try:
        print_page(next(search_results))
    except StopIteration:
        break
import { TwelveLabs, SearchData } from 'twelvelabs-js';

const client = new TwelveLabs({ apiKey: '<YOUR_API_KEY>'});

let searchResults = await client.search.query({
  indexId: '<YOUR_INDEX_ID>',
  query: '<YOUR_QUERY>',
  options: ['conversation', 'text_in_video', 'visual'],
  operator: 'or',
});
printPage(searchResults.data);
while (true) {
  const page = await searchResults.next();
  if (page === null) break;
  else printPage(page);
}
// Utility function to print a specific page
function printPage(searchData) {
  (searchData as SearchData[]).forEach((clip) => {
    console.log(
      `video_id= ${clip.videoId} score=${clip.score} start=${clip.start} end=${clip.end} confidence=${clip.confidence} metadata=${JSON.stringify(clip.metadata)} modules=${JSON.stringify(clip.modules)}`,
    );
  });
}

The example output below was truncated for brevity:

video_id=639963d8ce36463e0199c8ca score=87 start=611.8125 end=614.84375 metadata=[{'type': 'visual'}, {'type': 'text_in_video', 'text': "But Lucas Moura scores a hat-trick in second-half (last goal at the 96th minute) and qualify Tottenham for the 2019 UCL final"}] modules=[{'type': 'visual', 'confidence': 'high'}, {'type': 'text_in_video', 'confidence': 'low'}]
video_id=639963d8ce36463e0199c8ca score=86.03 start=468.0625 end=472.21875 metadata=[{'type': 'visual'}] modules=[{'type': 'visual', 'confidence': 'high'}]
video_id=639963bbce36463e0199c8c9 score=85.98 start=416.96875 end=436.34375 metadata=[{'type': 'visual'}] modules=[{'type': 'visual', 'confidence': 'high'}]

The following example sets the value options parameter to ["conversation", "text_in_video", "visual"] and the value of the operator parameter to and to specify that the platform should return the results for which all the selected sources of information match:

from twelvelabs import TwelveLabs

client = TwelveLabs(api_key="<YOUR_API_KEY>")

search_results = client.search.query(
  index_id=INDEX_ID,
  query=QUERY,
  options=["conversation", "text_in_video", "visual"],
  operator="and"
)
# Utility function to print a specific page
def print_page(page):
  for clip in page:
    print(
        f" video_id={clip.video_id} score={clip.score} start={clip.start} end={clip.end} confidence={clip.confidence} metadata={clip.metadata} modules={clip.modules}"
    )

print_page(search_results.data)

while True:
    try:
        print_page(next(search_results))
    except StopIteration:
        break
import { TwelveLabs, SearchData } from 'twelvelabs-js';

const client = new TwelveLabs({ apiKey: '<YOUR_API_KEY>'});

let searchResults = await client.search.query({
  indexId: '<YOUR_INDEX_ID>',
  query: '<YOUR_QUERY>',
  options: ['conversation', 'text_in_video', 'visual'],
  operator: 'and',
});
printPage(searchResults.data);
while (true) {
  const page = await searchResults.next();
  if (page === null) break;
  else printPage(page);
}
// Utility function to print a specific page
function printPage(searchData) {
  (searchData as SearchData[]).forEach((clip) => {
    console.log(
      `video_id= ${clip.videoId} score=${clip.score} start=${clip.start} end=${clip.end} confidence=${clip.confidence} metadata=${JSON.stringify(clip.metadata)} modules=${JSON.stringify(clip.modules)}`,
    );
  });
}

The example output below was truncated for brevity:

video_id=62b3ee6cd01d61be020682f1 score=100.46 start=470 end=482 metadata=[{'type': 'visual'}, {'type': 'conversation', 'text': "We don't care about the goalkeeper who he is away from 234. Wonderful, wonderful, wonderful. How good is he? A near supernatural goal from Lionel messi. He is just brilliant."}, {'type': 'text_in_video', 'text': "22 - WONDER GOAL AGAINST BIGGEST COUNTRY RIVAL"}] modules=[{'type': 'visual', 'confidence': 'high'}, {'type': 'visual', 'confidence': 'medium'}, {'type': 'text_in_video', 'confidence': 'medium'}]

Pagination, filtering, sorting, and grouping

Understanding and utilizing pagination, filtering, and grouping enhances your ability to find and organize information. Follow the links below for more details: