Text queries

Text queries allow you to use natural language to find video segments matching specific keywords or phrases.

Note the following about using text queries:

  • The platform supports full natural language-based search. The following examples are valid queries: “birds flying near a castle,” “sun shining on the water,” “chickens on the road,” “an officer holding a child’s hand,” and “crowd cheering in the stadium.”
  • The platform supports queries in multiple languages. For a complete list, see the Supported languages page.

Examples

This section provides examples of performing search requests using single queries. Ensure that the prerequisites are met before proceeding.

Visual

The platform analyzes video content as you would see and hear from it, including actions, objects, sounds, and events, excluding human speech. To search using visual cues, invoke the query method of the search object with the following parameters:

  • index_id: A string representing the unique identifier of the index to search.
  • query_text: A string representing your search query. Note that the platform supports full natural language-based search. The following examples are valid queries: “birds flying near a castle,” “sun shining on the water,” “chickens on the road,” “an officer holding a child’s hand,” and “crowd cheering in the stadium.”
  • options: An array of strings specifying the sources of information the platform uses when performing a search. This example searches using visual cues.
1from twelvelabs import TwelveLabs
2
3client = TwelveLabs(api_key="<YOUR_API_KEY>")
4
5search_results = client.search.query(
6  index_id="<YOUR_INDEX_ID>", 
7  query_text="<YOUR_QUERY>", 
8  options=["visual"]
9)
10
11# Utility function to print a specific page
12def print_page(page):
13  for clip in page:
14    print(
15        f" video_id={clip.video_id} score={clip.score} start={clip.start} end={clip.end} confidence={clip.confidence} metadata={clip.metadata}"
16    )
17
18print_page(search_results.data)
19
20while True:
21    try:
22        print_page(next(search_results))
23    except StopIteration:
24        break

The example output below was truncated for brevity:

video_id=65d60bcf48db9fa780cb415e score=83.73 start=273.96875 end=289.0625 confidence=high metadata=[{'type': 'visual'}]
video_id=65d60bcf48db9fa780cb415e score=83.55 start=397.921875 end=439.84375 confidence=high metadata=[{'type': 'visual'}]
video_id=65d60bcf48db9fa780cb415e score=83.46 start=294.5625 end=311.84375 confidence=high metadata=[{'type': 'visual'}]
video_id=65d60bcf48db9fa780cb415e score=83.45 start=233.0 end=247.78125 confidence=high metadata=[{'type': 'visual'}]

Conversation

The platform analyzes human speech within your videos. The following example sets the value of the options parameter to ["conversation"] to search using human speech as the source of information. Note that you can perform semantic and exact searches by setting the value of the conversation_option parameter. See the Conversation option page for details.

1from twelvelabs import TwelveLabs
2
3client = TwelveLabs(api_key="<YOUR_API_KEY>")
4
5search_results = client.search.query(
6    index_id="<YOUR_INDEX_ID>", 
7    query_text="<YOUR_QUERY>" 
8    options=["conversation"], 
9    conversation_option="semantic" # Optional
10)
11
12# Utility function to print a specific page
13def print_page(page):
14  for clip in page:
15    print(
16        f" video_id={clip.video_id} score={clip.score} start={clip.start} end={clip.end} confidence={clip.confidence} metadata={clip.metadata}"
17    )
18
19print_page(search_results.data)
20
21while True:
22    try:
23        print_page(next(search_results))
24    except StopIteration:
25        break

The example output below was truncated for brevity:

 video_id=65d5fbad48db9fa780cb415c score=70.05 start=347.04 end=363.929 confidence=low metadata=[{'type': 'conversation', 'text': "And with that goal, Lionel Messi becomes Argentina's all time leading scorer."}]
 video_id=65d5fbad48db9fa780cb415c score=58.75 start=366.869 end=405.589 confidence=low metadata=[{'type': 'conversation', 'text': 'I guess it deserved that goal scorer years ago just like good saves in the second period. Just when you think comes up with something more special.'}]

Text in video

The platform detects and extracts text (OCR) shown within your videos. The following example sets the value of the options parameter to ["text_in_video"] to search for text that matches your search query:

1from twelvelabs import TwelveLabs
2
3client = TwelveLabs(api_key="<YOUR_API_KEY>")
4
5search_results = client.search.query(
6  index_id="<YOUR_INDEX_ID>", 
7  query_text="<YOUR_QUERY>", 
8  options=["text_in_video"]
9)
10
11# Utility function to print a specific page
12def print_page(page):
13  for clip in page:
14    print(
15        f" video_id={clip.video_id} score={clip.score} start={clip.start} end={clip.end} confidence={clip.confidence} metadata={clip.metadata}"
16    )
17
18print_page(search_results.data)
19
20while True:
21    try:
22        print_page(next(search_results))
23    except StopIteration:
24        break

The platform detects and extracts brand logos as shown within your videos. For this, call the query method of the search object with the following parameters:

  • query_text: The name of the company. This example uses Starbucks.
  • options: The source of information the platform must use (logo).
  • index_id: The unique identifier of the index containing your videos.

The example code below finds when the Starbucks company logo appears in your videos:

1from twelvelabs import TwelveLabs
2
3client = TwelveLabs(api_key="<YOUR_API_KEY>")
4
5search_results = client.search.query(
6  index_id="<YOUR_INDEX_ID>", 
7  query_text="Starbucks", 
8  options=["logo"]
9)
10
11# Utility function to print a specific page
12def print_page(page):
13  for clip in page:
14    print(
15        f" video_id={clip.video_id} score={clip.score} start={clip.start} end={clip.end} confidence={clip.confidence} metadata={clip.metadata}"
16    )
17
18print_page(search_results.data)
19
20while True:
21    try:
22        print_page(next(search_results))
23    except StopIteration:
24        break

The following example output was truncated for brevity:

video_id=63eb67e49678ed7709731d63 score=92.28 start=104 end=106 confidence=high metadata=[{'type': 'logo', 'text': 'Starbucks'}]

In this example output, the Starbucks logo appears at 104 seconds from the start of the video:


Visual and conversation

The following example sets the value options parameter to ["visual", "conversation"] and the value of the operator parameter to or to specify that the platform should return the results for which any of the selected sources of information match:

1from twelvelabs import TwelveLabs
2
3client = TwelveLabs(api_key="<YOUR_API_KEY>")
4
5search_results = client.search.query(
6  index_id="<YOUR_INDEX_ID>",
7  query_text="<YOUR_QUERY>",
8  options=["visual","conversation"],
9  operator="or"
10)
11# Utility function to print a specific page
12def print_page(page):
13  for clip in page:
14    print(
15        f" video_id={clip.video_id} score={clip.score} start={clip.start} end={clip.end} confidence={clip.confidence} metadata={clip.metadata} modules={clip.modules}"
16    )
17
18print_page(search_results.data)
19
20while True:
21    try:
22        print_page(next(search_results))
23    except StopIteration:
24        break

The example output below was truncated for brevity:

video_id=639963a1ce36463e0199c8c7 score=76.79 start=420.66 end=430.23 confidence=medium metadata=[{'type': 'visual'}, {'type': 'conversation'}] modules=[{'type': 'conversation', 'confidence': 'medium'}, {'type': 'visual', 'confidence': 'low'}]
video_id=639963a1ce36463e0199c8c7 score=72.27 start=279.21875 end=289.75 confidence=low metadata=[{'type': 'visual'}] modules=[{'type': 'visual', 'confidence': 'low'}]
video_id=639963a1ce36463e0199c8c7 score=72.2 start=87.38 end=98.73 confidence=low metadata=[{'type': 'conversation'}] modules=[{'type': 'conversation', 'confidence': 'low'}]

The following example sets the value options parameter to ["visual", "conversation"] and the value of the operator parameter to and to specify that the platform should return the results for which all the selected sources of information match:

1from twelvelabs import TwelveLabs
2
3client = TwelveLabs(api_key="<YOUR_API_KEY>")
4
5search_results = client.search.query(
6  index_id="<YOUR_INDEX_ID>",
7  query_text="<YOUR_QUERY>",
8  options=["visual","conversation"],
9  operator="and"
10)
11# Utility function to print a specific page
12def print_page(page):
13  for clip in page:
14    print(
15        f" video_id={clip.video_id} score={clip.score} start={clip.start} end={clip.end} confidence={clip.confidence} metadata={clip.metadata} modules={clip.modules}"
16    )
17
18print_page(search_results.data)
19
20while True:
21    try:
22        print_page(next(search_results))
23    except StopIteration:
24        break

The example output below was truncated for brevity:

video_id=639963a1ce36463e0199c8c7 score=76.79 start=420.66 end=430.23 confidence=medium metadata=[{'type': 'visual'}, {'type': 'conversation'}] modules=[{'type': 'conversation', 'confidence': 'medium'}, {'type': 'visual', 'confidence': 'low'}]
video_id=6399637ace36463e0199c8c6 score=58.99 start=665.745 end=676.795 confidence=low metadata=[{'type': 'visual'}, {'type': 'conversation'}] modules=[{'type': 'conversation', 'confidence': 'low'}, {'type': 'visual', 'confidence': 'low'}]

Conversation and text in video

The following example sets the value options parameter to ["conversation", "text_in_video"] and the value of the operator parameter to or to specify that the platform should return the results for which any of the selected sources of information match:

1from twelvelabs import TwelveLabs
2
3client = TwelveLabs(api_key="<YOUR_API_KEY>")
4
5search_results = client.search.query(
6  index_id="<YOUR_INDEX_ID>",
7  query_text="<YOUR_QUERY>",
8  options=["conversation", "text_in_video"],
9  operator="or"
10)
11# Utility function to print a specific page
12def print_page(page):
13  for clip in page:
14    print(
15        f" video_id={clip.video_id} score={clip.score} start={clip.start} end={clip.end} confidence={clip.confidence} metadata={clip.metadata} modules={clip.modules}"
16    )
17
18print_page(search_results.data)
19
20while True:
21    try:
22        print_page(next(search_results))
23    except StopIteration:
24        break

The example output below was truncated for brevity:

video_id=639963d8ce36463e0199c8ca score=69.72 start=430 end=433 confidence=low metadata=[{'type': 'text_in_video', 'text': "Messi scores a goal similar to the Maradona's "Goal of the Century" ( vs Getafe in 2007)"}] modules=[{'type': 'text_in_video', 'confidence': 'low'}]
video_id=639963d8ce36463e0199c8ca score=69.72 start=597 end=621 confidence=low metadata=[{'type': 'text_in_video', 'text': "But Lucas Moura scores a hat-trick in second-half (last goal at the 96th minute) and qualify Tottenham for the 2019 UCL final"}] modules=[{'type': 'text_in_video', 'confidence': 'low'}]
video_id=639963afce36463e0199c8c8 score=62.33 start=271.1 end=311.64 confidence=low metadata=[{'type': 'conversation', 'text': " Been some something special for his hat trick goal last. Well, I'll tell you what, you're not gonna see. Much better goals than that. What a hat trick. And what a third goal down. But it's the moment of magic that real Madrid had been waiting for his way, way on side."}] modules=[{'type': 'conversation', 'confidence': 'low'}]

The following example sets the value options parameter to ["conversation", "text_in_video"] and the value of the operator parameter to and to specify that the platform should return only the results for which all the selected sources of information match:

1from twelvelabs import TwelveLabs
2
3client = TwelveLabs(api_key="<YOUR_API_KEY>")
4
5search_results = client.search.query(
6  index_id=INDEX_ID,
7  query_text="<YOUR_QUERY>",
8  options=["conversation", "text_in_video"],
9  operator="and"
10)
11# Utility function to print a specific page
12def print_page(page):
13  for clip in page:
14    print(
15        f" video_id={clip.video_id} score={clip.score} start={clip.start} end={clip.end} confidence={clip.confidence} metadata={clip.metadata}"
16    )
17
18print_page(search_results.data)
19
20while True:
21    try:
22        print_page(next(search_results))
23    except StopIteration:
24        break

The example output below was truncated for brevity:

video_id=62b3ee6cd01d61be020682f1 confidence=high score=96.26 start=469.58 end=489.06 metadata=[{'type': 'conversation', 'text': "We don't care about the goalkeeper who he is away from 234. Wonderful, wonderful, wonderful. How good is he? A near supernatural goal from Lionel Messi. He is just brilliant."}, {'type': 'text_in_video', 'text': "22 - WONDER GOAL AGAINST BIGGEST COUNTRY RIVAL"}] modules=[{'type': 'conversation', 'confidence': 'high'}, {'type': 'text_in_video', 'confidence': 'high'}]

Conversation, text in video, and visual

The following example sets the value options parameter to ["conversation", "text_in_video", "visual"] and the value of the operator parameter to or to specify that the platform should return the results for which any of the selected sources of information match:

1from twelvelabs import TwelveLabs
2
3client = TwelveLabs(api_key="<YOUR_API_KEY>")
4
5search_results = client.search.query(
6  index_id="<YOUR_INDEX_ID>",
7  query_text="<YOUR_QUERY>",
8  options=["conversation", "text_in_video", "visual"],
9  operator="or"
10)
11# Utility function to print a specific page
12def print_page(page):
13  for clip in page:
14    print(
15        f" video_id={clip.video_id} score={clip.score} start={clip.start} end={clip.end} confidence={clip.confidence} metadata={clip.metadata} modules={clip.modules}"
16    )
17
18print_page(search_results.data)
19
20while True:
21    try:
22        print_page(next(search_results))
23    except StopIteration:
24        break

The example output below was truncated for brevity:

video_id=639963d8ce36463e0199c8ca score=87 start=611.8125 end=614.84375 metadata=[{'type': 'visual'}, {'type': 'text_in_video', 'text': "But Lucas Moura scores a hat-trick in second-half (last goal at the 96th minute) and qualify Tottenham for the 2019 UCL final"}] modules=[{'type': 'visual', 'confidence': 'high'}, {'type': 'text_in_video', 'confidence': 'low'}]
video_id=639963d8ce36463e0199c8ca score=86.03 start=468.0625 end=472.21875 metadata=[{'type': 'visual'}] modules=[{'type': 'visual', 'confidence': 'high'}]
video_id=639963bbce36463e0199c8c9 score=85.98 start=416.96875 end=436.34375 metadata=[{'type': 'visual'}] modules=[{'type': 'visual', 'confidence': 'high'}]

The following example sets the value options parameter to ["conversation", "text_in_video", "visual"] and the value of the operator parameter to and to specify that the platform should return the results for which all the selected sources of information match:

1from twelvelabs import TwelveLabs
2
3client = TwelveLabs(api_key="<YOUR_API_KEY>")
4
5search_results = client.search.query(
6  index_id=INDEX_ID,
7  query_text="<YOUR_QUERY>",
8  options=["conversation", "text_in_video", "visual"],
9  operator="and"
10)
11# Utility function to print a specific page
12def print_page(page):
13  for clip in page:
14    print(
15        f" video_id={clip.video_id} score={clip.score} start={clip.start} end={clip.end} confidence={clip.confidence} metadata={clip.metadata} modules={clip.modules}"
16    )
17
18print_page(search_results.data)
19
20while True:
21    try:
22        print_page(next(search_results))
23    except StopIteration:
24        break

The example output below was truncated for brevity:

video_id=62b3ee6cd01d61be020682f1 score=100.46 start=470 end=482 metadata=[{'type': 'visual'}, {'type': 'conversation', 'text': "We don't care about the goalkeeper who he is away from 234. Wonderful, wonderful, wonderful. How good is he? A near supernatural goal from Lionel messi. He is just brilliant."}, {'type': 'text_in_video', 'text': "22 - WONDER GOAL AGAINST BIGGEST COUNTRY RIVAL"}] modules=[{'type': 'visual', 'confidence': 'high'}, {'type': 'visual', 'confidence': 'medium'}, {'type': 'text_in_video', 'confidence': 'medium'}]
Built with