Guides
SDKsSample appsIntegrationsDiscordPlayground

Search results

Suggest Edits

When you perform a search, the platform returns all the relevant matches. Filtering narrows the scope of your query. The platform allows you to filter your search results based on metadata or the level of confidence that the search results match your query.

📘

Note

The examples on this page are specific to using simple queries. However, the principles demonstrated are similar when using combined queries.

Filtering search results based on metadata

To filter your search results based on metadata, add the filter parameter to the body of your request.

The filter parameter is of type Object and can contain one or more of the following fields:

  • id: An array of strings that restricts your search results based on the specified video IDs.
  • duration: A numeric field that restricts your search results based on duration, expressed in seconds.
  • width: A numeric field that restricts your search results based on width.
  • height: A numeric field that restricts your search results based on height.
  • size: A numeric field that restricts your search results based on size, expressed in bytes.

To indicate the relationship between a field and its value, you can use the exact match or comparison operators.

Exact match operator

The exact match operator matches only the results that equal the value you specify. The syntax is as follows: <field>: <value>.

Comparison operators

Use the comparison operators (lte and gte) to match based on the arithmetic comparison. The syntax is as follows: <field>:{"gte": <value>, "lte": <value}.

Filter composition

You can filter on multiple fields by adding the fields you want to filter on to the filter query parameter as shown in the Filter on size, width, and height section below.

Filtering on the level of confidence

The level of confidence represents the degree of similarity between your query and the search results. You can use the threshold parameter to filter on the level of confidence. This allows you to narrow down a response obtained from the platform by retrieving only the most relevant results. The threshold parameter can take one of the following values:

  • low (this is the default value)
  • medium
  • high

Examples

For a description of each field in the request and response, see the API Reference > Make a search request page.

Prerequisites

  • You’re familiar with the concepts that are described on the Platform overview page.
  • You’ve uploaded at least one video to your index, and the platform has finished indexing it. For details, see the Upload videos page.

Filter on a specific video ID

The following example code uses the id field of the filter query parameter to filter on a specific video ID:

SEARCH_URL = f"{API_URL}/search/"

data = {
  "query": "bear",
  "index_id": INDEX_ID,
  "search_options": ["conversation"],
  "filter": {
    "id": ["629deb6ff05c91527bc5c1c9"]
  }
}

response = requests.post(SEARCH_URL, headers=headers, json=data)
print (f"Status code: {response.status_code}")
pprint(response.json())

const SEARCH_URL = `${API_URL}/search/`

const data = JSON.stringify(
    {
    'query': 'bear',
    'index_id': INDEX_ID,
    'search_options': ['conversation'],
    'filter': {
    'id': ['629deb6ff05c91527bc5c1c9']
    }
})
const config = {
    method: 'post',
    url: SEARCH_URL,
    headers: headers,
    data: data
}
const resp = await axios(config)
const response = await resp.data
console.log(`Status code: ${resp.status}`)
console.log(response)

The following example output was truncated for brevity:

Status code: 200
{'conversation_option': 'semantic',
 'data': [{'confidence': 'low',
           'end': 433.42,
           'metadata': [{'text': "Leave it alone, I have a gun, I'm walking "
                                 "backwards on the trail back to the camp, I'm "
                                 'walking down the trail and mom and two cubs '
                                 'so I am continually to walk backwards.',
                         'type': 'conversation'}],
           'score': 72.95,
           'start': 416.65,
           'video_id': '629deb6ff05c91527bc5c1c9'},
          {'confidence': 'low',
           'end': 279.86,
           'metadata': [{'text': 'Mhm Well that puts an end to my tarpon '
                                 "fishing, get back now when you're playing "
                                 "with lions, it's important to never show "
                                 'fear chuck.',
                         'type': 'conversation'}],
           'score': 68.81,
           'start': 201.74,
           'video_id': '629deb6ff05c91527bc5c1c9'},
          {'confidence': 'low',
           'end': 490.25,
           'metadata': [{'text': 'The mom and the cubs keep following me and '
                                 'or walking at least as fast as I am. People '
                                 'get it. Oh, you, you, you, oh, oh no!',
                         'type': 'conversation'}],
           'score': 68.74,
           'start': 436.88,
           'video_id': '629deb6ff05c91527bc5c1c9'}],
 'page_info': {'limit_per_page': 10,
               'page_expired_at': '2022-06-07T04:14:51Z',
               'total_results': 5},
 'query': 'bear',
 'search_options': ['conversation'],
 'search_pool': {'index_id': '629deb409ea24f052b971993',
                 'total_count': 3,
                 'total_duration': 2198}
}

Note that this query returns only the results found in the specified videos.

Filter on multiple video IDs

The following example code uses the id field of the filter query parameter to filter on multiple video IDs:

SEARCH_URL = f"{API_URL}/search/"

data = {
  "query": "bear",
  "index_id": INDEX_ID,
  "search_options": ["conversation"],
  "filter": {
    "id": ["629deb6ff05c91527bc5c1c9" , "629deb9df05c91527bc5c1cb"]
  }
}

response = requests.post(SEARCH_URL, headers=headers, json=data)
print (f"Status code: {response.status_code}")
pprint(response.json())

const SEARCH_URL = `${API_URL}/search/`

const data = JSON.stringify(
    {
    'query': 'bear',
    'index_id': INDEX_ID,
    'search_options': ['conversation'],
    'filter': {
      'id': ['629deb6ff05c91527bc5c1c9', '629deb9df05c91527bc5c1cb']
    }
})
const config = {
    method: 'post',
    url: SEARCH_URL,
    headers: headers,
    data: data
}
const resp = await axios(config)
const response = await resp.data
console.log(`Status code: ${resp.status}`)
console.log(response)

The following example output was truncated for brevity:

Status code: 200
{'conversation_option': 'semantic',
 'data': [{'confidence': 'high',
           'end': 937.75,
           'metadata': [{'text': 'Bear encounters can be scary, exciting. And '
                                 'even funny. It all really depends on the '
                                 "circumstances you're in.",
                         'type': 'conversation'}],
           'score': 88.4,
           'start': 930.44,
           'video_id': '629deb9df05c91527bc5c1cb'},
          {'confidence': 'high',
           'end': 618.76,
           'metadata': [{'text': 'bear in this next video took things a step '
                                 "further by actually breaking into a person's "
                                 'home in pursuit of none other than some '
                                 'leftover pizza. Okay.',
                         'type': 'conversation'}],
           'score': 87.82,
           'start': 605.34,
           'video_id': '629deb9df05c91527bc5c1cb'},
          {'confidence': 'high',
           'end': 805.3,
           'metadata': [{'text': ' If you were driving through the forest and '
                                 'saw a bear up ahead, what would you do? '
                                 "Surely you'd feel pretty safe knowing you "
                                 'were inside your car. But what if that bear '
                                 'came up to you and began peeking through '
                                 "your window. That's exactly the situation.",
                         'type': 'conversation'}],
           'score': 84.73,
           'start': 788.34,
           'video_id': '629deb9df05c91527bc5c1cb'}],
 'page_info': {'limit_per_page': 10,
               'next_page_token': '48b13711-21e2-4651-8890-52f26a4769ea-1',
               'page_expired_at': '2022-06-07T04:46:19Z',
               'total_results': 40},
 'query': 'bear',
 'search_options': ['conversation'],
 'search_pool': {'index_id': '629deb409ea24f052b971993',
                 'total_count': 3,
                 'total_duration': 2198}
}

Note that this query returns the results found in both videos.

Filter on duration

The example code below uses the duration field of the filter query parameter to return only the matches found in videos that are longer than or equal to 700 seconds AND shorter than or equal to 750 seconds:

SEARCH_URL = f"{API_URL}/search/"

data = {
  "query": "bear",
  "index_id": INDEX_ID,
  "search_options": ["conversation"],
  "filter": {
    "duration": {
        "gte" : 700, "lte": 750
      }
  }
}

response = requests.post(SEARCH_URL, headers=headers, json=data)
print (f"Status code: {response.status_code}")
pprint(response.json())

const SEARCH_URL = `${API_URL}/search/`

const data = JSON.stringify(
    {
    'query': 'bear',
    'index_id': INDEX_ID,
    'search_options': ['conversation'],
    'filter': {
      'duration': {
        'gte' : 700, 'lte': 750
      }
    }
})
const config = {
    method: 'post',
    url: SEARCH_URL,
    headers: headers,
    data: data
}
const resp = await axios(config)
const response = await resp.data
console.log(`Status code: ${resp.status}`)
console.log(response)

The output should look similar to the following one:

Status code: 200
{'conversation_option': 'semantic',
 'data': [{'confidence': 'high',
           'end': 480.35,
           'metadata': [{'text': 'Who? A couple of big bears. Hey bear! Hey '
                                 "bear, go away bear. I don't live there. "
                                 "Don't jump at me bear. I want no trouble.",
                         'type': 'conversation'}],
           'score': 92.28,
           'start': 452.14,
           'video_id': '629deb89f05c91527bc5c1ca'},
          {'confidence': 'mid',
           'end': 521.16,
           'metadata': [{'text': "You got nothing to hump at me for they won't "
                                 'let me bring my bear mace on the plane. '
                                 "You're all good. Keep going there. Cool. "
                                 "That's a lot.",
                         'type': 'conversation'}],
           'score': 80.41,
           'start': 482.54,
           'video_id': '629deb89f05c91527bc5c1ca'}],
 'page_info': {'limit_per_page': 10,
               'page_expired_at': '2022-06-07T05:01:01Z',
               'total_results': 2},
 'query': 'bear',
 'search_options': ['conversation'],
 'search_pool': {'index_id': '629deb409ea24f052b971993',
                 'total_count': 3,
                 'total_duration': 2198}
}

Filtering on size, width, and height

The example code below uses the size, width, and height fields of the filter query parameter to return only the matches found in videos that meet all the following criteria:

  • Size is greater than or equal to 50000000 bytes and less and equal to 53000000 bytes.
  • Width is greater than or equal to 850.
  • Height is greater than or equal to 400 and less and equal to 500.
SEARCH_URL = f"{API_URL}/search/"

data = {
  "query": "bear",
  "index_id": INDEX_ID,
  "search_options": ["conversation"],
  "filter": {
    "size": {
      "gte": 50000000, "lte": 53000000
    },
    "width":
      {
        "gte": 850
      },
    "height":
      {
        "gte": 400, "lte": 500
      }
  }
}

response = requests.post(SEARCH_URL, headers=headers, json=data)
print (f"Status code: {response.status_code}")
pprint(response.json())

const SEARCH_URL = `${API_URL}/search/`

const data = JSON.stringify(
    {
    'query': 'bear',
    'index_id': INDEX_ID,
    'search_options': ['conversation'],
    'filter': {
      'size': {
        'gte': 50000000, 'lte': 53000000
      },
      'width':
        {
          "gte": 850
        },
      'height':
        {
          'gte': 400, 'lte': 500
        }
    }
})
const config = {
    method: 'post',
    url: SEARCH_URL,
    headers: headers,
    data: data
}
const resp = await axios(config)
const response = await resp.data
console.log(`Status code: ${resp.status}`)
console.log(response)

The following example output was truncated for brevity:

Status code: 200
{
  "conversation_option": "semantic",
  "data": [
    {
      "score": 88.4,
      "start": 930.44,
      "end": 937.75,
      "metadata": [
        {
          "type": "conversation",
          "text": "Bear encounters can be scary, exciting. And even funny. It all really depends on the circumstances you're in."
        }
      ],
      "video_id": "629deb9df05c91527bc5c1cb",
      "confidence": "high"
    },
    {
      "score": 84.74,
      "start": 937.79,
      "end": 947.56,
      "metadata": [
        {
          "type": "conversation",
          "text": "While bears aren't necessarily always aggressive creatures, it's probably best to keep your distance. Should you ever run into one in the wild?"
        }
      ],
      "video_id": "629deb9df05c91527bc5c1cb",
      "confidence": "high"
    },
    {
      "score": 84.73,
      "start": 788.34,
      "end": 805.3,
      "metadata": [
        {
          "type": "conversation",
          "text": " If you were driving through the forest and saw a bear up ahead, what would you do? Surely you'd feel pretty safe knowing you were inside your car. But what if that bear came up to you and began peeking through your window. That's exactly the situation."
        }
      ],
      "video_id": "629deb9df05c91527bc5c1cb",
      "confidence": "high"
    }
  ],
  "page_info": {
    "limit_per_page": 10,
    "next_page_token": "c1f893d1-0d05-4185-8181-394529c9dca6-1",
    "page_expired_at": "2022-06-07T05:16:05Z",
    "total_results": 35
  },
  "query": "bear",
  "search_options": [
    "conversation"
  ],
  "search_pool": {
    "index_id": "629deb409ea24f052b971993",
    "total_count": 3,
    "total_duration": 2198
  }
}

Filtering on custom metadata

The example code below filters on a custom field named views of type integer. For details about specifying custom metadata, see the Provide custom metadata page. The platform will return only the results found in the videos for which the value of the views field equals 120000.

SEARCH_URL = f"{API_URL}/search"

data = {
  "query": "red car",
  "index_id": INDEX_ID,
  "search_options": ["visual"],
  "filter": {
    "views": 120000
  }
}

response = requests.post(SEARCH_URL, headers=headers, json=data)
print (f'Status code: {response.status_code}')
pprint(response.json())

const SEARCH_URL = `${API_URL}/search`

const data = {
  'query': 'bear chasing a man',
  'index_id': INDEX_ID,
  'search_options': ['visual'],
  'filter': {
    'views': 120000
  }
}

const resp = await axios.post(
  SEARCH_URL,
  data,
  {
    'headers': {
      'x-api-key': API_KEY
    }
  }
)
const { data: response } = resp;
console.log(`Status code: ${resp.status}`)
console.log(JSON.stringify(response,null,4))

The following output was truncated for brevity:

Status code: 200
{
  "search_pool": {
    "total_count": 13,
    "total_duration": 8731,
    "index_id": "639961c9e219c90227c371a2"
  },
  "data": [
    {
      "score": 84.52,
      "start": 409.95625,
      "end": 417.6375,
      "metadata": [
        {
          "type": "visual"
        }
      ],
      "video_id": "63a2a2a7ce36463e0199c8e5",
      "confidence": "high",
      "thumbnail_url": "https://project-one-thumbnail.s3.us-west-2.amazonaws.com/63a2a2a7ce36463e0199c8e5/410.jpeg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAYRWJPOVHXE5SJ77T%2F20221228%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20221228T164646Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&x-id=GetObject&X-Amz-Signature=baf02f7a95bb2bc4d38f6b0a776e4382c4ff2a20ca9a75a021c6f4d3f385583a"
    },
    {
      "score": 66.62,
      "start": 81.64999999999999,
      "end": 83.2625,
      "metadata": [
        {
          "type": "visual"
        }
      ],
      "video_id": "63a2a2a7ce36463e0199c8e5",
      "confidence": "low",
      "thumbnail_url": "https://project-one-thumbnail.s3.us-west-2.amazonaws.com/63a2a2a7ce36463e0199c8e5/82.jpeg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAYRWJPOVHXE5SJ77T%2F20221228%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20221228T164646Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&x-id=GetObject&X-Amz-Signature=395ffcb28f548f879c15ab3004c4e975c89a99b0b4dc2300b7bbd61af5a1f54b"
    },
    {
      "score": 64.48,
      "start": 554.21875,
      "end": 564.625,
      "metadata": [
        {
          "type": "visual"
        }
      ],
      "video_id": "63a2a2a7ce36463e0199c8e5",
      "confidence": "low",
      "thumbnail_url": "https://project-one-thumbnail.s3.us-west-2.amazonaws.com/63a2a2a7ce36463e0199c8e5/555.jpeg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAYRWJPOVHXE5SJ77T%2F20221228%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20221228T164646Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&x-id=GetObject&X-Amz-Signature=0916e50d66158aaabb500b066cee20afc1528df8fb798ce342342358d96bd723"
    }
  ],
  "page_info": {
    "limit_per_page": 10,
    "total_results": 32,
    "page_expired_at": "2022-12-28T16:46:46Z",
    "next_page_token": "5f5e2da3-f8ff-4f49-a001-bba58b5d5e83-1"
  }
}

Filtering on the level of confidence

The following example code specifies that the minimum level of confidence shouldn't be lower than medium:

SEARCH_URL = f"{API_URL}/search"

data = {
  "query": "car accidents",
  "index_id": INDEX_ID,
  "search_options": ["visual"],
  "threshold": "medium",
}

response = requests.post(SEARCH_URL, headers=headers, json=data)
print(f"Status code: {response.status_code}")
pprint(response.json())


const SEARCH_URL = `${API_URL}/search`

const data = {
  'query': 'car accidents',
  'index_id': INDEX_ID,
  'search_options': ['visual'],
  'threshold': 'medium',
}

const resp = await axios.post(
  SEARCH_URL,
  data,
  {
    'headers': {
      'x-api-key': API_KEY
    }
  }
)
const { data: response } = resp;
console.log(`Status code: ${resp.status}`)
console.log(JSON.stringify(response,null,4))

The following example output was truncated for brevity:

Status code: 200
{
  "search_pool": {
    "total_count": 15,
    "total_duration": 10228,
    "index_id": "639961c9e219c90227c371a2"
  },
  "data": [
    {
      "score": 80.28,
      "start": 25.421875,
      "end": 31.375,
      "metadata": [
        {
          "type": "visual"
        }
      ],
      "video_id": "63b2c707ce36463e0199c906",
      "confidence": "medium"
    },
    {
      "score": 75.25,
      "start": 266.765625,
      "end": 270.5,
      "metadata": [
        {
          "type": "visual"
        }
      ],
      "video_id": "639963a1ce36463e0199c8c7",
      "confidence": "medium"
    {
      "score": 75.09,
      "start": 119.796875,
      "end": 125.71875,
      "metadata": [
        {
          "type": "visual"
        }
      ],
      "video_id": "639963a1ce36463e0199c8c7",
      "confidence": "high"
    },
    {
      "score": 73.96,
      "start": 300.475,
      "end": 304.15625,
      "metadata": [
        {
          "type": "visual"
        }
      ],
      "video_id": "639963a1ce36463e0199c8c7",
      "confidence": "high"
    }
  ],
  "page_info": {
    "limit_per_page": 10,
    "total_results": 4,
    "page_expired_at": "2023-01-17T08:47:23Z"
  }
}

In the example output above, note that the data array is composed of a list of objects. Each object corresponds to a matching video segment and has a field named confidence whose value is either medium or high.

Updated over 1 year ago