Make any-to-video search requests

Use this endpoint to search for relevant matches in an index using text or various media queries.

Text queries:

Use the query_text parameter to specify your query.

Media queries:

Set the query_media_type parameter to the corresponding media type (example: image).
Specify either one of the following parameters:
- query_media_url: Publicly accessible URL of your media file.
- query_media_file: Local media file. If both query_media_url and query_media_file are specified in the same request, query_media_url takes precedence.

Image requirements

Your images must meet the following requirements:

Format: JPEG and PNG.
Dimension: Must be at least 64 x 64 pixels.
Size: Must not exceed 5MB.

Note

This endpoint is rate-limited. For details, see the Rate limits page.

Use this endpoint to search for relevant matches in an index using text or various media queries. **Text queries**: - Use the `query_text` parameter to specify your query. **Media queries**: - Set the `query_media_type` parameter to the corresponding media type (example: `image`). - Specify either one of the following parameters: - `query_media_url`: Publicly accessible URL of your media file. - `query_media_file`: Local media file. If both `query_media_url` and `query_media_file` are specified in the same request, `query_media_url` takes precedence. <Accordion title="Image requirements"> Your images must meet the following requirements: - **Format**: JPEG and PNG. - **Dimension**: Must be at least 64 x 64 pixels. - **Size**: Must not exceed 5MB. </Accordion> <Note title="Note"> This endpoint is rate-limited. For details, see the [Rate limits](/v1.3/docs/get-started/rate-limits) page. </Note>

Request

Request to perform a search on a video index.

query_media_typeenumOptional

The type of media you wish to use. This parameter is required for media queries. For example, to perform an image-based search, set this parameter to image.

Allowed values:

query_media_urlstringOptional

The publicly accessible URL of the media file you wish to use. This parameter is required for media queries if query_media_file is not provided.

query_media_filefileOptional

The media you wish to use as a local file. This parameter is required for media queries if query_media_url is not provided.

query_textstringOptional

The text query to search for. This parameter is required for text queries. Note that the platform supports full natural language-based search.

index_idstringRequired

The unique identifier of the index to search.

search_optionslist of enumsRequired

Specifies the sources of information the platform uses when performing a search. You must include the search_options parameter separately for each desired source of information.

Notes

The search options you specify must be a subset of the model options used when you created the index.
You can specify multiple search options in conjunction with the operator parameter described below to broaden or narrow your search.

Example: To search using both visual and audio cues, include this parameter twice in the request as shown below:

1 --form search_options=visual \
2 --form search_options=audio \

Specifies the [sources of information](/v1.3/docs/concepts/modalities#search-options) the platform uses when performing a search. You must include the `search_options` parameter separately for each desired source of information. <Note title="Notes"> - The search options you specify must be a subset of the [model options](/v1.3/docs/concepts/modalities#model-options) used when you created the index. - You can specify multiple search options in conjunction with the `operator` parameter described below to broaden or narrow your search. Example: To search using both visual and audio cues, include this parameter twice in the request as shown below: ```JSON --form search_options=visual \ --form search_options=audio \ ``` </Note>

Allowed values:

adjust_confidence_leveldoubleOptional

This parameter specifies the strictness of the thresholds for assigning the high, medium, or low confidence levels to search results. If you use a lower value, the thresholds become more relaxed, and more search results will be classified as having high, medium, or low confidence levels. You can use this parameter to include a broader range of potentially relevant video clips, even if some results might be less precise.

Min: 0 Max: 1 Default: 0.5

group_byenumOptionalDefaults to clip

Use this parameter to group or ungroup items in a response. It can take one of the following values:

video: The platform will group the matching video clips in the response by video.
clip: The matching video clips in the response will not be grouped.

Default: clip

Use this parameter to group or ungroup items in a response. It can take one of the following values: - `video`: The platform will group the matching video clips in the response by video. - `clip`: The matching video clips in the response will not be grouped. **Default:** `clip`

Allowed values:

thresholdenumOptionalDefaults to low

Use this parameter to filter on the level of confidence that the results match your query.

Default: low

Allowed values:

sort_optionenumOptionalDefaults to score

Use this parameter to specify the sort order for the response.

When performing a search, the platform determines the level of confidence that each video clip matches your search terms. By default, the search results are sorted on the level of confidence in descending order.

If you set this parameter to score and group_by is set to video, the platform will determine the maximum value of the score field for each video and sort the videos in the response by the maximum value of this field. For each video, the matching video clips will be sorted by the level of confidence.

If you set this parameter to clip_count and group_by is set to video, the platform will sort the videos in the response by the number of clips. For each video, the matching video clips will be sorted by the level of confidence. You can use clip_count only when the matching video clips are grouped by video.

Default: score

Use this parameter to specify the sort order for the response. When performing a search, the platform determines the level of confidence that each video clip matches your search terms. By default, the search results are sorted on the level of confidence in descending order. If you set this parameter to `score` and `group_by` is set to `video`, the platform will determine the maximum value of the `score` field for each video and sort the videos in the response by the maximum value of this field. For each video, the matching video clips will be sorted by the level of confidence. If you set this parameter to `clip_count` and `group_by` is set to `video`, the platform will sort the videos in the response by the number of clips. For each video, the matching video clips will be sorted by the level of confidence. You can use `clip_count` only when the matching video clips are grouped by video. **Default:** `score`

Allowed values:

operatorenumOptionalDefaults to or

Combines multiple search options using or or and. Use and to find segments matching all search options. Use or to find segments matching any search option.

Default: or.

Allowed values:

page_limitintegerOptionalDefaults to 10

The number of items to return on each page. When grouping by video, this parameter represents the number of videos per page. Otherwise, it represents the maximum number of video clips per page.

Max: 50.

filterstringOptional

Specifies a stringified JSON object to filter your search results. Supports both system-generated metadata (example: video ID, duration) and user-defined metadata.

Syntax for filtering

The following table describes the supported data types, operators, and filter syntax:

Data type	Operator	Description	Syntax
String	`=`	Matches results equal to the specified value.	`{"field": "value"}`
Array of strings	`=`	Matches results with any value in the specified array. Supported only for `id`.	`{"id": ["value1", "value2"]}`
Numeric (integer, float)	`=`, `lte`, `gte`	Matches results equal to or within a range of the specified value.	`{"field": number}` or `{"field": { "gte": number, "lte": number }}`
Boolean	`=`	Matches results equal to the specified boolean value.	`{"field": true}` or `{"field": false}`.

System-generated metadata

The table below describes the system-generated metadata available for filtering your search results:

Field name	Description	Type	Example
`id`	Filters by specific video IDs.	Array of strings	`{"id": ["67cec9caf45d9b64a58340fc", "67cec9baf45d9b64a58340fa"]}`.
`duration`	Filters based on the duration of the video containing the segment that matches your query.	Number or object with `gte` and `lte`	`{"duration": 600}` or `{"duration": { "gte": 600, "lte": 800 }}`
`width`	Filters by video width (in pixels).	Number or object with `gte` and `lte`	`{"width": 1920}` or `{"width": { "gte": 1280, "lte": 1920}}`
`height`	Filters by video height (in pixels).	Number or object with `gte` and `lte`.	`{"height": 1080}` or `{"height": { "gte": 720, "lte": 1080 }}`.
`size`	Filters by video size (in bytes)	Number or object with `gte` and `lte`.	`{"size": 1048576}` or `{"size": { "gte": 1048576, "lte": 5242880}}`
`filename`	Filters by the exact file name.	String	`{"filename": "Animal Encounters part 1"}`

User-defined metadata

To filter by user-defined metadata:

Add metadata to your video by calling the PUT method of the /indexes/:index-id/videos/:video-id endpoint
Reference the custom field in your filter object. For example, to filter videos where a custom field named needsReview of type boolean is true, use {"needs_review": true}.

For more details and examples, see the Filter search results page.

Specifies a stringified JSON object to filter your search results. Supports both system-generated metadata (example: video ID, duration) and user-defined metadata. **Syntax for filtering** The following table describes the supported data types, operators, and filter syntax: | Data type | Operator | Description | Syntax | |:----------|:---------|:------------|:-------| | String | `=` | Matches results equal to the specified value. | `{"field": "value"}` | Array of strings | `=` | Matches results with any value in the specified array. Supported only for `id`. | `{"id": ["value1", "value2"]}` | | Numeric (integer, float) | `=`, `lte`, `gte` | Matches results equal to or within a range of the specified value. | `{"field": number}` or `{"field": { "gte": number, "lte": number }}` | | Boolean | `=` | Matches results equal to the specified boolean value. | `{"field": true}` or `{"field": false}`. | <br/> **System-generated metadata** The table below describes the system-generated metadata available for filtering your search results: | Field name | Description | Type | Example | |:-----------|:------------|:-----|:--------| | `id` | Filters by specific video IDs. | Array of strings | `{"id": ["67cec9caf45d9b64a58340fc", "67cec9baf45d9b64a58340fa"]}`. | | `duration` | Filters based on the duration of the video containing the segment that matches your query. | Number or object with `gte` and `lte` | `{"duration": 600}` or `{"duration": { "gte": 600, "lte": 800 }}` | | `width` | Filters by video width (in pixels). | Number or object with `gte` and `lte` | `{"width": 1920}` or `{"width": { "gte": 1280, "lte": 1920}}` | | `height` | Filters by video height (in pixels). | Number or object with `gte` and `lte`. | `{"height": 1080}` or `{"height": { "gte": 720, "lte": 1080 }}`. | | `size` | Filters by video size (in bytes) | Number or object with `gte` and `lte`. | `{"size": 1048576}` or `{"size": { "gte": 1048576, "lte": 5242880}}` | | `filename` | Filters by the exact file name. | String | `{"filename": "Animal Encounters part 1"}` | <br/> **User-defined metadata** To filter by user-defined metadata: 1. Add metadata to your video by calling the [`PUT`](/v1.3/api-reference/videos/update) method of the `/indexes/:index-id/videos/:video-id` endpoint 2. Reference the custom field in your filter object. For example, to filter videos where a custom field named `needsReview` of type boolean is `true`, use `{"needs_review": true}`. For more details and examples, see the [Filter search results](/v1.3/docs/guides/search/filtering) page.

include_user_metadatabooleanOptional

Specifies whether to include user-defined metadata in the search results.

Response

Successfully performed a search request.

datalist of objects or null

An array that contains your search results. For each match found, the model returns the following fields:

page_infoobject or null

An object that provides information about pagination.

search_poolobject or null

An object that contains details about the index you queried.

Errors

Specifies a stringified JSON object to filter your search results. Supports both system-generated metadata (example: video ID, duration) and user-defined metadata.

Syntax for filtering

The following table describes the supported data types, operators, and filter syntax:

Data type	Operator	Description	Syntax
String	`=`	Matches results equal to the specified value.	`{"field": "value"}`
Array of strings	`=`	Matches results with any value in the specified array. Supported only for `id`.	`{"id": ["value1", "value2"]}`
Numeric (integer, float)	`=`, `lte`, `gte`	Matches results equal to or within a range of the specified value.	`{"field": number}` or `{"field": { "gte": number, "lte": number }}`
Boolean	`=`	Matches results equal to the specified boolean value.	`{"field": true}` or `{"field": false}`.

System-generated metadata

The table below describes the system-generated metadata available for filtering your search results:

Field name	Description	Type	Example
`id`	Filters by specific video IDs.	Array of strings	`{"id": ["67cec9caf45d9b64a58340fc", "67cec9baf45d9b64a58340fa"]}`.
`duration`	Filters based on the duration of the video containing the segment that matches your query.	Number or object with `gte` and `lte`	`{"duration": 600}` or `{"duration": { "gte": 600, "lte": 800 }}`
`width`	Filters by video width (in pixels).	Number or object with `gte` and `lte`	`{"width": 1920}` or `{"width": { "gte": 1280, "lte": 1920}}`
`height`	Filters by video height (in pixels).	Number or object with `gte` and `lte`.	`{"height": 1080}` or `{"height": { "gte": 720, "lte": 1080 }}`.
`size`	Filters by video size (in bytes)	Number or object with `gte` and `lte`.	`{"size": 1048576}` or `{"size": { "gte": 1048576, "lte": 5242880}}`
`filename`	Filters by the exact file name.	String	`{"filename": "Animal Encounters part 1"}`

User-defined metadata

To filter by user-defined metadata:

Add metadata to your video by calling the PUT method of the /indexes/:index-id/videos/:video-id endpoint
Reference the custom field in your filter object. For example, to filter videos where a custom field named needsReview of type boolean is true, use {"needs_review": true}.

For more details and examples, see the Filter search results page.

1	curl -X POST https://api.twelvelabs.io/v1.3/search \
2	-H "x-api-key: <apiKey>" \
3	-H "Content-Type: multipart/form-data" \
4	-F query_media_file=@<file1> \
5	-F query_text="A man walking a dog" \
6	-F index_id="6298d673f1090f1100476d4c" \
7	-F search_options="visual" \
8	-F adjust_confidence_level='0.5' \
9	-F group_by="clip" \
10	-F sort_option="score" \
11	-F operator="or" \
12	-F page_limit='10' \
13	-F filter="{\"id\":[\"66284191ea717fa66a274832\"]}"

1	{
2	"data": [
3	{
4	"score": 85.08,
5	"start": 238.75,
6	"end": 259.62109375,
7	"video_id": "639963a1ce36463e0199c8c7",
8	"confidence": "high",
9	"thumbnail_url": "https://example.com/thumbnail.jpg",
10	"transcription": "A woman vlogs about her summer day, sharing her experience"
11	}
12	],
13	"search_pool": {
14	"total_count": 10,
15	"total_duration": 8731,
16	"index_id": "639961c9e219c90227c371a2"
17	}
18	}

1	--form search_options=visual \
2	--form search_options=audio \

Image requirements

Note

Headers

Request

Notes

Response

Errors

Image requirements

Note

Notes