For AI agents: a documentation index is available at the root level at /llms.txt and /llms-full.txt. Append /llms.txt to any URL for a page-level index, or .md for the markdown version of any page.
Sample appsIntegrationsDiscordPlaygroundDevEx repo
GuidesSDK ReferenceAPI Reference
GuidesSDK ReferenceAPI Reference
  • Python SDK
    • The TwelveLabs class
    • Manage indexes
    • Upload content
    • Index content
    • Manage videos
    • Manage entities
    • Search
    • Create embeddings v2
    • Create embeddings v1
    • Analyze and segment videos
  • Node.js SDK
    • The TwelveLabs class
    • Manage indexes
    • Upload content
    • Index content
    • Manage videos
    • Manage entities
    • Search
    • Create embeddings v2
    • Create embeddings v1
    • Analyze and segment videos
LogoLogo
Sample appsIntegrationsDiscordPlaygroundDevEx repo
On this page
  • Methods
  • Make a search request
  • Parameters
  • Return value
  • API Reference
  • Related guides
  • Error codes
Node.js SDK

Search

Was this page helpful?
Previous

Create embeddings v2

Next
Built with

The SearchWrapper class provides methods to perform search requests.

Methods

Make a search request

Description: This method performs a search across a specific index using text, media, or a combination of both as your query and returns a paginated iterator of search results.

Text queries:

  • Use the queryText parameter to specify your query.

Media queries:

  • Set the queryMediaType parameter to the corresponding media type (example: image).
  • For a single image, specify one of the following parameters:
    • queryMediaUrl: Publicly accessible URL of your media file.
    • queryMediaFile: Local media file. If you specify both, queryMediaUrl takes precedence.
  • For multiple images (up to 10), specify one of the following parameters:
    • queryMediaUrls: Publicly accessible URLs of your media files.
    • queryMediaFiles: Local media files.

Composed text and media queries:

  • Use the queryText parameter for your text query.
  • Set queryMediaType to image.
  • Specify your images using queryMediaUrl, queryMediaFile, queryMediaUrls, or queryMediaFiles.
Note

When using images in your search queries (either as media queries or in composed searches), ensure your image files meet the requirements.

Entity search:

  • To find a specific person in your videos, enclose the unique identifier of the entity you want to find in the queryText parameter.

For instructions on setting up and using this feature, see the Entity search page.

Note

This method is rate-limited. For details, see the Rate limits page.

Function signature and example:

1query(
2  request: SearchWrapper.QueryRequest,
3  requestOptions?: Search.RequestOptions
4): Promise<core.Page<TwelvelabsApi.SearchItem>>

Parameters

NameTypeRequiredDescription
request[TwelvelabsApi.SearchCreateRequest]YesThe search request parameters.
requestOptions[Search.RequestOptions]NoRequest-specific configuration.

The SearchCreateRequest interface defines the parameters for performing a search:

NameTypeRequiredDescription
indexIdstringYesThe unique identifier of the index to search.
searchOptionsTwelvelabsApi.
SearchCreateRequestSearchOptionsItem[]		YesSpecifies the modalities the video understanding model uses to find relevant information.
Available options:
- visual: Searches visual content.
- audio: Searches non-speech audio.
- transcription: Spoken words
You can specify multiple search options in conjunction with the operator parameter to broaden or narrow your search.
For guidance, see the Search options section.
transcriptionOptionsTwelvelabsApi.
SearchCreateRequestTranscriptionOptionsItem[]		NoSpecifies how the platform matches your text query with the words spoken in the video. This parameter applies only when the search_options parameter contains the transcription value.
Available options:
- lexical: Exact word matching
- semantic: Meaning-based matching
For details on when to use each option, see the Transcription options section.
Default: ["lexical", "semantic"].
queryTextstringNoThe text query to search for. This parameter is required for text queries. Marengo supports up to 500 tokens per query.
queryMediaTypeTwelvelabsApi.SearchCreateRequestQueryMediaTypeNoThe type of media you wish to use. This parameter is required for media queries. For example, to perform an image-based search, set this parameter to image. Use queryText together with this parameter when you want to perform a composed image+text search.
queryMediaFileFile | fs.ReadStream | Blob | (File | fs.ReadStream | Blob)[]NoA local media file to use as a query. Pass a single file or an array of up to 10 files. This parameter is required for media queries if queryMediaUrl is not provided.
queryMediaUrlstringNoThe publicly accessible URL of a media file to use as a query. This parameter is required for media queries if queryMediaFile is not provided.
queryMediaFiles(File | fs.ReadStream | Blob)[]NoA list of local media files to use as a query. You can provide up to 10 images in total.
queryMediaUrlsstring[]NoA list of publicly accessible URLs of media files to use as a query. You can provide up to 10 images in total.
groupByTwelvelabsApi.SearchCreateRequestGroupByNoUse this parameter to group or ungroup items in a response. Values: video, clip. Default: clip.
operatorTwelvelabsApi.SearchCreateRequestOperatorNoLogical operator for combining search options. Values: or, and. Default: or.
pageLimitnumberNoThe number of items to return on each page. Max: 50.
filterstringNoA stringified object to filter search results based on video metadata or custom fields.
includeUserMetadatabooleanNoSpecifies whether to include user-defined metadata in the search results.

Return value

Returns a Promise that resolves to a Page<SearchItem> object that implements AsyncIterable, allowing you to iterate through the paginated search results.

The Page class contains the following properties and methods:

NameTypeDescription
dataT[]An array containing the current page of items.
getNextPage()Promise<this>Retrieves the next page and returns the updated Page object.
hasNextPage()booleanReturns whether there is a next page to load.
Symbol.asyncIteratorAsyncIterator<T>Allows iteration through all items across all pages using for await loops.

The SearchItem interface contains the following properties:

NameTypeDescription
startnumberThe start time of the clip in seconds.
endnumberThe end time of the clip in seconds.
videoIdstringThe unique identifier of the video. Once the platform indexes a video, it assigns a unique identifier.
ranknumberThe relevance ranking assigned by the model. Lower numbers indicate higher relevance, starting with 1 for the most relevant result.
thumbnailUrlstringThe URL of the thumbnail image for the clip.
transcriptionstringA transcription of the spoken words that are captured in the video.
idstringThe unique identifier of the video. Only appears when the group_by=video parameter is used in the request.
userMetadataRecord<string, unknown>User-defined metadata associated with the video.
clipsSearchItemClipsItem[]An array that contains detailed information about the clips that match your query. The platform returns this array only when the group_by parameter is set to video in the request.

The SearchItemClipsItem interface contains the following properties:

NameTypeDescription
startnumberThe start time of the clip in seconds.
endnumberThe end time of the clip in seconds.
ranknumberThe relevance ranking assigned by the model. Lower numbers indicate higher relevance, starting with 1 for the most relevant result.
thumbnailUrlstringThe URL of the thumbnail image for the clip.
transcriptionstringA transcription of the spoken words that are captured in the clip.
videoIdstringThe unique identifier of the video for the corresponding clip.
userMetadataRecord<string, unknown>User-defined metadata associated with the video.

API Reference

Any-to-video search.

Related guides

  • Search
  • Filtering
  • Grouping

Error codes

This section lists the most common error messages you may encounter while performing search requests.

  • search_option_not_supported
    • Search option {search_option} is not supported for index {index_id}. Please use one of the following search options: {supported_search_option}.
  • search_option_combination_not_supported
    • Search option {search_option} is not supported with {other_combination}.
  • search_filter_invalid
    • Filter used in search is invalid. Please use the valid filter syntax by following filtering documentation.
  • search_page_token_expired
    • The token that identifies the page to be retrieved is expired or invalid. You must make a new search request. Token: {next_page_token}.
  • index_not_supported_for_search:
    • You can only perform search requests on indexes with an engine from the Marengo family enabled.

For a list of general errors that apply to all endpoints, see the Error codes page.