Query engineering
Query engineering is the process of crafting and refining search queries to enhance the relevancy and accuracy of your search results. The key to successful query engineering is understanding the platform's capabilities and then formulating your queries to maximize these capabilities effectively.
Steps in query engineering
While there isn't a singular approach to querying our multimodal foundation model, the steps presented in this section will enhance your querying engineering proficiency and help you develop an understanding through experimentation.
- Clearly define the objective: Determine what you want to retrieve. It can be a specific action, event, object, or something more complex, like a series of activities or people with various attributes.
- Choose the type of query: Based on the objective established in the previous step, select the appropriate type of query - either single or combined. Choose a single query to search for matches of the same query across multiple sources of information. Choose a combined query to search for different elements spanning across multiple sources of information.
- Create the initial query: Craft your initial query, specifying the sources of information you want to search within.
- Refine and improve: If the platform isn't providing the expected search results, make your prompt more explicit. For example, instead of using broad terms like
man
orguitar
, use more descriptive phrases likeman playing guitar in the rain
. Experiment with different adjectives, sequences, or sources of information. This helps in narrowing down the search.
Tips for writing better queries
Effective query engineering can significantly enhance the quality of results returned by the platform. The tips in this section will help you write better queries.
Be as specific and descriptive as possible
When crafting a query, specificity is often paramount. While single keywords might yield many results, they often don't capture the context. Thus, using descriptive phrases significantly enhances the relevance of your search results.
Examples:
- Instead of
dancer
, useballet dancer performing on a stage with dim lighting
. The additional context gives the platform insight into the type of dance and ambiance. - Instead of
dog
, usedog playing fetch in a park
. The revised query clearly specifies an activity and setting.
See the Find the perfect adrenaline rush activity for your next vacation section below for a practical example of how using descriptive phrases significantly improves the accuracy of your search results.
Use synonyms
Every word comes with a set of synonyms that can offer slight variances in meaning. If the platform doesn't return the expected results when using a specific word, try its synonyms.
Examples:
- Instead of
automobile
, considercar
orvehicle
to narrow down your search results. - Instead of
house
, experiment withbungalow
orvilla
to target a particular style.
Consider subjectivity
Videos often contain content that can be open to interpretation, given the inherently subjective nature of certain concepts. When searching for such information, it's important to understand that your query may not capture the full breadth of relevant content. Thus, iterative querying (adjusting, refining, and trying different phrasings or perspectives) becomes essential.
Examples:
- Using
beautiful scenery
as your search query might return both a serene mountain or a beach sunset. To refine this query, specify eithersnow-capped mountain scenery
ortropical beach at sunset
. - Using
dogs doing something funny
as your search query might return both a dog chasing its tail or a puppy getting startled by a harmless object. Similar to the above, refine this query by specifying what you find humorous.
Use natural language
Instead of relying on traditional keyword-based search terms, help the platform deliver more accurate and contextually relevant results by expressing your queries in a more conversational or natural manner.
Examples:
- Instead of
guitar, solo, concert
, usefootage where a guitarist plays a solo during a concert
. - Instead of
cat, dog, chase
, usevideos where a cat is chasing a dog
.
Rephrase domain-specific terms
Occasionally, the platform might not accurately recognize domain-specific terms. A more descriptive query that uses everyday language can enhance the accuracy of your search results in such instances.
Examples:
- In a football context, instead of
hurdle
, usea football player jumping over another player who has their feet on the ground
. - In mountaineering, instead of
alpenglow
, usea reddish glow near the horizon
.
See the Find football hurdles section below for a practical example of how rephrasing domain-specific terms can enhance the precision of your search results.
Practical examples of query engineering
The examples in this section are from the Playground. However, the principles demonstrated are similar when invoking the API programmatically.
Find the perfect adrenaline rush activity for your next vacation
This example demonstrates how using descriptive phrases significantly enhances the accuracy of your search results.
Objective: Find a video showing the perfect adrenaline rush activity for your next vacation.
Initial query: Use adrenaline rush
as your initial query. As shown in the screenshot below, the search results might be too broad:
Step-by-step refinement:
-
Refine your query by specifying the location -
adrenaline rush activities on the beach
. -
In the screenshot above, the first search result is from a video showing people skydiving from a helicopter, and you want to find more videos similar to this. To do so, you can refine your query even further by specifying both the activity and the location -
People skydiving from a helicopter and landing on the beach
. Now, the search results are narrowed down:
Find football hurdles
This example demonstrates how using everyday language instead of jargon can improve the precision of the search results.
Objective: Find videos showing football hurdles.
Initial query: Use football hurdles
as your initial query. As shown in the screenshot below, the search results are too broad, and only a few of them show the exact moments when players hurdle over their opponents:
Step-by-step refinement:
- Use everyday language to refine your query -
A football player jumping over another player who has their feet on the ground
. Now, the search results are narrowed down:
Next steps
- Depending on your use case, see the Use single queries or Use combined queries sections for step-by-step guides on using the Twelve Labs Video Understanding Platform to perform search requests.
Updated 8 months ago