Marengo
Marengo is an embedding model for comprehensive video understanding. It analyzes visual, audio, and text information to provide a holistic understanding similar to human comprehension.
Available versions
Create embeddings from video, audio, images, text, and documents.
Search your content with the platform, and create embeddings from video, audio, images, and text.
Examples
The examples in this section are from the Playground. However, the principles demonstrated are similar when invoking the API programmatically.
Steve Jobs introducing the iPhone
In the example screenshot below, the query was “How did Steve Jobs introduce the iPhone?”. The Marengo video understanding model used information found in the visual and conversation audio to perform the following tasks:
- Visual recognition of a famous person (Steve Jobs)
- Joint speech and visual recognition to semantically search for the moment when Steve Jobs introduced the iPhone. Note that semantic search finds information based on the intended meaning of the query rather than the literal words you used, meaning that the platform identified the matching video fragments even if Steve Jobs didn’t explicitly say the words in the query.
Polar bear holding a Coca-Cola bottle
In the example screenshot below, the query was “Polar bear holding a Coca-Cola bottle.” The Marengo video understanding model used information found in the visual and logo modalities to perform the following tasks:
- Recognition of a cartoon character (polar bear)
- Identification of an object (bottle)
- Detection of a specific brand logo (Coca-Cola)
- Identification of an action (polar bear holding a bottle)
To see this example in the Playground, ensure you’re logged in, and then open this URL in your browser.
Using different languages
This section provides examples of using different languages to perform search requests.
Spanish
In the example screenshot below, the query was “¿Cómo presentó Steve Jobs el iPhone?” (“How did Steve Jobs introduce the iPhone?”). The Marengo video understanding model used information from the visual and audio modalities.
Chinese
In the example screenshot below, the query was “猫做有趣的事情” (“Cats doing funny things.”). The Marengo video understanding model used information from the visual modality.
To see this example in the Playground, ensure you’re logged in, and then open this URL in your browser.
Support
For support or feedback regarding Marengo, contact support@twelvelabs.io.