Grouping and ungrouping refer to the process of organizing the results of a search query in a specific way. The platform allows for grouping based on the unique identifiers of the videos. For example, this is useful when building a user interface, because it allows your users to better understand and navigate the search results. On the other hand, ungrouping presents your search results in a flat list. This is useful if you want to view all of the search results in a simple manner. Note that this feature can only be used with simple queries.

To group or ungroup items in a response, use the group_by parameter, specifying one of the following values:

video: The platform will group the matching video clips in the response by video.
clip: The matching video clips in the response will not be grouped.

📘
Note
The group_by parameter is optional and its default value is clip. If omitted, the platform will use the default value. For clarity, the examples in this section always specify the group_by parameter.

For a description of each field in the request and response, see the API Reference> Make a search request page.

You can interact with the platform using one of the available SDKs or an HTTP client like requests or axios. Follow the steps in one of the sections below, depending on your use case.

Prerequisites

You're familiar with the concepts that are described on the Platform overview page.
You've already created an index, and the Marengo video understanding engine is enabled for this index.
You've already uploaded a video, and the platform has finished indexing it.

Use an SDK

Refer to this section if you use one of the available SDKs.

Grouping items in a response

The following example code groups the matching video clips in the response by video:

from twelvelabs import TwelveLabs

client = TwelveLabs(api_key="<YOUR_API_KEY>")

search_results = client.search.query(
  index_id="<YOUR_INDEX_ID>, 
  query="<YOUR_QUERY>, 
  options=["visual"],
  group_by="video"
)

# Utility function to print a specific page
def print_page(page):
    for video in page:
        print(f"Video id: {video.id}")
        for clip in video.clips:
            print(clip)
            print(
                f"\tscore={clip.score} start={clip.start} end={clip.end} confidence={clip.confidence} metadata={clip.metadata}"
            )

print_page(search_results.data)

while True:
    try:
        print_page(next(search_results))
    except StopIteration:
        break

import { TwelveLabs, GroupByVideoSearchData } from 'twelvelabs-js';

const client = new TwelveLabs({ apiKey: '<YOUR_API_KEY>'});

let searchResults = await client.search.query({
  indexId: '<YOUR_INDEX_ID>'
  query: '<YOUR_QUERY>',
  options: ['visual'],
  groupBy: 'video',
});
printPage(searchResults);
while (true) {
  const page = await searchResults.next();
  if (page === null) break;
  else printPage(page);
}
// Utility function to print a specific page
function printPage(searchData) {
  (searchData.data as GroupByVideoSearchData[]).forEach((video) => {
    console.log(`videoId=${video.id}`);
    video.clips?.forEach((clip) => {
      console.log(
        `     score=${clip.score} start=${clip.start} end=${clip.end} confidence=${clip.confidence}`,
      );
    });
  });
}

The output should look similar to the following one:

Video id: 65d60bcf48db9fa780cb415e
score=83.73 start=273.96875 end=289.0625 video_id='65d60bcf48db9fa780cb415e' metadata=[{'type': 'visual'}] confidence='high'
	score=83.73 start=273.96875 end=289.0625 confidence=high metadata=[{'type': 'visual'}]
score=83.55 start=397.921875 end=439.84375 video_id='65d60bcf48db9fa780cb415e' metadata=[{'type': 'visual'}] confidence='high'
	score=83.55 start=397.921875 end=439.84375 confidence=high metadata=[{'type': 'visual'}]
score=83.46 start=294.5625 end=311.84375 video_id='65d60bcf48db9fa780cb415e' metadata=[{'type': 'visual'}] confidence='high'
Video id: 65d5fbad48db9fa780cb415c
score=83.36 start=342.6875 end=353.140625 video_id='65d5fbad48db9fa780cb415c' metadata=[{'type': 'visual'}] confidence='high' thumbnail_url=None module_confidence=None
	score=83.36 start=342.6875 end=353.140625 confidence=high metadata=[{'type': 'visual'}]
score=83.32 start=164.671875 end=200.71875 video_id='65d5fbad48db9fa780cb415c' metadata=[{'type': 'visual'}] confidence='high' thumbnail_url=None module_confidence=None
	score=83.32 start=164.671875 end=200.71875 confidence=high metadata=[{'type': 'visual'}]
score=83.31 start=329.96875 end=337.75 video_id='65d5fbad48db9fa780cb415c' metadata=[{'type': 'visual'}] confidence='high' thumbnail_url=None module_confidence=None
	score=83.31 start=329.96875 end=337.75 confidence=high metadata=[{'type': 'visual'}]

In this example, note that the data array contains a list of objects. Each object corresponds to a video that matches your query and is composed of the following key-value pairs:

clips: An array that groups the information about all the matching video clips in that video.
id: The unique identifier of the video that matched your query.

Ungrouping items in a response

The following example performs a search request, and the matching video clips in the response are not grouped:

from twelvelabs import TwelveLabs

client = TwelveLabs(api_key="<YOUR_API_KEY>")

search_results = client.search.query(
  index_id="<YOUR_INDEX_ID>",
  query=  "<YOUR_QUERY>"
  options=["visual"],
  group_by='clip'
)
# Utility function to print a specific page
def print_page(page):
  for clip in page:
    print(
        f" video_id={clip.video_id} score={clip.score} start={clip.start} end={clip.end} confidence={clip.confidence} metadata={clip.metadata}"
    )

print_page(search_results.data)

while True:
    try:
        print_page(next(search_results))
    except StopIteration:
        break

import { TwelveLabs, SearchData } from 'twelvelabs-js';

const client = new TwelveLabs({ apiKey: '<YOUR_API_KEY>'});

let searchResults = await client.search.query({
  indexId: '<YOUR_INDEX_ID>'
  query: '<YOUR_QUERY>',
  options: ['visual'],
  group_by='clip',
});
printPage(searchResults.data);
while (true) {
  const page = await searchResults.next();
  if (page === null) break;
  else printPage(page);
}
// Utility function to print a specific page
function printPage(searchData) {
  (searchData as SearchData[]).forEach((clip) => {
    console.log(
      `video_id= ${clip.videoId} score=${clip.score} start=${clip.start} end=${clip.end} confidence=${clip.confidence} metadata=${JSON.stringify(clip.metadata)}`,
    );
  });
}

The output should look similar to the following one:

video_id=65d6131c48db9fa780cb415f score=52.04 start=4.5625 end=14.8125 confidence=low metadata=[{'type': 'visual'}]
 video_id=65d6131c48db9fa780cb415f score=50.94 start=20.9375 end=25.09375 confidence=low metadata=[{'type': 'visual'}]
 video_id=65d60bcf48db9fa780cb415e score=50.94 start=20.9375 end=25.09375 confidence=low metadata=[{'type': 'visual'}]

Use an HTTP client

Refer to this section if you use clients such as requests or axios.

Grouping items in a response

The following example code searches for car accidents, and the matching video clips in the response are grouped by video:

SEARCH_URL = f”{API_URL}/search”

data = {
  “query”: “car accidents”,
  “index_id”: INDEX_ID,
  “search_options”: [“visual”],
  “group_by”: “video”
}

response = requests.post(SEARCH_URL, headers=headers, json=data)
print (f’Status code: {response.status_code}’)
pprint(response.json())

const SEARCH_URL = `${API_URL}/search`

const data = {
  'query': 'bear chasing a man',
  'index_id': INDEX_ID,
  'search_options': ['visual'],
  'group_by': 'video'
}

const resp = await axios.post(
  SEARCH_URL,
  data,
  {
    'headers': {
      'x-api-key': API_KEY
    }
  }
)
const { data: response } = resp;
console.log(`Status code: ${resp.status}`)
console.log(JSON.stringify(response,null,4))

The following output has been truncated for brevity:

Status code: 200
{
  "search_pool": {
    "total_count": 13,
    "total_duration": 8731,
    "index_id": "639961c9e219c90227c371a2"
  },
  "data": [
    {
      "clips": [
        {
          "score": 86.69,
          "start": 137.1875,
          "end": 149.1875,
          "metadata": [
            {
              "type": "visual"
            }
          ],
          "video_id": "63996260ce36463e0199c8c5",
          "confidence": "high",
          "thumbnail_url": "https://project-one-thumbnail.s3.us-west-2.amazonaws.com/63996260ce36463e0199c8c5/138.jpeg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAYRWJPOVHXE5SJ77T%2F20221229%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20221229T121123Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&x-id=GetObject&X-Amz-Signature=77fbecb5fea214120d86e89c144660b20414f0957970621bf299e3d0e261ebec"
        },
        {
          "score": 86.17,
          "start": 173.1875,
          "end": 193.5625,
          "metadata": [
            {
              "type": "visual"
            }
          ],
          "video_id": "63996260ce36463e0199c8c5",
          "confidence": "high",
          "thumbnail_url": "https://project-one-thumbnail.s3.us-west-2.amazonaws.com/63996260ce36463e0199c8c5/174.jpeg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAYRWJPOVHXE5SJ77T%2F20221229%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20221229T121123Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&x-id=GetObject&X-Amz-Signature=4c2049b56aaa3e1b0aec958fbdce359cd89390830b370cb54f926e77b51c04a2"
        }
      ],
      "id": "63996260ce36463e0199c8c5"
    },
    {
      "clips": [
        {
          "score": 86.14,
          "start": 624.46875,
          "end": 630.46875,
          "metadata": [
            {
              "type": "visual"
            }
          ],
          "video_id": "63996246ce36463e0199c8c4",
          "confidence": "high",
          "thumbnail_url": "https://project-one-thumbnail.s3.us-west-2.amazonaws.com/63996246ce36463e0199c8c4/625.jpeg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAYRWJPOVHXE5SJ77T%2F20221229%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20221229T121123Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&x-id=GetObject&X-Amz-Signature=8d7d1014ed115ca6b8e76a488662e00afb5d5aad581f8701d0f82785c14c0e56"
        },
        {
          "score": 85.37,
          "start": 432.46875,
          "end": 442.46875,
          "metadata": [
            {
              "type": "visual"
            }
          ],
          "video_id": "63996246ce36463e0199c8c4",
          "confidence": "high",
          "thumbnail_url": "https://project-one-thumbnail.s3.us-west-2.amazonaws.com/63996246ce36463e0199c8c4/433.jpeg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAYRWJPOVHXE5SJ77T%2F20221229%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20221229T121123Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&x-id=GetObject&X-Amz-Signature=755fb037c4f6be94583ff261282e35d11b6d0db6af8a6de7bcfc00a63a013bb4"
        }
      ],
      "id": "63996246ce36463e0199c8c4"
    }
  ],
  "page_info": {
    "limit_per_page": 10,
    "total_results": 4,
    "total_inner_matches": 32,
    "page_expired_at": "2022-12-29T12:11:23Z"
  }
}

In the example output above, note that the data array contains a list of objects. Each object corresponds to a video that matches your query and is composed of the following key-value pairs:

clips: An array that groups the information about all the matching video clips in that video.
id: The unique identifier of the video that matched your query.

Ungrouping items in a response

The following example code searches for car accidents, and the matching video clips in the response are not grouped:

SEARCH_URL = f”{API_URL}/search”

data = {
  “query”: “car accidents”,
  “index_id”: INDEX_ID,
  “search_options”: [“visual”],
  “group_by”: “clip”
}

response = requests.post(SEARCH_URL, headers=headers, json=data)
print (f’Status code: {response.status_code}’)
pprint(response.json())

const SEARCH_URL = `${API_URL}/search`

const data = {
  'query': 'bear chasing a man',
  'index_id': INDEX_ID,
  'search_options': ['visual'],
  'group_by': 'clip'
}

const resp = await axios.post(
  SEARCH_URL,
  data,
  {
    'headers': {
      'x-api-key': API_KEY
    }
  }
)
const { data: response } = resp;
console.log(`Status code: ${resp.status}`)
console.log(JSON.stringify(response,null,4))

The following output was truncated for brevity:

Status code: 200
{
  "search_pool": {
    "total_count": 13,
    "total_duration": 8731,
    "index_id": "639961c9e219c90227c371a2"
  },
  "data": [
    {
      "score": 86.69,
      "start": 137.1875,
      "end": 149.1875,
      "metadata": [
        {
          "type": "visual"
        }
      ],
      "video_id": "63996260ce36463e0199c8c5",
      "confidence": "high",
      "thumbnail_url": "https://project-one-thumbnail.s3.us-west-2.amazonaws.com/63996260ce36463e0199c8c5/138.jpeg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAYRWJPOVHXE5SJ77T%2F20221229%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20221229T075204Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&x-id=GetObject&X-Amz-Signature=1464a7f7516b5c42669135f7822e1f762f25036c0bb9b1382b40cc7de09188eb"
    },

    {
      "score": 85.4,
      "start": 483.6,
      "end": 498.20625,
      "metadata": [
        {
          "type": "visual"
        }
      ],
      "video_id": "63996232ce36463e0199c8c3",
      "confidence": "high",
      "thumbnail_url": "https://project-one-thumbnail.s3.us-west-2.amazonaws.com/63996232ce36463e0199c8c3/484.jpeg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAYRWJPOVHXE5SJ77T%2F20221229%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20221229T075204Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&x-id=GetObject&X-Amz-Signature=0b7564985d556b65da677719cfedb0fcbf2d8c8fada6af2bc289f6e526d3150d"
    },
    {
      "score": 85.37,
      "start": 432.46875,
      "end": 442.46875,
      "metadata": [
        {
          "type": "visual"
        }
      ],
      "video_id": "63996246ce36463e0199c8c4",
      "confidence": "high",
      "thumbnail_url": "https://project-one-thumbnail.s3.us-west-2.amazonaws.com/63996246ce36463e0199c8c4/433.jpeg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAYRWJPOVHXE5SJ77T%2F20221229%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20221229T075204Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&x-id=GetObject&X-Amz-Signature=d7eae8609413622fe7d2afdd85d40637e9ab0b903a8f7b20d8f85252c5ef5a1d"
    }
  ],
  "page_info": {
    "limit_per_page": 10,
    "total_results": 222,
    "page_expired_at": "2022-12-29T07:52:04Z",
    "next_page_token": "52a2c8dc-8ff0-40c6-9a4e-8c2ce56212b3-1"
  }
}

In the example output above, note that the data array contains a list of objects, and each object has a property named video_id, which represents the unique identifier of the video that matched your query.

📘Note

Prerequisites

Use an SDK

Grouping items in a response

Ungrouping items in a response

Use an HTTP client

Grouping items in a response

Ungrouping items in a response

📘
Note