This section shows how to classify the content of all the videos within an index by calling the POST method of the /classify/bulk endpoint.

Prerequisites

The examples in this section assume the following:

You’re familiar with the concepts that are described on the Platform overview page.
You've already created an index, and the Marengo video understanding engine is enabled for this index.
You've uploaded at least one video, and the platform has finished indexing it. For details, see the Upload videos page.

Examples

To specify the index containing the videos you want to classify, use the index_id field. For a description of each field in the request and response, see the API Reference > Classify all the videos in an index page.

Classifying videos based on visual cues

The following example code uses the options parameter to specify that the platform should classify your videos based on visual cues:

CLASSIFY_BULK_URL = f"{API_URL}/classify/bulk"

data =  {
  "options": ["visual"],
  "index_id": INDEX_ID,
  "classes": [
      {
      "name": "DanceTok",
      "prompts": [
        "Dance tutorial",
        "Dance group",
        "Dance competition"
      ]
    }
  ]
}

response = requests.post(CLASSIFY_BULK_URL, headers=headers, json=data)
print (f'Status code: {response.status_code}')
pprint(response.json())

const CLASSIFY_BULK_URL = `${API_URL}/classify/bulk`

const data =  {
  'options': ['visual'],
  'index_id': INDEX_ID,
  'classes': [
      {
      'name': 'DanceTok',
      'prompts': [
        'Dance tutorial',
        'Dance group',
        'Dance competition'
      ]
    }
  ]
}

const resp = await axios.post(
  CLASSIFY_BULK_URL,
  data,
  {
    'headers': {
      'x-api-key': API_KEY
    }
  }
)
const { data: response } = resp;
console.log(`Status code: ${resp.status}`)
console.log(JSON.stringify(response,null,4))

The following output has been truncated for brevity:

Status code: 200
{
  "data": [
    {
      "video_id": "6435651a9ab33c7198615a80",
      "classes": [
        {
          "name": "DanceTok",
          "score": 96.77333333333335,
          "duration_ratio": 1
        }
      ]
    },
    {
      "video_id": "643564429ab33c7198615a7b",
      "classes": [
        {
          "name": "DanceTok",
          "score": 93.71555555555557,
          "duration_ratio": 1.0000086662264358
        }
      ]
    },
    {
      "video_id": "6435647b9ab33c7198615a7e",
      "classes": [
        {
          "name": "DanceTok",
          "score": 87.03056529150149,
          "duration_ratio": 0.715890394472522
        }
      ]
    }
  ],
  "page_info": {
    "limit_per_page": 3,
    "total_results": 6,
    "page_expired_at": "2023-04-12T11:58:16Z",
    "next_page_token": "",
    "prev_page_token": ""
  }
}

In the example output above, note that the data array is composed of three objects. Each object contains the following:

A field named video_idrepresenting the unique identifier of the video that has been classified.
An array named classes containing information that helps you determine the likelihood that the class you've specified in the request appears in that video.

Retrieving detailed information about each matching video clip

The following example code sets the include_clips parameter to true to specify that the platform should retrieve detailed information about each matching video clip:

CLASSIFY_BULK_URL = f"{API_URL}/classify/bulk"

data =  {
  "options": ["visual"],
  "index_id": INDEX_ID,
  "include_clips": True,
  "classes": [
      {
      "name": "DanceTok",
      "prompts": [
        "Dance tutorial",
        "Dance group",
        "Dance competition"
      ]
    }
  ]
}

response = requests.post(CLASSIFY_BULK_URL, headers=headers, json=data)
print (f'Status code: {response.status_code}')
pprint(response.json())

const CLASSIFY_BULK_URL = `${API_URL}/classify/bulk`

const data =  {
  'options': ['visual'],
  'index_id': INDEX_ID,
  'include_clips': true,
  'classes': [
      {
      'name': 'DanceTok',
      'prompts': [
        'Dance tutorial',
        'Dance group',
        'Dance competition'
      ]
    }
  ]
}

const resp = await axios.post(
  CLASSIFY_BULK_URL,
  data,
  {
    'headers': {
      'x-api-key': API_KEY
    }
  }
)
const { data: response } = resp;
console.log(`Status code: ${resp.status}`)
console.log(JSON.stringify(response,null,4))

The following example output has been truncated for brevity:

Status code: 200
{
  "data": [
    {
      "video_id": "6435651a9ab33c7198615a80",
      "classes": [
        {
          "name": "DanceTok",
          "score": 96.77333333333335,
          "duration_ratio": 1,
          "clips": [
            {
              "start": 711.8125,
              "end": 774.15625,
              "score": 86.37,
              "option": "visual",
              "prompt": "Dance competition",
              "thumbnail_url": "https://project-one-thumbnail.s3.us-west-2.amazonaws.com/6435651a9ab33c7198615a80/712.jpeg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAYRWJPOVHXE5SJ77T%2F20230412%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20230412T115721Z&X-Amz-Expires=86400&X-Amz-SignedHeaders=host&x-id=GetObject&X-Amz-Signature=94defeefcebaa10e7b60390b8a564a582ffa273535b1f74b97ae8324a3775afa"
            },
            {
              "start": 207.425,
              "end": 214.46875,
              "score": 46.77,
              "option": "visual",
              "prompt": "Dance tutorial",
              "thumbnail_url": "https://project-one-thumbnail.s3.us-west-2.amazonaws.com/6435651a9ab33c7198615a80/208.jpeg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAYRWJPOVHXE5SJ77T%2F20230412%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20230412T115721Z&X-Amz-Expires=86400&X-Amz-SignedHeaders=host&x-id=GetObject&X-Amz-Signature=aeaa1f38ee2fff2502ccc1719e96417eadc6a2ac915318b7f0818c544155663b"
            }
          ]
        }
      ]
    },
    {
      "video_id": "643564b39ab33c7198615a7f",
      "classes": [
        {
          "name": "DanceTok",
          "score": 95.46863419938227,
          "duration_ratio": 0.9962264438246706,
          "clips": [
            {
              "start": 473.09375,
              "end": 595.09375,
              "score": 84.92,
              "option": "visual",
              "prompt": "Dance tutorial",
              "thumbnail_url": "https://project-one-thumbnail.s3.us-west-2.amazonaws.com/643564b39ab33c7198615a7f/474.jpeg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAYRWJPOVHXE5SJ77T%2F20230412%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20230412T115721Z&X-Amz-Expires=86400&X-Amz-SignedHeaders=host&x-id=GetObject&X-Amz-Signature=07fd122f6ea7305012ece1c1c46bcfbf0e7997f8e74418ce6a8e7ed030381090"
            },
            {
              "start": 369.09375,
              "end": 371.09375,
              "score": 56.52,
              "option": "visual",
              "prompt": "Dance group",
              "thumbnail_url": "https://project-one-thumbnail.s3.us-west-2.amazonaws.com/643564b39ab33c7198615a7f/370.jpeg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAYRWJPOVHXE5SJ77T%2F20230412%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20230412T115721Z&X-Amz-Expires=86400&X-Amz-SignedHeaders=host&x-id=GetObject&X-Amz-Signature=0cb1d1ed7a99f4b3bb78d331a8865c9d7d7d9b1a441013517956c7fc61b8c460"
            }
          ]
        }
      ]
    }
  ],
  "page_info": {
    "limit_per_page": 10,
    "total_results": 6,
    "page_expired_at": "2023-04-12T12:02:22Z",
    "next_page_token": "",
    "prev_page_token": ""
  }
}

In the example output above, note that, for each video, the platform returns an array named clips containing detailed information about a single matching video clip.

Classifying videos based on multiple classes

The following example code classifies the content of your videos based on two classes - DanceTok and CookTok:

CLASSIFY_URL = f"{API_URL}/classify/bulk"

data =  {
  "options": ["visual"],
  "index_id": INDEX_ID,
  "classes": [
      {
      "name": "DanceTok",
      "prompts": [
        "Dance tutorial",
        "Dance group",
        "Dance competition"
      ]
    },
    {
      "name": "CookTok",
      "prompts": [
        "Cooking tutorial",
        "Cooking ustensils review"
      ]
    }
  ]
}
response = requests.post(CLASSIFY_URL, headers=headers, json=data)
print (f'Status code: {response.status_code}')
pprint(response.json())

const CLASSIFY_BULK_URL = `${API_URL}/classify/bulk`
const  data =  {
  'conversation_option': 'semantic',
  'options': ['visual'],
  'index_id': INDEX_ID,
  'classes': [
      {
      'name': 'DanceTok',
      'prompts': [
        'Dance tutorial',
        'Dance group',
        'Dance competition'
      ]
    },
    {
      'name': 'CookTok',
      'prompts': [
        'Cooking tutorial',
        'Cooking ustensils review'
      ]
    }
  ]
}
const resp = await axios.post(
  CLASSIFY_BULK_URL,
  data,
  {
    'headers': {
      'x-api-key': API_KEY
    }
  }
)
const { data: response } = resp;
console.log(`Status code: ${resp.status}`)
console.log(JSON.stringify(response,null,4))

The output should look similar to the following one:

Status code: 200
{
  "data": [
    {
      "video_id": "6435651a9ab33c7198615a80",
      "classes": [
        {
          "name": "DanceTok",
          "score": 96.77333333333335,
          "duration_ratio": 1
        }
      ]
    },
    {
      "video_id": "643564b39ab33c7198615a7f",
      "classes": [
        {
          "name": "DanceTok",
          "score": 95.46863419938227,
          "duration_ratio": 0.9962264438246706
        }
      ]
    },
    {
      "video_id": "643564539ab33c7198615a7c",
      "classes": [
        {
          "name": "DanceTok",
          "score": 95.0584126984127,
          "duration_ratio": 0.9483934493906572
        }
      ]
    }
  ],
  "page_info": {
    "limit_per_page": 3,
    "total_results": 7,
    "page_expired_at": "2023-04-12T12:21:18Z",
    "next_page_token": "142ae635-d410-4465-b2c7-d677b6abcbed-1",
    "prev_page_token": ""
  }
}

In the example output above, note that the data array contains three objects, each corresponding to a different video. For each video, the response contains information that helps you determine the likelihood that each of the classes you've specified in the request appears in that video.

Retrieving a detailed score for each class

The following example code sets the show_detailed_score parameter to true to specify that the platform should retrieve the maximum score, average score, duration weighted score, and normalized score for each class:

CLASSIFY_URL = f"{API_URL}/classify/bulk"

data =  {
  "options": ["visual"],
  "index_id": INDEX_ID,
  "show_detailed_score": True,
  "classes": [
      {
      "name": "DanceTok",
      "prompts": [
        "Dance tutorial",
        "Dance group",
        "Dance competition"
      ]
    }
  ]
}

response = requests.post(CLASSIFY_URL, headers=headers, json=data)
print (f'Status code: {response.status_code}')
pprint(response.json())

const CLASSIFY_URL = `${API_URL}/classify/bulk`
const data =  {
  'options': ['visual'],
  'index_id': INDEX_ID,
  'show_detailed_score': true,
  'classes': [
      {
      'name': 'DanceTok',
      'prompts': [
        'Dance tutorial',
        'Dance group',
        'Dance competition'
      ]
    }
  ]
}
const resp = await axios.post(
  CLASSIFY_URL,
  data,
  {
    'headers': {
      'x-api-key': API_KEY
    }
  }
)
const { data: response } = resp;
console.log(`Status code: ${resp.status}`)
console.log(JSON.stringify(response,null,4))

The output should look similar to the following one:

Status code: 200
{
  "data": [
    {
      "video_id": "6435651a9ab33c7198615a80",
      "classes": [
        {
          "name": "DanceTok",
          "score": 96.77333333333335,
          "duration_ratio": 1,
          "detailed_scores": {
            "max_score": 86.37,
            "avg_score": 72.46005376344091,
            "duration_weighted_score": 96.77333333333335,
            "normalized_score": 100
          }
        }
      ]
    },
    {
      "video_id": "643564b39ab33c7198615a7f",
      "classes": [
        {
          "name": "DanceTok",
          "score": 95.46863419938227,
          "duration_ratio": 0.9962264438246706,
          "detailed_scores": {
            "max_score": 84.92,
            "avg_score": 68.3653164556962,
            "duration_weighted_score": 95.46863419938227,
            "normalized_score": 100
          }
        }
      ]
    },
    {
      "video_id": "643564779ab33c7198615a7d",
      "classes": [
        {
          "name": "DanceTok",
          "score": 95.0584126984127,
          "duration_ratio": 0.9483934493906572,
          "detailed_scores": {
            "max_score": 84.83,
            "avg_score": 70.94315068493151,
            "duration_weighted_score": 95.0584126984127,
            "normalized_score": 100
          }
        }
      ]
    }
  ],
  "page_info": {
    "limit_per_page": 3,
    "total_results": 7,
    "page_expired_at": "2023-04-12T12:26:15Z",
    "next_page_token": "5a8cae11-83eb-47f8-bf43-5674774c17e1-1",
    "prev_page_token": ""
  }
}

Filtering

This endpoint supports filtering. For details, see the Filtering > Content Classification page.

Pagination

When using the /classify/bulk endpoint, pagination works in a similar fashion to the /search endpoint. For details, see the Pagination > Search results page.