This section shows how to classify a set of videos by calling the POST method of the /classify endpoint.

Prerequisites

The examples in this guide assume the following:

You’re familiar with the concepts that are described on the Platform overview page.
You've already created an index, and the Marengo video understanding engine is enabled for this index.
You've uploaded one or more videos, and the platform has finished indexing them. The unique identifiers of your videos are stored in a set of variables named VIDEO_ID_1 to VIDEO_ID_X. For details, see the Upload videos page.

Examples

To specify the set of videos you want to classify, use the video_ids field. It is an array of strings, and each element represents the unique identifier of a video.For a description of each field in the request and response, see the API Reference > Classify a set of videos page.

Classifying a set of videos based on visual cues

The following example code classifies two videos based on visual cues by setting the value of the video_ids field to [VIDEO_ID_1, VIDEO_ID_2] and the value of the options field to ["visual"]:

CLASSIFY_URL = f"{API_URL}/classify"
headers = {
    "x-api-key": API_KEY
}
data =  {
  "options": ["visual"],
  "video_ids": [VIDEO_ID_1, VIDEO_ID_2],
  "classes": [
      {
      "name": "DanceTok",
      "prompts": [
        "Dance tutorial",
        "Dance group",
        "Dance competition"
      ]
    }
  ]
}
response = requests.post(CLASSIFY_URL, headers=headers, json=data)
print (f'Status code: {response.status_code}')
pprint(response.json())

const CLASSIFY_URL = `${API_URL}/classify`
const data =  {
  'options': ['visual'],
  'video_ids': [VIDEO_ID1, VIDEO_ID_2],
  'classes': [
      {
      'name': 'DanceTok',
      'prompts': [
        'Dance tutorial',
        'Dance group',
        'Dance competition'
      ]
    }
  ]
}
const resp = await axios.post(
  CLASSIFY_URL,
  data,
  {
    'headers': {
      'x-api-key': API_KEY
    }
  }
)
const { data: response } = resp;
console.log(`Status code: ${resp.status}`)
console.log(JSON.stringify(response,null,4))

The output should look similar to the following one:

Status code: 200
{
  "data": [
    {
      "video_id": "6444c376456b2dab381e702d",
      "classes": [
        {
          "name": "DanceTok",
          "score": 96.77,
          "duration_ratio": 1
        }
      ]
    },
    {
      "video_id": "64428d88456b2dab381e7022",
      "classes": [
        {
          "name": "DanceTok",
          "score": 47.72,
          "duration_ratio": 0.6
        }
      ]
    }
  ],
  "page_info": {
    "limit_per_page": 2,
    "total_results": 2,
    "page_expired_at": "2023-05-22T05:19:09Z",
    "next_page_token": "",
    "prev_page_token": ""
  }
}

In the example output above, note that the data array has two elements, and each element contains information about a specific video.

Retrieving information about each matching video clip

The following example code sets the include_clips parameter to true to specify that the platform should retrieve detailed information about each matching video clip:

CLASSIFY_URL = f"{API_URL}/classify"

data =  {
  "options": ["visual"],
  "video_ids": [VIDEO_ID_1, VIDEO_ID_2],
  "include_clips": True,
  "classes": [
      {
      "name": "DanceTok",
      "prompts": [
        "Dance tutorial",
        "Dance group",
        "Dance competition"
      ]
    }
  ]
}

response = requests.post(CLASSIFY_URL, headers=headers, json=data)
print (f'Status code: {response.status_code}')
pprint(response.json())

const CLASSIFY_URL = `${API_URL}/classify`
const data =  {
  'options': ['visual'],
  'video_ids': [VIDEO_ID_1, VIDEO_ID_2],
  'include_clips': true,
  'classes': [
      {
      'name': 'DanceTok',
      'prompts': [
        'Dance tutorial',
        'Dance group',
        'Dance competition'
      ]
    }
  ]
}
const resp = await axios.post(
  CLASSIFY_URL,
  data,
  {
    'headers': {
      'x-api-key': API_KEY
    }
  }
)
const { data: response } = resp;
console.log(`Status code: ${resp.status}`)
console.log(JSON.stringify(response,null,4))

The following example output has been truncated for brevity:

Status code: 200
{
  "data": [
    {
      "video_id": "64428d88456b2dab381e7022",
      "classes": [
        {
          "name": "DanceTok",
          "score": 39.47,
          "duration_ratio": 0.33,
          "clips": [
            {
              "start": 224,
              "end": 226,
              "score": 64.59,
              "option": "text_in_video",
              "prompt": "Dance group",
              "thumbnail_url": "https://project-one-thumbnail.s3.us-west-2.amazonaws.com/64428d88456b2dab381e7022/225.jpeg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAYRWJPOVHXE5SJ77T%2F20230522%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20230522T055246Z&X-Amz-Expires=604799&X-Amz-SignedHeaders=host&x-id=GetObject&X-Amz-Signature=3a7eacdd9cb08e7d6a1910f17849dedd3c7f022bc70e0a1c4eded51fe1e06473"
            },
            {
              "start": 185,
              "end": 189.125,
              "score": 27.81,
              "option": "visual",
              "prompt": "Dance tutorial",
              "thumbnail_url": "https://project-one-thumbnail.s3.us-west-2.amazonaws.com/64428d88456b2dab381e7022/186.jpeg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAYRWJPOVHXE5SJ77T%2F20230522%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20230522T055246Z&X-Amz-Expires=604799&X-Amz-SignedHeaders=host&x-id=GetObject&X-Amz-Signature=f464308b1fef056c8a3bb575d737cc9dbb1fd759a1b2cf3328262f92120ab09b"
            }
          ]
        }
      ]
    },
    {
      "video_id": "6444c376456b2dab381e702d",
      "classes": [
        {
          "name": "DanceTok",
          "score": 35.99,
          "duration_ratio": 0.16,
          "clips": [
            {
              "start": 153.9375,
              "end": 169.63541666666669,
              "score": 39.04,
              "option": "visual",
              "prompt": "Dance group",
              "thumbnail_url": "https://project-one-thumbnail.s3.us-west-2.amazonaws.com/6444c376456b2dab381e702d/154.jpeg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAYRWJPOVHXE5SJ77T%2F20230522%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20230522T055246Z&X-Amz-Expires=604799&X-Amz-SignedHeaders=host&x-id=GetObject&X-Amz-Signature=9f955860782309a699f68b3f390ff09edaec605dffa3fa34a250c3c88fbc6750"
            },
            {
              "start": 45.625,
              "end": 50.8125,
              "score": 28.88,
              "option": "visual",
              "prompt": "Dance group",
              "thumbnail_url": "https://project-one-thumbnail.s3.us-west-2.amazonaws.com/6444c376456b2dab381e702d/46.jpeg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAYRWJPOVHXE5SJ77T%2F20230522%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20230522T055246Z&X-Amz-Expires=604799&X-Amz-SignedHeaders=host&x-id=GetObject&X-Amz-Signature=915b117d4aa1d3c9f4d8802b3594c2cc9a2e39c5e3d4eb0a52cf5bc6c6a97987"
            }
          ]
        }
      ]
    }
  ],
  "page_info": {
    "limit_per_page": 2,
    "total_results": 2,
    "page_expired_at": "2023-05-22T05:57:46Z",
    "next_page_token": "",
    "prev_page_token": ""
  }
}

In the example output above, note that, for each video, the platform returns an array named clips. Each element of these arrays contains detailed information about a single video clip.

Classifying videos based on multiple classes

The following example code classifies the content of a video based on two classes - DanceTok and CookTok:

CLASSIFY_URL = f"{API_URL}/classify"

data =  {
  "options": ["visual"],
  "video_ids": [VIDEO_ID_1, VIDEO_ID_],
  "classes": [
      {
      "name": "DanceTok",
      "prompts": [
        "Dance tutorial",
        "Dance group",
        "Dance competition"
      ]
    },
    {
      "name": "CookTok",
      "prompts": [
        "Cooking tutorial",
        "Cooking ustensils review"
      ]
    }
  ]
}
response = requests.post(CLASSIFY_URL, headers=headers, json=data)
print (f'Status code: {response.status_code}')
pprint(response.json())

const CLASSIFY_URL = `${API_URL}/classify`
const  data =  {
  'options': ['visual'],
  'video_ids': [VIDEO_ID_1, VIDEO_ID_2],
  'classes': [
      {
      'name': 'DanceTok',
      'prompts': [
        'Dance tutorial',
        'Dance group',
        'Dance competition'
      ]
    },
    {
      'name': 'CookTok',
      'prompts': [
        'Cooking tutorial',
        'Cooking ustensils review'
      ]
    }
  ]
}
const resp = await axios.post(
  CLASSIFY_URL,
  data,
  {
    'headers': {
      'x-api-key': API_KEY
    }
  }
)
const { data: response } = resp;
console.log(`Status code: ${resp.status}`)
console.log(JSON.stringify(response,null,4))

The output should look similar to the following one:

Status code: 200
{
  "data": [
    {
      "video_id": "64428d88456b2dab381e7022",
      "classes": [
        {
          "name": "DanceTok",
          "score": 37.83,
          "duration_ratio": 0.33
        },
        {
          "name": "CookTok",
          "score": 33.35,
          "duration_ratio": 0.18
        }
      ]
    },
    {
      "video_id": "6444c376456b2dab381e702d",
      "classes": [
        {
          "name": "DanceTok",
          "score": 35.99,
          "duration_ratio": 0.16
        },
        {
          "name": "CookTok",
          "score": 31.06,
          "duration_ratio": 0.11
        }
      ]
    }
  ],
  "page_info": {
    "limit_per_page": 2,
    "total_results": 2,
    "page_expired_at": "2023-05-22T06:10:16Z",
    "next_page_token": "",
    "prev_page_token": ""
  }
}

Retrieving a detailed score for each class

The following example code sets the show_detailed_score parameter to true to specify that the platform must retrieve the maximum, average, and normalized scores for each class.

CLASSIFY_URL = f"{API_URL}/classify"

data =  {
  "options": ["visual"],
  "video_id": VIDEO_ID,
  "show_detailed_score": True,
  "classes": [
      {
      "name": "DanceTok",
      "prompts": [
        "Dance tutorial",
        "Dance group",
        "Dance competition"
      ]
    }
  ]
}

response = requests.post(CLASSIFY_URL, headers=headers, json=data)
print (f'Status code: {response.status_code}')
pprint(response.json())

const CLASSIFY_URL = `${API_URL}/classify`
const data =  {
  'options': ['visual'],
  'video_id': VIDEO_ID,
  'show_detailed_score': true,
  'classes': [
      {
      'name': 'DanceTok',
      'prompts': [
        'Dance tutorial',
        'Dance group',
        'Dance competition'
      ]
    }
  ]
}
const resp = await axios.post(
  CLASSIFY_URL,
  data,
  {
    'headers': {
      'x-api-key': API_KEY
    }
  }
)
const { data: response } = resp;
console.log(`Status code: ${resp.status}`)
console.log(JSON.stringify(response,null,4))

The output should look similar to the following one:

Status code: 200
{
  "data": [
    {
      "video_id": "64428d88456b2dab381e7022",
      "classes": [
        {
          "name": "DanceTok",
          "score": 37.83,
          "duration_ratio": 0.33,
          "detailed_scores": {
            "max_score": 40.44,
            "avg_score": 31.56,
            "normalized_score": 53.14695139084012
          }
        },
        {
          "name": "CookTok",
          "score": 33.35,
          "duration_ratio": 0.18,
          "detailed_scores": {
            "max_score": 37.63,
            "avg_score": 30.3,
            "normalized_score": 46.85304860915988
          }
        }
      ]
    },
    {
      "video_id": "6444c376456b2dab381e702d",
      "classes": [
        {
          "name": "DanceTok",
          "score": 35.99,
          "duration_ratio": 0.16,
          "detailed_scores": {
            "max_score": 39.04,
            "avg_score": 33.45,
            "normalized_score": 53.676360924683074
          }
        },
        {
          "name": "CookTok",
          "score": 31.06,
          "duration_ratio": 0.11,
          "detailed_scores": {
            "max_score": 36.51,
            "avg_score": 32.2,
            "normalized_score": 46.32363907531692
          }
        }
      ]
    }
  ],
  "page_info": {
    "limit_per_page": 2,
    "total_results": 2,
    "page_expired_at": "2023-05-22T06:13:23Z",
    "next_page_token": "",
    "prev_page_token": ""
  }
}

Filtering

This endpoint supports filtering. For details, see the Filtering > Content Classification page.

Pagination

When using the /classify endpoint, pagination works in a similar fashion to the /search endpoint. For details, see the Pagination > Search results page.