This section shows how to classify the content of all the videos within an index by calling the POST method of the /classify/bulk endpoint. For a description of each field in the request and response, see the API Reference > Classify all the videos in an index page.

Prerequisites

The examples in this section assume the following:

A valid Twelve Labs account. For details about creating an account and retrieving your API key, see the Authentication page.
You’re familiar with the concepts that are described on the Quickstart page.
You've created at least one index. The unique identifier of your index is stored in a variable named INDEX_ID. For details, see the Creating indexes page.
You've uploaded at least one video, and the API service has finished indexing it. For details, see the Uploading videos page.

Examples

Note that, although the example code in this guide is written in Python and Node.js, the API is compatible with most programming languages, and you can also use Postman or other REST clients to send requests and view responses.

Classifying videos based on visual cues

The following example code defines a set of car shapes and uses the options parameter to specify that the API service should classify your videos based on visual cues:

CLASSIFY_BULK_URL = f"{API_URL}/classify/bulk"
headers = {
    "x-api-key": API_KEY
}
data =  {
  "conversation_option": "semantic",
  "options": ["visual"],
  "index_id" : INDEX_ID,
  "labels": [
      {
      "name": "Car shapes",
      "prompts": [
        "SUV",
        "Hatchback",
        "Crossover",
        "Sedan",
        "Coupe",
        "Minivan",
        "Station Wagon",
        "Pickup Truck",
        "Truck",
        "Convertible"
      ]
    }
  ]
}
response = requests.post(CLASSIFY_BULK_URL, headers=headers, json=data)
print (f'Status code: {response.status_code}')
pprint(response.json())

const CLASSIFY_BULK_URL = `${API_URL}/classify/bulk`
const data =  {
  "conversation_option": "semantic",
  "options": ["visual"],
  "index_id": INDEX_ID,
  "labels": [
      {
      "name": "Car shapes",
      "prompts": [
        "SUV",
        "Hatchback",
        "Crossover",
        "Sedan",
        "Coupe",
        "Minivan",
        "Station Wagon",
        "Pickup Truck",
        "Truck",
        "Convertible"
      ]
    }
  ]
}
const resp = await axios.post(
  CLASSIFY_BULK_URL,
  data,
  {
    "headers": {
      "x-api-key": API_KEY
    }
  }
)
const { data: response } = resp;
console.log(`Status code: ${resp.status}`)
console.log(JSON.stringify(response,null,4))

The following output has been truncated for brevity:

{
  "data": [
    {
      "video_id": "63996232ce36463e0199c8c3",
      "labels": [
        {
          "name": "Car Shapes",
          "max_score": 65.84,
          "duration": 116.30312500000002,
          "confidence": "low"
        }
      ]
    },
    {
      "video_id": "639963fcce36463e0199c8cc",
      "labels": [
        {
          "name": "Car Shapes",
          "max_score": 58.02,
          "duration": 526.578125,
          "confidence": "low"
        }
      ]
    },
    {
      "video_id": "63a2a2a7ce36463e0199c8e5",
      "labels": [
        {
          "name": "Car Shapes",
          "max_score": 72.45,
          "duration": 61.062500000000114,
          "confidence": "low"
        }
      ]
    }
  ],
  "page_info": {
    "total_count": 13,
    "limit": 10,
    "next_token": "7391d901-e00e-404f-b7f0-21f3c3dd84e1-1",
    "expired_at": "2022-12-21T13:20:28Z"
  }
}

In the example output above, note that the data array is composed of three objects. Each object contains the following:

A field named video_idrepresenting the unique identifier of the video that has been classified.
An array named labels containing information about each video. Note that, when classifying a video, the API service finds all video fragments that match the label you've specified in the request. For each video fragment found, the API service determines the level of confidence that the fragment matches the label. The max_score field is determined by comparing the confidence scores of each fragment and selecting the highest one.

Retrieving information about each matching video fragment

The following example code sets the include_clips parameter to true to specify that the API service should retrieve detailed information about each matching video fragment:

CLASSIFY_BULK_URL = f"{API_URL}/classify/bulk"
headers = {
    "x-api-key": API_KEY
}
data =  {
  "conversation_option": "semantic",
  "options": ["visual"],
  "index_id" : INDEX_ID,
  "include_clips": True,
  "labels": [
      {
      "name": "Car shapes",
      "prompts": [
        "SUV",
        "Hatchback",
        "Crossover",
        "Sedan",
        "Coupe",
        "Minivan",
        "Station Wagon",
        "Pickup Truck",
        "Truck",
        "Convertible"
      ]
    }
  ]
}
response = requests.post(CLASSIFY_BULK_URL, headers=headers, json=data)
print (f'Status code: {response.status_code}')
pprint(response.json())

const CLASSIFY_BULK_URL = `${API_URL}/classify/bulk`
const data =  {
  "conversation_option": "semantic",
  "options": ["visual"],
  "index_id": INDEX_ID,
  "include_clips": true,
  "labels": [
      {
      "name": "Car shapes",
      "prompts": [
        "SUV",
        "Hatchback",
        "Crossover",
        "Sedan",
        "Coupe",
        "Minivan",
        "Station Wagon",
        "Pickup Truck",
        "Truck",
        "Convertible"
      ]
    }
  ]
}
const resp = await axios.post(
  CLASSIFY_BULK_URL,
  data,
  {
    "headers": {
      "x-api-key": API_KEY
    }
  }
)
const { data: response } = resp;
console.log(`Status code: ${resp.status}`)
console.log(JSON.stringify(response,null,4))

The following example output has been truncated for brevity:

{
  "data": [
    {
      "video_id": "63a09373ce36463e0199c8de",
      "labels": [
        {
          "name": "Car Shapes",
          "max_score": 87.15,
          "duration": 288.8906249999999,
          "confidence": "high",
          "clips": [
            {
              "start": 9.25,
              "end": 12.78125,
              "score": 83.46,
              "confidence": "high",
              "option": "visual",
              "prompt": "Minivan",
              "thumbnail_url": "https://project-one-thumbnail.s3.us-west-2.amazonaws.com/63a09373ce36463e0199c8de/10.jpeg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAYRWJPOVHXE5SJ77T%2F20221222%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20221222T063247Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&x-id=GetObject&X-Amz-Signature=e6cd4f05a99ab3d85f4e69aa8d2ed8ddeedc9f5cfd648a86420eba6d5351cd65"
            },
            {
              "start": 439.46875,
              "end": 447.796875,
              "score": 84.82,
              "confidence": "high",
              "option": "visual",
              "prompt": "Truck",
              "thumbnail_url": "https://project-one-thumbnail.s3.us-west-2.amazonaws.com/63a09373ce36463e0199c8de/440.jpeg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAYRWJPOVHXE5SJ77T%2F20221222%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20221222T063247Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&x-id=GetObject&X-Amz-Signature=f25551358535e6dbf5c05c8f972d79cc5e9731c76db0bca4aebf658c775e3030"
            },
            {
              "start": 559.59375,
              "end": 562.25,
              "score": 83.56,
              "confidence": "high",
              "option": "visual",
              "prompt": "SUV",
              "thumbnail_url": "https://project-one-thumbnail.s3.us-west-2.amazonaws.com/63a09373ce36463e0199c8de/560.jpeg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAYRWJPOVHXE5SJ77T%2F20221222%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20221222T063247Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&x-id=GetObject&X-Amz-Signature=bf025ceae11a2669e54341345e621a935322591565d1c819a2e35b0fce8ea422"
            }
          ]
        }
      ]
    },
    {
      "video_id": "63996203ce36463e0199c8c2",
      "labels": [
        {
          "name": "Car Shapes",
          "max_score": 87.15,
          "duration": 288.8906249999999,
          "confidence": "high",
          "clips": [
            {
              "start": 9.25,
              "end": 12.78125,
              "score": 83.46,
              "confidence": "high",
              "option": "visual",
              "prompt": "Minivan",
              "thumbnail_url": "https://project-one-thumbnail.s3.us-west-2.amazonaws.com/63996203ce36463e0199c8c2/10.jpeg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAYRWJPOVHXE5SJ77T%2F20221222%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20221222T063247Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&x-id=GetObject&X-Amz-Signature=95e7f7f0bf385440c39934ef0c645fb4d2d4bfd90cfd8d161341f05b109e2624"
            },
            {
              "start": 412.0833333333333,
              "end": 415.72916666666663,
              "score": 83.37,
              "confidence": "high",
              "option": "visual",
              "prompt": "Hatchback",
              "thumbnail_url": "https://project-one-thumbnail.s3.us-west-2.amazonaws.com/63996203ce36463e0199c8c2/413.jpeg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAYRWJPOVHXE5SJ77T%2F20221222%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20221222T063247Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&x-id=GetObject&X-Amz-Signature=2425ff92b2d393fa2dc47d49275b19604bb52597e5377ac8fafd908047ea0077"
            },
            {
              "start": 559.59375,
              "end": 562.25,
              "score": 83.56,
              "confidence": "high",
              "option": "visual",
              "prompt": "SUV",
              "thumbnail_url": "https://project-one-thumbnail.s3.us-west-2.amazonaws.com/63996203ce36463e0199c8c2/560.jpeg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAYRWJPOVHXE5SJ77T%2F20221222%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20221222T063247Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&x-id=GetObject&X-Amz-Signature=c120504fc71cf402f2571ab2bfcd3d48fd6bcb5c7382b4f7cfe29fd099c3e0b2"
            }
          ]
        }
      ]
    }
  ],
  "page_info": {}
}

In the example output above, note that, for each video, the API service returns an array named clips containing detailed information about a single matching video fragment.

Classifying a video based on multiple labels

The following example code classifies the content of your videos based on two labels - Pets and Wild animals:

CLASSIFY_BULK_URL = f"{API_URL}/classify/bulk"
headers = {
    "x-api-key": API_KEY
}
data =  {
  "conversation_option": "semantic",
  "options": ["visual"],
  "threshold": "high",
  "index_id": INDEX_ID,
  "labels": [
      {
      "name": "Pets",
      "prompts": [
        "Cat",
        "Dog",
        "Guinea pig",
        "Parrot",
        "Goldfish"
      ]
    },
    {
      "name": "Wild Animals",
      "prompts": [
        "Crocodile",
        "Snake",
        "Hippo",
        "Lion",
        "Bear",
        "Wolf",
        "Lion",
        "Tiger"
      ]
    }
  ]
}
response = requests.post(CLASSIFY_BULK_URL, headers=headers, json=data)
print (f'Status code: {response.status_code}')
pprint(response.json())

const CLASSIFY_BULK_URL = `${API_URL}/classify/bulk`
const  data =  {
  "conversation_option": "semantic",
  "options": ["visual"],
  "threshold": "high",
  "index_id": INDEX_ID,
  "labels": [
      {
      "name": "Pets",
      "prompts": [
        "Cat",
        "Dog",
        "Guinea pig",
        "Parrot",
        "Goldfish"
      ]
    },
    {
      "name": "Wild Animals",
      "prompts": [
        "Crocodile",
        "Snake",
        "Hippo",
        "Lion",
        "Bear",
        "Wolf",
        "Lion",
        "Tiger"
      ]
    }
  ]
}
const resp = await axios.post(
  CLASSIFY_BULK_URL,
  data,
  {
    "headers": {
      "x-api-key": API_KEY
    }
  }
)
const { data: response } = resp;
console.log(`Status code: ${resp.status}`)
console.log(JSON.stringify(response,null,4))

The output should look similar to the following one:

Status code: 200
{
    "data": [
        {
            "video_id": "63996246ce36463e0199c8c4",
            "labels": [
                {
                    "name": "Pets",
                    "max_score": 83.73,
                    "duration": 13.71249999999992,
                    "confidence": "high"
                },
                {
                    "name": "Wild Animals",
                    "max_score": 85.58,
                    "duration": 125.77031249999995,
                    "confidence": "high"
                }
            ]
        },
        {
            "video_id": "63a2a2a7ce36463e0199c8e5",
            "labels": [
                {
                    "name": "Pets",
                    "max_score": 86.31,
                    "duration": 185.74843749999988,
                    "confidence": "high"
                },
                {
                    "name": "Wild Animals",
                    "max_score": 85.7,
                    "duration": 75.94843750000007,
                    "confidence": "high"
                }
            ]
        },
        {
            "video_id": "63996260ce36463e0199c8c5",
            "labels": [
                {
                    "name": "Pets",
                    "max_score": 83.23,
                    "duration": 4,
                    "confidence": "high"
                },
                {
                    "name": "Wild Animals",
                    "max_score": 87.4,
                    "duration": 531.2916666666666,
                    "confidence": "high"
                }
            ]
        }
    ],
    "page_info": {}
}

In the example output above, note that the data array contains three objects, each corresponding to a different video. For each video, the response contains information that helps you determine the likelihood that each of the labels you've specified in the request appears in that video.

Filtering

This endpoint supports filtering. For details, see the Filtering > Content classification page.

Pagination

When using the /classify/bulk endpoint, pagination works in a similar fashion to the /search endpoint. For details, see the Pagination > Search results page.