Retrieving thumbnails

To help users quickly preview videos, the API service allows you to retrieve thumbnails of your videos that you can display in your application.

You can retrieve thumbnails when:

  • Your application displays previews for one or more videos. For example, users may want to see thumbnails of all the videos in an index before deciding which video they wish to delete.
  • Your application displays previews for a set of search results. For example, when users perform a search, your application could use thumbnails to give users a small view of each matching video fragment.

Note the following about retrieving thumbnails:

  • Before you can retrieve and display thumbnails in your application, you must enable thumbnail generation, and indexes are the most granular level at which you can enable this feature
  • The API service returns a unique URL for each thumbnail, and the URLs expire in one hour.
  • You cannot disable thumbnails once an index has been created.

The steps for retrieving thumbnails are as follows:

  1. Create an index and enable thumbnail generation for it.
  2. Upload videos to your index.
  3. Retrieve the unique identifier of your video.
  4. Retrieve thumbnails.

Note that, although the example code in this guide is written in Python and Node.js the API is compatible with most programming languages, and you can also use Postman or other REST clients to send requests and view responses.

Prerequisites

  • Your video must meet the following requirements:
    • Video resolution: must be greater or equal than 360p and less than 1080p (FHD)
    • Duration: must be between 10 seconds and 2 hours (7,200s)
      If you require different options, send us an email at support[at]twelvelabs.io.
  • A valid Twelve Labs account. For details about creating an account and retrieving your API key, see the Authentication page.
  • You’re familiar with the concepts that are described on the Quickstart page.

1. Create an index

To create an index and enable thumbnail generation for it, the steps are similar to those described in the Quickstart > Create an index section, with the following exception: in the body of the request, you must set the addons parameter to ["thumbnail"].

The following example code shows how you can create an index and enable thumbnail generation for it. Make sure to replace the placeholder surrounded by <> with the name of your index.

INDEX_NAME = "<YOUR_INDEX_NAME>" # Use a descriptive name for your index 
INDEXES_URL = f"{API_URL}/indexes"
data = {
    "engine_id": "marengo2",
    "index_options": ["visual", "conversation", "text_in_video"],
    "index_name": INDEX_NAME,
    "addons": ["thumbnail"] # Enable thumbnail generation
}
response = requests.post(INDEXES_URL, headers=headers, json=data)
INDEX_ID = response.json().get("_id")
print (f"Status code: {response.status_code}")
pprint (response.json())
const INDEX_NAME = '<YOUR_INDEX_NAME>' // Use a descriptive name for your index
const INDEXES_URL = `${API_URL}/indexes`
let data = JSON.stringify({
  'engine_id': 'marengo2',
  'index_options': ['visual', 'conversation', 'text_in_video'],
  'index_name': INDEX_NAME,
  'addons': ['thumbnail'] // Enable thumbnail generation
  })
let config = {
  method: 'post',
  url: INDEXES_URL,
  headers: headers,
  data: data
}
let resp = await axios(config)
let response = await resp.data
const INDEX_ID = response._id
console.log(`Status code: ${resp.status}`)
console.log(response)

In the example code above, the addons field of the data dictionary enables thumbnail generation for this index.

The output should look similar to the following one:

Status code: 200
{
  "_id": "63976041117eacc7c7398a66"
}

2. Upload videos

Once you've created an index and enabled thumbnail generation for it, you can upload videos to it.

The following example code shows how you can upload a video. Make sure to replace the placeholders surrounded by <> with your values.

TASKS_URL = f"{API_URL}/tasks"
file_name = "<FILE_NAME>" # Example: "test.mp4"
file_path = "<FILE_PATH>" # Example: "/Downloads/test.mp4"
file_stream = open(file_path,"rb")
data = {
    "index_id": INDEX_ID, 
    "language": "en"
}
file_param=[
    ("video_file", (file_name, file_stream, "application/octet-stream")),]
response = requests.post(TASKS_URL, headers=headers, data=data, files=file_param)
TASK_ID = response.json().get("_id")

const TASKS_URL = `${API_URL}/tasks`
const file_path = '<FILE_PATH>' // Example: "/Downloads/test.mp4"
const file_stream = fs.createReadStream(file_path)
let formData = new FormData()
formData.append('INDEX_ID', INDEX_ID)
formData.append('language', 'en')
formData.append('video_file', file_stream)
config = {
      method: 'post',
      url: TASKS_URL,
      headers: headers,
      data : formData
};
resp = await axios(config)
response = await resp.data
const TASK_ID = response._id
console.log(`Status code: ${resp.status}`)
console.log(response)

The output should look similar to the following one:

Status code: 200
{
  "_id": "639760f4117eacc7c7398a69"
}

3. Retrieve the unique identifier of your video

  1. Use the /tasks/{_id} endpoint to monitor the indexing process. Wait until the status shows as ready:

    TASK_STATUS_URL = f"{API_URL}/tasks/{TASK_ID}"
    while True:
        response = requests.get(TASK_STATUS_URL, headers=headers)
        STATUS = response.json().get("status")
        if STATUS == "ready":
            break
        time.sleep(10)
    
    const TASK_STATUS_URL = `${API_URL}/tasks/${TASK_ID}`
    const uploadResp = await new Promise((res) => {
    const interval = setInterval(async () => {
          const { data: response } = await axios.get(
                TASK_STATUS_URL,
                {
                    headers: {
                        "x-api-key": API_KEY
                    }
                }
            );
            if (response.status == "ready") {
              clearInterval(interval);
                res(response);
            }
        }, 1000);
    });
    
  2. Store the unique identifier of your video in a variable named VIDEO_ID and print it:

    VIDEO_ID = response.json().get('video_id')
    print(f"VIDEO ID: {VIDEO_ID}")
    
    const VIDEO_ID = uploadResp.video_id
    

    The output should look similar to the following one:

    VIDEO ID: 6391c8a669ff3402ec515aba
    

    Note that you can also use webhooks to monitor the status of the indexing process. For details, see the Using webhooks section.

4. Retrieve thumbnails

There are two ways to retrieve thumbnails, depending on your use case:

  • To retrieve a thumbnail of a video, you can call the GET method of the /indexes/{index-id}/videos/{video_id}/thumbnail endpoint.
  • To retrieve and display thumbnails for a set of search results, you can use the thumbnail_url field returned by the POST method of the /search endpoint.

Note that the API service returns a unique URL for each thumbnail, and the URLs expire in one hour.

Retrieve a thumbnail of a video

The GET method of the /indexes/{index-id}/videos/{video_id}/thumbnail endpoint retrieves a thumbnail of an individual video. By default, when calling this method, the API service retrieves a thumbnail from the middle of the video. Optionally, you can pass the time query parameter to specify the time, in seconds, at which the API service must retrieve the thumbnail.

Retrieve a thumbnail from the middle of the video

The following example code retrieves a thumbnail from the middle of the video by calling the /indexes/{index-id}/videos/{video_id}/thumbnail endpoint and passing it the following arguments:

  • The unique identifier of your index
  • The unique identifier of your video.
THUMBNAILS_URL = f"{API_URL}/indexes/{INDEX_ID}/videos/{VIDEO_ID}/thumbnail"
response = requests.get(THUMBNAILS_URL, headers=headers)
print (f'Status code: {response.status_code}')
pprint (response.json())
THUMBNAILS_URL = `${API_URL}/indexes/${INDEX_ID}/videos/${VIDEO_ID}/thumbnail`
config = {
  method: 'get',
  url: THUMBNAILS_URL,
  headers: headers,
}
resp = await axios(config)
response = await resp.data
console.log(`Status code: ${resp.status}`)
console.log(response)

The output should look similar to the following one:

Status code: 200
{
  "thumbnail": "https://project-one-thumbnail.s3.us-west-2.amazonaws.com/6320748774fed2f3511f2a92/10.jpeg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAYRWJPOVHXE5SJ77T%2F20220914%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20220914T075213Z&X-Amz-Expires=86400&X-Amz-SignedHeaders=host&x-id=GetObject&X-Amz-Signature=46b3bac13fb8b022313244780c238cb6bfc53eddcbee3be23a8d2dd22d99543e"
}

Retrieve a thumbnail from a specific time

The following example code retrieves a thumbnail from a specific time by calling the /indexes/{index-id}/videos/{video_id}/thumbnail endpoint, and passing it the following parameters:

  • The unique identifier of your index
  • The unique identifier of your video
  • The time, in seconds, at which the API service must retrieve a thumbnail

Make sure to replace the placeholder surrounded by <> with your value.

TIME = <SPECIFY_VALUE> #Example: 50
THUMBNAILS_URL = f"{API_URL}/indexes/{YOUR_INDEX_ID}/videos/{YOUR_VIDEO_ID}/thumbnail?time={TIME}"
response = requests.get(THUMBNAILS_URL, headers=headers)
print (f'Status code: {response.status_code}')
pprint (response.json())
const TIME = <SPECIFY_VALUE> //Example: 50
THUMBNAILS_URL = `${API_URL}/indexes/${INDEX_ID}/videos/${VIDEO_ID}/thumbnail?time=${TIME}`
config = {
  method: 'get',
  url: THUMBNAILS_URL,
  headers: headers,
}
resp = await axios(config)
response = await resp.data
console.log(`Status code: ${resp.status}`)
console.log(response)

Retrieve thumbnails for a set of search results

When you perform a search on an index for which thumbnail generation is enabled, the API service returns a field named thumbnail_url that contains a thumbnail from the middle of each matching video fragment.

The following example code performs a search on an index for which thumbnail generation is enabled. Make sure to replace the placeholder surrounded by <> with your search query.

SEARCH_URL = f"{API_URL}/search"
data = {
    "query": "<YOUR_SEARCH_QUERY>",
    "index_id": INDEX_ID,
    "search_options": ["conversation", "visual", "text_in_video"],
    "operator": "or"
}
response = requests.post(SEARCH_URL, headers=headers, json=data)
print (f'Status code: {response.status_code}')
pprint (response.json())
const SEARCH_URL = `${API_URL}/search`
data = JSON.stringify(
  {
  'query': '<YOUR_SEARCH_QUERY>',
  'index_id': INDEX_ID,
  'search_options': ['conversation', 'visual', 'text_in_video'],
  'operator': 'or'
})
config = {
  method: 'post',
  url: SEARCH_URL,
  headers: headers,
  data: data
}
resp = await axios(config)
response = await resp.data
console.log(`Status code: ${resp.status}`)
console.log(response)

The following example output was truncated for brevity:

Status code: 200
{'conversation_option': 'semantic',
 'data': [{'confidence': 'high',
           'end': 426,
           'metadata': [{'type': 'visual'}],
           'score': 84.77,
           'start': 420,
           'thumbnail_url': 'https://project-one-thumbnail.s3.us-west-2.amazonaws.com/63284a7ff8fcd9232826c11e/421.jpeg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAYRWJPOVHXE5SJ77T%2F20220919%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20220919T161325Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&x-id=GetObject&X-Amz-Signature=5f37a929ea92e6c0a98707c55c9f2d73087ac889d486e3a3d1223fcc19bcd047',
           'video_id': '63284a7ff8fcd9232826c11e'},
          {'confidence': 'high',
           'end': 502,
           'metadata': [{'type': 'visual'}],
           'score': 83.98,
           'start': 484,
           'thumbnail_url': 'https://project-one-thumbnail.s3.us-west-2.amazonaws.com/63284a7ff8fcd9232826c11e/485.jpeg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAYRWJPOVHXE5SJ77T%2F20220919%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20220919T161325Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&x-id=GetObject&X-Amz-Signature=b4b741b06a44c2a206903fc65cfd7932a4186e66e8f473fa83317710c59df8f1',
           'video_id': '63284a7ff8fcd9232826c11e'}],
 'page_info': {'limit_per_page': 10,
               'next_page_token': 'eb87df6f-c32e-48e0-93ac-4346020288f5-1',
               'page_expired_at': '2022-09-19T16:18:25Z',
               'total_results': 35},
 'query': 'bear chasing a man',
 'response_type': 'clip',
 'search_options': ['conversation', 'visual', 'text_in_video'],
 'search_pool': {'index_id': '63284a57e0a250d0763d12a3',
                 'total_count': 1,
                 'total_duration': 532}
}

In this example output, the data array contains two objects, and each object has a property named thumbnail_url containing the URL of the thumbnail for the matching video fragment.