Multipart uploads

The Multipart Upload API is available as a limited, private preview. To request access to the preview version, contact us at sales@twelvelabs.io.

The MultipartUploadClient interface provides methods to manage multipart upload sessions.

Utilize multipart uploads to upload large files by dividing them into smaller chunks. This approach enables reliable uploads of large files, particularly when processing chunks in parallel or resuming interrupted transfers. TwelveLabs recommends using this method for files larger than 200 MB, with a maximum size limit of 4 GB.

This method creates an asset that you can use in different workflows.

Workflow

1

Determine the total size of your file in bytes. You’ll need this value when creating the upload session.

2

Create an upload session: Call the multipartUpload.create` method, providing details about your file, including its size. The response contains, among other information, the unique identifier of the asset, a list of upload URLs, and the size of each chunk in bytes.

3

Split your file: Use the chunk size from the response to divide your file into chunks of the specified size.

4

Upload chunks: Transfer each chunk to its designated presigned URL. You can upload the chunks in parallel for improved performance. Save the ETag from each upload response for progress reporting.

5

(Optional) Request additional URLs: Call the multipartUpload.getAdditionalPresignedUrls method if you need URLs for remaining chunks or if existing URLs expire.

6

Report progress: Submit completed chunks via the multipartUpload.getAdditionalPresignedUrls method in batches as chunks finish uploading. Use the ETag from each chunk upload as proof of successful transfer.

7

Confirm completion: The upload is complete when the multipartUpload.getStatus method returns status: 'completed'. Use the asset ID from step 1 to perform additional operations on your uploaded video.

8

What you do next depends on your use case:

  • For creating embeddings (videos, audio, images): Use the asset ID with the Embed API v2.
  • For search and analysis (videos): Index an asset using the asset ID.

Methods

List incomplete uploads

Description: This method returns a list of all incomplete multipart upload sessions in your account.

Function signature and example:

1listIncompleteUploads(
2 request?: TwelvelabsApi.MultipartUploadListIncompleteUploadsRequest,
3 requestOptions?: MultipartUpload.RequestOptions
4): Promise<core.Page<TwelvelabsApi.IncompleteUploadSummary>>

Parameters:

NameTypeRequiredDescription
pagenumberNoA number that identifies the page to retrieve. Default: 1.
pageLimitnumberNoThe number of items to return on each page. Default: 10. Max: 50.
requestOptionsMultipartUpload.RequestOptionsNoRequest-specific configuration.

Return value: Returns a Promise that resolves to a Page<IncompleteUploadSummary> object that allows you to iterate through the paginated results.

The Page<T> interface contains the following properties and methods:

NameTypeDescription
dataT[]An array containing the current page of items.
hasNextPage()booleanReturns whether there is a next page to load.
getNextPage()Promise<Page<T>>Retrieves the next page and returns the updated Page object.
[Symbol.asyncIterator]()AsyncIterator<T>Allows iteration through all items across all pages using for await loops.

The IncompleteUploadSummary interface contains the following properties:

NameTypeDescription
uploadIdstringThe unique identifier of your upload session.
statusTwelvelabsApi.MultipartUploadStatusTypeThe current status of the upload session.
totalSizenumberTotal size of the file in bytes.
chunkSizenumberThe size of each chunk in bytes.
totalChunksnumberThe total number of chunks in this upload session.
createdAtDateThe date and time when the upload session was created.
expiresAtDateThe date and time when the upload session expires.

API Reference: List incomplete uploads

Create a multipart upload session

Description: This method creates a multipart upload session.

Supported content: Video and audio

File size: 4GB maximum.

Additional requirements depend on your workflow:

Function signature and example:

1create(
2 request: TwelvelabsApi.CreateAssetUploadRequest,
3 requestOptions?: MultipartUpload.RequestOptions
4): core.HttpResponsePromise<TwelvelabsApi.CreateAssetUploadResponse>

Parameters:

NameTypeRequiredDescription
filenamestringYesThe original file name of the asset.
totalSizenumberYesThe total size of the file in bytes. The platform uses this value to:
- Calculate the optimal chunk size.
- Determine the total number of chunks required
- Generate the initial set of presigned URLs
requestOptionsMultipartUpload.RequestOptionsNoRequest-specific configuration.

Return value: Returns a Promise that resolves to a CreateAssetUploadResponse object containing the upload session details.

The CreateAssetUploadResponse interface contains the following properties:

NameTypeDescription
uploadIdstringThe unique identifier of this upload session. Store this value, as you’ll need it for the following subsequent operations:
- Reporting completed chunks
- Requesting additional presigned URLs
- Retrieving the status of this upload session

This identifier remains valid for 24 hours from the time of creation.
assetIdstringThe unique identifier for the asset being created. Store this value, as you’ll need it to reference the asset in other API calls. Note that this identifier is reserved immediately, but the asset becomes available for other operations only after the upload is completed successfully.
uploadUrlsTwelvelabsApi.PresignedUrlChunk[]The initial set of presigned URLs for uploading chunks. Each URL corresponds to a specific chunk.

NOTES:
- URLs expire after one hour.
- Depending on the size of the file, this initial set may not include URLs for all chunks. You can request more using the getAdditionalPresignedUrls method.
uploadHeadersRecord<string, string>Headers to include when uploading chunks to the presigned URLs.
chunkSizenumberThe size in bytes for each chunk, except for the last chunk, which may be smaller. Use this value to divide your file into chunks of this exact size.
totalChunksnumberThe total number of chunks into which your file must be split. Calculated as: ceiling(totalSize / chunkSize).
expiresAtDateA string representing the date and time, in RFC 3339 format (“YYYY-MM-DDTHH:mm:ssZ”), when the upload URL will expire. Upload URLs expire 24 hours from their creation. After expiration, you cannot resume the upload, and you must create a new upload session.

The PresignedUrlChunk interface contains the following properties:

NameTypeDescription
chunkIndexnumberThe index of this chunk.
urlstringThe presigned URL for uploading this chunk. Each URL can only be used once and expires after one hour.
expiresAtDateThe date and time when the presigned URL expires.

API Reference: Create a multipart upload session

Retrieve the status of an upload session

Description: This method provides information about an upload session, including its current status, chunk-level progress, and completion state.

Use this method to:

  • Verify upload completion (status = completed)
  • Identify any failed chunks that require a retry
  • Monitor the upload progress by comparing uploaded_size with total_size
  • Determine if the session has expired
  • Retrieve the status information for each chunk

You must call this method after reporting chunk completion to confirm the upload has transitioned to the completed status before using the asset.

Function signature and example:

1getStatus(
2 uploadId: string,
3 request?: TwelvelabsApi.MultipartUploadGetStatusRequest,
4 requestOptions?: MultipartUpload.RequestOptions
5): Promise<core.Page<TwelvelabsApi.ChunkInfo>>

Parameters:

NameTypeRequiredDescription
uploadIdstringYesThe unique identifier of the upload session.
pagenumberNoA number that identifies the page to retrieve. Default: 1.
pageLimitnumberNoThe number of items to return on each page. Default: 10. Max: 50.
requestOptionsMultipartUpload.RequestOptionsNoRequest-specific configuration.

Return value: Returns a Promise that resolves to a Page<ChunkInfo> object that allows you to iterate through the paginated chunk status information.

The Page<T> interface contains the following properties and methods:

NameTypeDescription
dataT[]An array containing the current page of items.
hasNextPage()booleanReturns whether there is a next page to load.
getNextPage()Promise<Page<T>>Retrieves the next page and returns the updated Page object.
[Symbol.asyncIterator]()AsyncIterator<T>Allows iteration through all items across all pages using for await loops.

The ChunkInfo interface contains the following properties:

NameTypeDescription
indexnumberThe index of the chunk. The platform uses 1-based indexing, and this value matches the value of the chunkIndex field in the list of upload URLs.
statusTwelvelabsApi.ChunkInfoStatusThe current status of this chunk. Values:
- completed: Successfully uploaded and reported.
- pending: Not yet reported. A chunk may be in this status if it has been uploaded but not yet reported.
- failed: The upload process failed; you must retry uploading this chunk.
uploadedAtDateThe date and time, in the RFC 3339 format (“YYYY-MM-DDTHH:mm:ssZ”), when this chunk was successfully reported as uploaded. The value of this field is null for pending or failed chunks.
updatedAtDateThe date and time when this chunk was last updated.
errorstringA detailed error message explaining why this chunk failed. The platform returns this field only when the status is failed.

API Reference: Retrieve the status of an upload session

Report uploaded chunks

Description: This method reports successfully uploaded chunks to the platform. The platform finalizes the upload after you report all chunks.

For optimal performance, report chunks in batches and in any order.

Function signature and example:

1reportChunkBatch(
2 uploadId: string,
3 request: TwelvelabsApi.ReportChunkBatchRequest,
4 requestOptions?: MultipartUpload.RequestOptions
5): core.HttpResponsePromise<TwelvelabsApi.ReportChunkBatchResponse>

Parameters:

NameTypeRequiredDescription
uploadIdstringYesThe unique identifier of the upload session.
completedChunksTwelvelabsApi.CompletedChunk[]YesThe list of chunks successfully uploaded that you’re reporting to the platform. Report only after receiving an ETag.
requestOptionsMultipartUpload.RequestOptionsNoRequest-specific configuration.

The CompletedChunk interface contains the following properties:

NameTypeDescription
chunkIndexnumberThe number that identifies which chunk you uploaded. When you received presigned URLs from the platform, each URL was assigned an index number. Use that same number. The chunks are numbered starting from 1.
proofstringThe ETag value you received after uploading the chunk. When you upload a chunk to a presigned URLs, the response includes an ETag. Use this value and submit it as proof of successful upload.
proofType"etag"The verification method. Supported value: etag. Default: etag.
chunkSizenumberThe number of bytes uploaded for this chunk. For all chunks except the last, this value equals chunkSize. For the last chunk, it may be smaller.

Return value: Returns a Promise that resolves to a ReportChunkBatchResponse object containing information about the reported chunks.

The ReportChunkBatchResponse interface contains the following properties:

NameTypeDescription
urlstringThe URL for accessing your asset. The platform returns this field only when all chunks are reported and the upload is complete. If absent, continue uploading and reporting the remaining chunks.
assetIdstringThe unique identifier of this asset.
processedChunksnumberThe number of chunks accepted from this specific request. This equals the number of chunks in your completedChunks array minus any duplicates.
duplicateChunksnumberThe number of chunks in this request that were already reported. Duplicates are ignored and don’t affect your upload.
totalCompletednumberThe cumulative count of all unique chunks successfully reported across all requests. When this equals totalChunks, the upload is complete.

API Reference: Report uploaded chunks

Request presigned URLs for the remaining chunks

Description: This method generates new presigned URLs for specific chunks that require uploading. Use this method in the following situations:

  • Your initial URLs have expired (URLs expire after one hour).
  • The initial set of presigned URLs does not include URLs for all chunks.
  • You need to retry failed chunk uploads with new URLs.

To specify which chunks need URLs, use the start and count parameters. For example, to generate URLs for chunks 21 to 30, use start=21 and count=10.

The response provides new URLs, each with a fresh expiration time of one hour.

Function signature and example:

1getAdditionalPresignedUrls(
2 uploadId: string,
3 request: TwelvelabsApi.RequestAdditionalPresignedUrLsRequest,
4 requestOptions?: MultipartUpload.RequestOptions
5): core.HttpResponsePromise<TwelvelabsApi.RequestAdditionalPresignedUrLsResponse>

Parameters:

NameTypeRequiredDescription
uploadIdstringYesThe unique identifier of the upload session.
startnumberYesThe index of the first chunk number to generate URLs for. Chunks are numbered from 1.
countnumberYesThe number of presigned URLs to generate starting from the index. You can request a maximum of 50 URLs in a single API call.
requestOptionsMultipartUpload.RequestOptionsNoRequest-specific configuration.

Return value: Returns a Promise that resolves to a RequestAdditionalPresignedUrLsResponse object containing the new presigned URLs.

The RequestAdditionalPresignedUrLsResponse interface contains the following properties:

NameTypeDescription
uploadIdstringThe unique identifier of the upload session associated with these URLs.
startIndexnumberThe index of the first chunk number in this set of URLs. Matches the start value from your request.
countnumberThe number of new URLs created. Matches the count value from your request.
uploadUrlsTwelvelabsApi.PresignedUrlChunk[]An array of additional presigned URLs for uploading chunks.
generatedAtDateThe date and time, in the RFC 3339 format (“YYYY-MM-DDTHH:mm:ssZ”), when these URLs were created. URLs remain valid for 1 hour from this time.
expiresAtDateThe date and time when the upload session expires.

API Reference: Request presigned URLs for the remaining chunks