> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.twelvelabs.io/v1.3/agents/get-started/quickstart/create-a-response/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.twelvelabs.io/_mcp/server.

# Create a response

> Upload videos and images, build a knowledge store, and generate an overview of your content.

> **Research preview**
>
> Jockey is in research preview. Availability, limits, and API surface may change before general availability.

This guide shows how to upload videos and images, build a knowledge store, and generate an overview of your content. You can also extract and track entities, search your content, and organize it by theme.

# Key concepts

This section explains the key concepts and terminology used in this guide:

* **Asset**: Your uploaded content. Once created, you can reference the same asset across multiple operations without uploading the file again.
* **Knowledge store**: A persistent store of your videos and images plus the understanding the platform derives from them - spatiotemporal context, a typed ontology, and embeddings - that together enable corpus-level reasoning.
* **Knowledge store item**: An asset added to a knowledge store. The platform processes each item asynchronously. When the item reaches the `ready` status, you can use it in downstream tasks.

# Workflow

Upload your videos and images as assets, then create a knowledge store. Add the assets to the knowledge store. The platform indexes the content asynchronously. When the items reach the `ready` status, generate a response from the knowledge store.

# Prerequisites

* To use the platform, you need an API key:

  If you don't have an account, [sign up](https://playground.twelvelabs.io/) for a free account.

  Go to the [API Keys](https://playground.twelvelabs.io/dashboard/api-keys) page.

  If you need to create a new key, select the **Create API Key** button. Enter a name and set the expiration period. The default is 12 months.

  Select the **Copy** icon next to your key to copy it to your clipboard.

* Depending on the programming language you are using, install the TwelveLabs SDK by entering one of the following commands:

  **`Python`**

  ```shell Python
  pip install --upgrade twelvelabs
  ```

  **`Node.js`**

  ```shell Node.js
  yarn add twelvelabs-js@latest # or npm install twelvelabs-js@latest
  ```

* Upload limits: Public video URLs up to 4 GB, local videos up to 200 MB, or images up to 32 MB. For local files up to 10 GB, see the [Upload content](/v1.3/agents/guides/upload-content) page.

# Starter code

Copy and paste the code below, replacing the placeholders surrounded by `<>` with your values. Adapt the query to match your content and what you want to learn from it.

**`Python`**

```python Python maxlines=40
from twelvelabs import TwelveLabs
import time

# Step 1: Initialize the client
client = TwelveLabs(api_key="<YOUR_API_KEY>")

# Step 2: Upload a video or an image
asset = client.assets.create(method="url", url="<YOUR_MEDIA_URL>")  # Use direct links to raw media files. Video hosting platforms and cloud storage sharing links are not supported
# Or use method="direct" and file=open("<YOUR_FILE_PATH>", "rb") to upload a local file up to 200 MB
print(f"Asset created: {asset.id}")

# Step 3: Check the status of the asset
while True:
    status = client.assets.retrieve(asset_id=asset.id).status
    if status == "ready":
        break
    elif status == "failed":
        raise Exception("Asset processing failed")
    print(f"Status: {status}, waiting...")
    time.sleep(5)
print("Asset ready")

# Step 4: Create a knowledge store
store = client.knowledge_stores.create(name="<YOUR_KNOWLEDGE_STORE_NAME>")
print(f"Knowledge store created: {store.id}")

# Step 5: Add the asset to the knowledge store
item = client.knowledge_store_items.create(
    knowledge_store_id=store.id,
    asset_id=asset.id,
    # asset_type="image",  # Uncomment if your asset is an image (the default is video)
)
print(f"Item added: {item.id}")

# Step 6: Check the status of the knowledge store item
while True:
    status = client.knowledge_store_items.retrieve(
        knowledge_store_id=store.id, item_id=item.id
    ).status
    if status == "ready":
        break
    elif status == "failed":
        raise Exception("Indexing failed")
    print(f"Status: {status}, waiting...")
    time.sleep(10)
print("Indexing complete")

# Step 7: Generate a response
response = client.responses.create(
    knowledge_store_id=store.id,
    input=[{"type": "message", "role": "user", "content": "Give me an overview of this content - main themes, key subjects, and any patterns."}],
)
for output in response.output:
    if output.type == "message":
        for content in output.content:
            print(content.text)
```

**`Node.js`**

```javascript Node.js maxlines=40
import { TwelveLabs } from "twelvelabs-js";
// Uncomment the next line if uploading a local file
// import fs from "fs";

// Step 1: Initialize the client
const client = new TwelveLabs({ apiKey: "<YOUR_API_KEY>" });

// Step 2: Upload a video or an image
const asset = await client.assets.create({ method: "url", url: "<YOUR_MEDIA_URL>" }); // Use direct links to raw media files. Video hosting platforms and cloud storage sharing links are not supported
// Or use method: "direct" and file: fs.createReadStream("<YOUR_FILE_PATH>") to upload a local file up to 200 MB
console.log(`Asset created: ${asset.id}`);

// Step 3: Check the status of the asset
while (true) {
  const { status } = await client.assets.retrieve(asset.id);
  if (status === "ready") break;
  if (status === "failed") throw new Error("Asset processing failed");
  console.log(`Status: ${status}, waiting...`);
  await new Promise((r) => setTimeout(r, 5000));
}
console.log("Asset ready");

// Step 4: Create a knowledge store
const store = await client.knowledgeStores.create({ name: "<YOUR_KNOWLEDGE_STORE_NAME>" });
console.log(`Knowledge store created: ${store.id}`);

// Step 5: Add the asset to the knowledge store
const item = await client.knowledgeStoreItems.create(store.id, {
  assetId: asset.id,
  // assetType: "image", // Uncomment if your asset is an image (the default is video)
});
console.log(`Item added: ${item.id}`);

// Step 6: Check the status of the knowledge store item
while (true) {
  const { status } = await client.knowledgeStoreItems.retrieve(store.id, item.id);
  if (status === "ready") break;
  if (status === "failed") throw new Error("Indexing failed");
  console.log(`Status: ${status}, waiting...`);
  await new Promise((r) => setTimeout(r, 10000));
}
console.log("Indexing complete");

// Step 7: Generate a response
const response = await client.responses.create({
  knowledgeStoreId: store.id,
  input: [{ type: "message", role: "user", content: "Give me an overview of this content - main themes, key subjects, and any patterns." }],
});
for (const output of response.output ?? []) {
  if (output.type === "message") {
    for (const content of output.content ?? []) {
      console.log(content.text);
    }
  }
}
```

# Code explanation

#### Import the SDK and initialize the client

Create a client instance with your API key to interact with the platform.

#### Upload a video or an image

Upload a video or an image using a publicly accessible URL to create an asset. The same call handles both.

#### Check the status of the asset

Asset processing is asynchronous. Poll the status of the asset until it is `ready` before you use it.

#### Create a knowledge store

Create a knowledge store. You add the asset to it in the next step, and the platform indexes it.

#### Add the asset to the knowledge store

Add the asset to the knowledge store. This creates a knowledge store item that the platform indexes. Set the `asset_type` parameter to `image` when the asset is an image. It defaults to `video`.

#### Check the status of the knowledge store item

Check the status of the knowledge store item until it reaches the `ready` status. Indexing runs asynchronously and usually takes longer than the asset processing in step 3.

#### Generate a response

Generate a response from your videos and images. This example requests an overview of the main themes, key subjects, and patterns across your content. Jockey searches, tracks entities, and reasons across the knowledge store, then returns a response. This example prints the response to the standard output.

# Next steps

* [Create a response](/v1.3/agents/guides/create-a-response) - the complete guide: generate responses from a knowledge store, customize [Jockey](/v1.3/agents/concepts/jockey)'s behavior with instructions, and inspect its intermediate outputs
* [Streaming](/v1.3/agents/guides/create-a-response/streaming) - receive tokens in real time instead of waiting for the full response
* [Structured output](/v1.3/agents/guides/create-a-response/structured-output) - receive typed JSON by providing a schema
* [Multi-turn sessions](/v1.3/agents/guides/create-a-response/multi-turn-sessions) - maintain conversation context across requests

## Agent Integration Reference

Structured integration guide for AI agents and LLMs. No narrative - schemas, sequencing, constraints.

### Authentication

```
Header: x-api-key: <YOUR_API_KEY>
Required on: every request
```

### Tools

Four tools, called in sequence. Each tool maps to one API endpoint.

#### 1. create\_asset

```json
{
  "name": "create_asset",
  "description": "Upload video, audio, or image content to Jockey",
  "endpoint": "POST /assets",
  "content_type": "multipart/form-data",
  "parameters": {
    "method": {"type": "string", "enum": ["direct", "url"], "required": true},
    "file": {"type": "binary", "required_if": "method == direct", "max_size": "200MB"},
    "url": {"type": "string", "required_if": "method == url", "max_size": "4GB", "note": "Direct link to a raw media file; video hosting platforms and cloud storage sharing links are not supported"},
    "filename": {"type": "string", "required": false},
    "enable_hls": {"type": "boolean", "required": false},
    "enable_thumbnail": {"type": "boolean", "required": false}
  },
  "returns": {"_id": "string", "status": "processing|ready|failed"}
}
```

**WAIT**: Poll `GET /assets/{_id}` until `status == "ready"` before proceeding.

#### 2. create\_knowledge\_store

```json
{
  "name": "create_knowledge_store",
  "description": "Create a knowledge store for your videos and images",
  "endpoint": "POST /knowledge-stores",
  "content_type": "application/json",
  "parameters": {
    "name": {"type": "string", "required": true},
    "description": {"type": "string", "required": false},
    "metadata": {"type": "object", "required": false, "max_pairs": 10}
  },
  "returns": {"_id": "string", "item_count": "integer"}
}
```

#### 3. create\_knowledge\_store\_item

```json
{
  "name": "create_knowledge_store_item",
  "description": "Add a video or an image to a knowledge store for indexing",
  "endpoint": "POST /knowledge-stores/{knowledge_store_id}/items",
  "content_type": "application/json",
  "parameters": {
    "knowledge_store_id": {"type": "string", "in": "path", "required": true},
    "asset_id": {"type": "string", "required": true},
    "asset_type": {"type": "'video' | 'image'", "required": false, "default": "video", "note": "Set to 'image' when the asset is an image"},
    "metadata": {"type": "object", "required": false}
  },
  "returns": {"_id": "string", "status": "queued|pending|processing|ready|failed"}
}
```

**WAIT**: Poll `GET /knowledge-stores/{knowledge_store_id}/items/{_id}` until `status == "ready"` before proceeding.

#### 4. create\_response

```json
{
  "name": "create_response",
  "description": "Query a knowledge store in natural language and get a response",
  "endpoint": "POST /responses",
  "content_type": "application/json",
  "parameters": {
    "input": {
      "type": "array",
      "required": true,
      "items": {
        "type": {"type": "string", "value": "message"},
        "role": {"type": "string", "enum": ["user", "assistant", "system"]},
        "content": {"type": "string"}
      }
    },
    "knowledge_store_id": {"type": "string", "required": true, "note": "ID of the knowledge store to reason over"},
    "session_id": {"type": "string", "required": false, "note": "Omit to start new session"},
    "instructions": {"type": "string", "required": false},
    "stream": {"type": "boolean", "default": false, "required": false},
    "include": {"type": "array", "required": false, "items": "intermediate_outputs"},
    "text": {"type": "object", "required": false, "note": "JSON schema for structured output"}
  },
  "returns": {
    "id": "string",
    "session_id": "string",
    "status": "completed|failed|in_progress|incomplete",
    "output": [{"type": "message", "content": [{"type": "output_text", "text": "string"}]}],
    "usage": {"input_tokens": "integer", "output_tokens": "integer"}
  }
}
```

### Response Schemas

#### Asset Response

```json
{
  "_id": "string",
  "method": "direct | url",
  "status": "processing | ready | failed",
  "filename": "string",
  "file_type": "string (MIME type)",
  "url": "string",
  "url_expires_at": "string (RFC 3339)",
  "created_at": "string (RFC 3339)",
  "user_metadata": "object"
}
```

#### Knowledge Store Response

```json
{
  "_id": "string",
  "name": "string",
  "description": "string",
  "ingestion_config": {
    "enrichment_config": {
      "type": "json_schema | description",
      "json_schema": "object",
      "description": "string"
    }
  },
  "item_count": "integer",
  "created_at": "string (RFC 3339)",
  "updated_at": "string (RFC 3339)",
  "metadata": "object"
}
```

#### Knowledge Store Item Response

```json
{
  "_id": "string",
  "asset_type": "video | image",
  "asset_id": "string",
  "status": "queued | pending | processing | ready | failed",
  "system_metadata": {
    "asset_type": "video | image (discriminator - matches the top-level asset_type)",
    "filename": "string",
    "width": "integer",
    "height": "integer",
    "codec_name": "string",
    "size": "integer",
    "duration": "number (video only)",
    "fps": "number (video only)"
  },
  "metadata": "object",
  "created_at": "string (RFC 3339)",
  "updated_at": "string (RFC 3339)"
}
```

#### Response Response

```json
{
  "id": "string",
  "knowledge_store_id": "string",
  "session_id": "string",
  "type": "response",
  "status": "completed | failed | in_progress | incomplete",
  "output": [
    {
      "type": "message | function_call | function_call_output",
      "id": "string",
      "status": "string",
      "role": "assistant",
      "content": [{ "type": "output_text", "text": "string" }]
    }
  ],
  "usage": { "input_tokens": "integer", "output_tokens": "integer" },
  "created_at": "string (ISO 8601)"
}
```

#### Error Response

```json
{
  "code": "string",
  "message": "string"
}
```

### State Machines

Asset: `processing` → `ready` | `failed`

Knowledge Store Item: `queued` → `pending` → `processing` → `ready` | `failed`

Response: `in_progress` → `completed` | `incomplete` | `failed`

### Sequencing Rules

```
create_asset ──[wait: status=ready]──► create_knowledge_store
                                              │
                                              ▼
                                     create_knowledge_store_item
                                              │
                                       [wait: status=ready]
                                              │
                                              ▼
                                       create_response
                                              │
                                       [reuse session_id for multi-turn]
```

* A knowledge store can be created at any time (no dependency on assets)
* An asset must be `ready` before it can be added as a knowledge store item
* A knowledge store item must be `ready` before the store can be queried
* `create_response` with `session_id` enables multi-turn conversation

Parallel operations:

* Multiple asset uploads can run simultaneously
* Multiple item additions can run simultaneously
* Knowledge store creation and asset uploads can run simultaneously

Sequential requirements:

* Asset must reach `ready` before adding to a knowledge store
* Knowledge store item must reach `ready` before querying

### Status Code Decision Table

```
IF status == "ready"      → PROCEED to next step
IF status == "failed"     → STOP, report error
IF status == "completed"  → READ output
IF status == "incomplete" → READ output (may be partial)
IF status is anything else → POLL and wait
```

### HTTP Status Codes

| Code  | Meaning                            | Retry?                    |
| ----- | ---------------------------------- | ------------------------- |
| `200` | Success                            | -                         |
| `201` | Created (assets, knowledge stores) | -                         |
| `202` | Accepted (knowledge store items)   | -                         |
| `400` | Bad request - fix parameters       | No                        |
| `401` | Unauthorized - check API key       | No                        |
| `403` | Forbidden - check permissions      | No                        |
| `404` | Not found - check resource ID      | No                        |
| `429` | Rate limited                       | Yes - exponential backoff |
| `500` | Server error                       | Yes - exponential backoff |

### Constraints

| Constraint             | Value                                      |
| ---------------------- | ------------------------------------------ |
| Direct upload max size | 200MB                                      |
| URL upload max size    | 4GB                                        |
| Metadata max pairs     | 10 per knowledge store or item collection  |
| Knowledge store        | Exactly 1 `knowledge_store_id` per request |
| `temperature`          | Accepted but reserved for future use       |
| `max_output_tokens`    | Accepted but reserved for future use       |

### Status Rules

| Resource             | Field    | Rule                                                       |
| -------------------- | -------- | ---------------------------------------------------------- |
| Asset                | `status` | Forward-only: processing → ready/failed                    |
| Knowledge Store Item | `status` | Forward-only: queued → pending → processing → ready/failed |

### Research Preview Limitations

| Limitation           | Detail                           |
| -------------------- | -------------------------------- |
| No webhooks          | Polling only for async status    |
| No real-time updates | Periodic processing, not instant |
| SaaS only            | No on-premise deployment         |

> Upload videos and images, build a knowledge store, and generate an overview of your content.