> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.twelvelabs.io/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.twelvelabs.io/_mcp/server.

# Build a content agent

> Build a multi-step agent that produces structured creative output from video footage using domain-specific ingestion, instructions, and multi-turn refinement.

This recipe shows you how to build a multi-step agent that analyzes a video collection and produces structured creative output. The example constructs a micro-drama outline with scene breakdowns, character arcs, and clip references.

**Use cases**:

* **Micro-drama outlines**: Extract narrative elements and construct scene breakdowns from video footage
* **Training curricula**: Identify learning objectives and demonstrations for instructional design
* **Compliance audits**: Find claims, disclosures, and regulation references for review
* **Content plans**: Discover topics, audience signals, and gaps for editorial strategy

# Key concepts

* **Knowledge store**: A persistent store of your videos and images plus the understanding the platform derives from them - spatiotemporal context, a typed ontology, and embeddings - that together enable corpus-level reasoning.
* **Ingestion configuration**: A configuration that controls how the platform processes content added to a knowledge store. You set the [`ingestion_config`](/v1.3/api-reference/knowledge-stores/create#request.body.ingestion_config) parameter at creation time and cannot change it afterward.
* **Instructions**: Additional guidance that shapes Jockey's behavior for a specific domain or task.
* **Structured output**: A JSON schema you provide with your request. Jockey constrains its output to match your schema, so you retrieve machine-readable results instead of plain text. For a guide on designing schemas and parsing responses, see the [Structured output](/v1.3/agents/guides/create-a-response/structured-output) page.
* **Multi-turn session**: A conversation where each request builds on previous turns. The first request creates a session, and follow-up requests reference that session to continue the conversation. For details, see the [Multi-turn sessions](/v1.3/agents/guides/create-a-response/multi-turn-sessions) page.

# Workflow

This recipe combines three capabilities into a reusable agent pattern. First, you create a knowledge store with a domain-specific ingestion configuration. This configuration controls what the platform extracts from your videos during processing. Then, you call the [Responses API](/v1.3/api-reference/responses/create) with a JSON schema, instructions that give [Jockey](/v1.3/agents/concepts/jockey) a specific role and domain guidance, and a prompt. Jockey reasons across your knowledge store and returns structured output that matches your schema. Finally, you use the session identifier to refine the results in a follow-up turn. You can adapt this pattern to any creative or analytical domain.

# Prerequisites

* To use the platform, you need an API key:

  If you don't have an account, [sign up](https://playground.twelvelabs.io/) for a free account.

  Go to the [API Keys](https://playground.twelvelabs.io/dashboard/api-keys) page.

  If you need to create a new key, select the **Create API Key** button. Enter a name and set the expiration period. The default is 12 months.

  Select the **Copy** icon next to your key to copy it to your clipboard.

* Depending on the programming language you are using, install the TwelveLabs SDK by entering one of the following commands:

  **`Python`**

  ```shell Python
  pip install --upgrade twelvelabs
  ```

  **`Node.js`**

  ```shell Node.js
  yarn add twelvelabs-js@latest # or npm install twelvelabs-js@latest
  ```

* You're familiar with the request and response format. See the [Create a response](/v1.3/agents/guides/create-a-response) page for details.

# Complete example

Copy and paste the code below, replacing the placeholders surrounded by `<>` with your values.

**`Python`**

```python Python maxlines=50
import json
from twelvelabs import TwelveLabs, IngestionConfig, EnrichmentConfig_Description, TextParam
from twelvelabs.types.text_param_format import TextParamFormat_JsonSchema

client = TwelveLabs(api_key="<YOUR_API_KEY>")

# Step 1: Create a knowledge store with a domain-specific ingestion configuration
store = client.knowledge_stores.create(
    name="Microdrama Source Material",
    ingestion_config=IngestionConfig(
        enrichment_config=EnrichmentConfig_Description(
            description="Focus on characters and their emotions, interpersonal dynamics, conflicts, tension points, visual mood shifts, dialogue tone, and dramatic turning points. Track recurring characters across videos."
        )
    ),
)
store_id = store.id

# Step 2: Add video assets to your knowledge store and wait
# for processing to complete. See the Quickstart for details.

# Define a JSON schema for the narrative output
drama_schema = {
    "type": "object",
    "properties": {
        "title": {"type": "string"},                # Title for the micro-drama
        "logline": {"type": "string"},               # One-sentence summary
        "characters": {
            "type": "array",
            "items": {
                "type": "object",
                "properties": {
                    "name": {"type": "string"},      # Character name
                    "role": {"type": "string"},      # Protagonist, antagonist, etc.
                    "arc": {"type": "string"},       # Character development summary
                },
            },
        },
        "scenes": {
            "type": "array",
            "items": {
                "type": "object",
                "properties": {
                    "scene_number": {"type": "integer"},     # Position in sequence
                    "description": {"type": "string"},       # What happens in the scene
                    "video_reference": {"type": "string"},   # Source video identifier
                    "timestamp": {"type": "string"},         # Clip timestamp
                    "dramatic_function": {"type": "string"}, # Role in the narrative arc
                },
            },
        },
        "central_conflict": {"type": "string"},      # Main dramatic tension
        "resolution": {"type": "string"},            # How the conflict resolves
    },
}

# Step 3: Request a narrative outline
response = client.responses.create(
    knowledge_store_id=store_id,
    instructions="You are a micro-drama creator. Analyze the video collection for narrative potential. Identify characters, conflicts, and dramatic moments. Construct a compelling 60-second micro-drama outline.",
    input=[{"type": "message", "role": "user", "content": "Create a micro-drama from these videos. Focus on the strongest emotional arc you can find."}],
    text=TextParam(format=TextParamFormat_JsonSchema(name="micro_drama", schema_=drama_schema)),
)

# Step 4: Parse and read the outline
session_id = response.session_id
for output in response.output:
    if output.type == "message":
        for content in output.content:
            drama = json.loads(content.text)
            print(f"Title: {drama['title']}")
            print(f"Logline: {drama['logline']}")
            print(f"Conflict: {drama['central_conflict']}")
            for scene in drama["scenes"]:
                print(f"  Scene {scene['scene_number']}: {scene['description']}")

# Step 5: Refine the output in a follow-up turn
response = client.responses.create(
    knowledge_store_id=store_id,
    session_id=session_id,
    input=[{"type": "message", "role": "user", "content": "Make the conflict sharper. Can you find a stronger turning point moment?"}],
    text=TextParam(format=TextParamFormat_JsonSchema(name="micro_drama", schema_=drama_schema)),
)

# Process the refined output
for output in response.output:
    if output.type == "message":
        for content in output.content:
            drama = json.loads(content.text)
            print(f"\nRefined Title: {drama['title']}")
            print(f"Logline: {drama['logline']}")
            print(f"Conflict: {drama['central_conflict']}")
            for scene in drama["scenes"]:
                print(f"  Scene {scene['scene_number']}: {scene['description']}")
```

**`Node.js`**

```javascript Node.js maxlines=50
import { TwelveLabs } from "twelvelabs-js";

const client = new TwelveLabs({ apiKey: "<YOUR_API_KEY>" });

// Step 1: Create a knowledge store with a domain-specific ingestion configuration
const store = await client.knowledgeStores.create({
  name: "Microdrama Source Material",
  ingestionConfig: {
    enrichmentConfig: {
      type: "description",
      description:
        "Focus on characters and their emotions, interpersonal dynamics, conflicts, tension points, visual mood shifts, dialogue tone, and dramatic turning points. Track recurring characters across videos.",
    },
  },
});
const storeId = store.id;

// Step 2: Add video assets to your knowledge store and wait
// for processing to complete. See the Quickstart for details.

// Define a JSON schema for the narrative output
const dramaSchema = {
  type: "object",
  properties: {
    title: { type: "string" }, // Title for the micro-drama
    logline: { type: "string" }, // One-sentence summary
    characters: {
      type: "array",
      items: {
        type: "object",
        properties: {
          name: { type: "string" }, // Character name
          role: { type: "string" }, // Protagonist, antagonist, etc.
          arc: { type: "string" }, // Character development summary
        },
      },
    },
    scenes: {
      type: "array",
      items: {
        type: "object",
        properties: {
          scene_number: { type: "integer" }, // Position in sequence
          description: { type: "string" }, // What happens in the scene
          video_reference: { type: "string" }, // Source video identifier
          timestamp: { type: "string" }, // Clip timestamp
          dramatic_function: { type: "string" }, // Role in the narrative arc
        },
      },
    },
    central_conflict: { type: "string" }, // Main dramatic tension
    resolution: { type: "string" }, // How the conflict resolves
  },
};

// Step 3: Request a narrative outline
let response = await client.responses.create({
  knowledgeStoreId: storeId,
  instructions:
    "You are a micro-drama creator. Analyze the video collection for narrative potential. Identify characters, conflicts, and dramatic moments. Construct a compelling 60-second micro-drama outline.",
  input: [{ type: "message", role: "user", content: "Create a micro-drama from these videos. Focus on the strongest emotional arc you can find." }],
  text: { format: { type: "json_schema", name: "micro_drama", schema: dramaSchema } },
});

// Step 4: Parse and read the outline
const sessionId = response.sessionId;
for (const output of response.output ?? []) {
  if (output.type === "message") {
    for (const content of output.content ?? []) {
      const drama = JSON.parse(content.text);
      console.log(`Title: ${drama.title}`);
      console.log(`Logline: ${drama.logline}`);
      console.log(`Conflict: ${drama.central_conflict}`);
      for (const scene of drama.scenes) {
        console.log(`  Scene ${scene.scene_number}: ${scene.description}`);
      }
    }
  }
}

// Step 5: Refine the output in a follow-up turn
response = await client.responses.create({
  knowledgeStoreId: storeId,
  sessionId,
  input: [{ type: "message", role: "user", content: "Make the conflict sharper. Can you find a stronger turning point moment?" }],
  text: { format: { type: "json_schema", name: "micro_drama", schema: dramaSchema } },
});

// Process the refined output
for (const output of response.output ?? []) {
  if (output.type === "message") {
    for (const content of output.content ?? []) {
      const drama = JSON.parse(content.text);
      console.log(`\nRefined Title: ${drama.title}`);
      console.log(`Logline: ${drama.logline}`);
      console.log(`Conflict: ${drama.central_conflict}`);
      for (const scene of drama.scenes) {
        console.log(`  Scene ${scene.scene_number}: ${scene.description}`);
      }
    }
  }
}
```

# Code explanation

#### Python

#### Create a domain-focused knowledge store

Create a knowledge store with an ingestion configuration tailored to your domain, so the platform extracts the signals your task needs.\

**Function call**: You call the [`knowledge_stores.create`](/v1.3/sdk-reference/python/knowledge-stores#create-a-knowledge-store) method.\

**Parameters**:

* `name`: The name of the knowledge store.
* `ingestion_config`: An object that controls what the platform extracts during processing. You cannot change it after creation. It contains an `enrichment_config` object:
  * `type`: The enrichment type. Set to `"description"` for a natural-language description, or `"json_schema"` for precise typed fields. For details, see the [Configure ingestion](/v1.3/agents/guides/create-a-knowledge-store/configure-ingestion) page.
  * `description`: A natural-language description of what to extract. This example focuses on narrative elements: characters, emotions, conflicts, and dramatic turning points.\


**Return value**: An object of type `KnowledgeStore` with a field named `id` representing the unique identifier of the newly created knowledge store. Pass it to the Responses API calls that follow.

#### Add assets and wait for processing

Add video assets to your knowledge store and wait for processing to complete. See the [Upload content](/v1.3/agents/guides/upload-content) and [Add assets to a knowledge store](/v1.3/agents/guides/add-assets) guides for details.

#### Request a narrative outline

Generate the structured outline from the collection.\

**Function call**: You call the [`responses.create`](/v1.3/sdk-reference/python/responses#create-a-response) method with domain `instructions`, a prompt, and a `text` parameter that describes your narrative schema.\

**Parameters**:

* `knowledge_store_id`: The unique identifier of the knowledge store to reason over.
* `input`: An array of input items. Each item is a message you send to Jockey. This example asks Jockey to create a micro-drama, focusing on the strongest emotional arc.
* `instructions`: A per-request system prompt that shapes Jockey's behavior for a domain or task. This example uses a micro-drama creator role that identifies narrative elements.
* `text`: An object that specifies the response format. To return structured JSON, set the `format` field. Within it, the `schema_` field defines the structure the output must match, and the `name` field identifies the schema. Design the schema to fit your use case; the inline comments in the example describe each field. For the schema rules, see the [Structured output](/v1.3/agents/guides/create-a-response/structured-output#json-schema-requirements) page.\


**Return value**: An object of type `ResponseObject`. Read the `session_id` field to continue in the follow-up turn, and the `output` items for the outline. The response also includes `id`, `status`, and `usage`.

#### Process the results

The `text` field of each content part in a `message` item of the `output` array is a JSON string that matches your schema. Parse it with the `json.loads()` method, then iterate the `scenes` array to read each scene. Keep the `session_id` from the response for the next step.

#### Refine the output in a follow-up turn

Continue the session with a new prompt that adjusts the result. Jockey retains the context from the first turn.\

**Function call**: You call the [`responses.create`](/v1.3/sdk-reference/python/responses#create-a-response) method again with the `session_id` and the same schema.\

**Parameters**:

* `knowledge_store_id`: The unique identifier of the knowledge store. Must match the store from the first request.
* `session_id`: The session identifier from the first response.
* `input`: An array of input items. Each item is a message you send to Jockey. This example asks Jockey to sharpen the conflict and find a stronger turning point.
* `text`: The same schema as the first request.\


**Return value**: An object of type `ResponseObject` with the same structure as the first outline. Parse it the same way.

#### Node.js

#### Create a domain-focused knowledge store

Create a knowledge store with an ingestion configuration tailored to your domain, so the platform extracts the signals your task needs.\

**Function call**: You call the [`knowledgeStores.create`](/v1.3/sdk-reference/node-js/knowledge-stores#create-a-knowledge-store) method. You pass all parameters as properties of a single object.\

**Parameters**:

* `name`: The name of the knowledge store.
* `ingestionConfig`: An object that controls what the platform extracts during processing. You cannot change it after creation. It contains an `enrichmentConfig` object:
  * `type`: The enrichment type. Set to `"description"` for a natural-language description, or `"json_schema"` for precise typed fields. For details, see the [Configure ingestion](/v1.3/agents/guides/create-a-knowledge-store/configure-ingestion) page.
  * `description`: A natural-language description of what to extract. This example focuses on narrative elements: characters, emotions, conflicts, and dramatic turning points.\


**Return value**: An `HttpResponsePromise` that resolves to an object of type `KnowledgeStore` with a field named `id` representing the unique identifier of the newly created knowledge store. Pass it to the Responses API calls that follow.

#### Add assets and wait for processing

Add video assets to your knowledge store and wait for processing to complete. See the [Upload content](/v1.3/agents/guides/upload-content) and [Add assets to a knowledge store](/v1.3/agents/guides/add-assets) guides for details.

#### Request a narrative outline

Generate the structured outline from the collection.\

**Function call**: You call the [`responses.create`](/v1.3/sdk-reference/node-js/responses#create-a-response) method with domain `instructions`, a prompt, and a `text` parameter that describes your narrative schema. You pass all parameters as properties of a single object.\

**Parameters**:

* `knowledgeStoreId`: The unique identifier of the knowledge store to reason over.
* `input`: An array of input items. Each item is a message you send to Jockey. This example asks Jockey to create a micro-drama, focusing on the strongest emotional arc.
* `instructions`: A per-request system prompt that shapes Jockey's behavior for a domain or task. This example uses a micro-drama creator role that identifies narrative elements.
* `text`: An object that specifies the response format. To return structured JSON, set the `format` field. Within it, the `type` field must be `"json_schema"`, the `schema` field defines the structure the output must match, and the `name` field identifies the schema. Design the schema to fit your use case; the inline comments in the example describe each field. For the schema rules, see the [Structured output](/v1.3/agents/guides/create-a-response/structured-output#json-schema-requirements) page.\


**Return value**: An `HttpResponsePromise` that resolves to an object of type `ResponseObject`. Read the `sessionId` field to continue in the follow-up turn, and the `output` items for the outline. The response also includes `id`, `status`, and `usage`.

#### Process the results

The `text` field of each content part in a `message` item of the `output` array is a JSON string that matches your schema. Parse it with the `JSON.parse()` method, then iterate the `scenes` array to read each scene. Keep the `sessionId` from the response for the next step.

#### Refine the output in a follow-up turn

Continue the session with a new prompt that adjusts the result. Jockey retains the context from the first turn.\

**Function call**: You call the [`responses.create`](/v1.3/sdk-reference/node-js/responses#create-a-response) method again with the `sessionId` and the same schema.\

**Parameters**:

* `knowledgeStoreId`: The unique identifier of the knowledge store. Must match the store from the first request.
* `sessionId`: The session identifier from the first response.
* `input`: An array of input items. Each item is a message you send to Jockey. This example asks Jockey to sharpen the conflict and find a stronger turning point.
* `text`: The same schema as the first request.\


**Return value**: An `HttpResponsePromise` that resolves to an object of type `ResponseObject` with the same structure as the first outline. Parse it the same way.

# Example response

The `text` field inside each content part is a JSON string that matches your schema. After parsing, a typical result looks like this:

```json
{
  "title": "The Unspoken Divide",
  "logline": "Two colleagues navigate a silent rivalry that threatens to unravel their shared project.",
  "characters": [
    {
      "name": "Maya",
      "role": "Protagonist",
      "arc": "Moves from passive avoidance to direct confrontation"
    },
    {
      "name": "Daniel",
      "role": "Antagonist",
      "arc": "Shifts from confident control to vulnerability when challenged"
    }
  ],
  "scenes": [
    {
      "scene_number": 1,
      "description": "Maya and Daniel present their project update, tension visible in their body language",
      "video_reference": "069eb4e8-aeb0-7e83-8000-86413fcc296a",
      "timestamp": "02:15-02:42",
      "dramatic_function": "Establishes the core tension between the two characters"
    },
    {
      "scene_number": 2,
      "description": "Maya reviews documents alone, frustration building as she discovers discrepancies",
      "video_reference": "069e1e97-27f8-7a8e-8000-bf32ecd7fc8c",
      "timestamp": "08:30-08:55",
      "dramatic_function": "Rising action - reveals the source of the conflict"
    },
    {
      "scene_number": 3,
      "description": "Maya confronts Daniel in the hallway, his composure breaks",
      "video_reference": "069eb4e8-aeb0-7e83-8000-86413fcc296a",
      "timestamp": "14:02-14:28",
      "dramatic_function": "Climax - the unspoken divide becomes spoken"
    }
  ],
  "central_conflict": "Maya suspects Daniel has been taking credit for her contributions, but neither acknowledges it directly",
  "resolution": "Maya presents her work independently, forcing a public acknowledgment of her contributions"
}
```

> **Notes**
>
> * The `video_reference` field contains the asset identifier of the knowledge store item.
> * The `timestamp` field is a range in `MM:SS-MM:SS` format indicating the clip boundaries.
> * Jockey returns narrative outlines with clip references (video identifiers and timestamps), not rendered videos. Use these references with your editing tool or pipeline to assemble the final content.

# Variations

Change the ingestion configuration, instructions, or prompt to adapt this pattern for different domains and creative directions.

* **Change the genre**: Set the `instructions` parameter to "horror micro-drama creator", "comedy sketch writer", or "documentary short producer."
* **Focus on a character**: Change the prompt to "Build the drama around the most frequently appearing person."
* **Create a series**: Use a [multi-turn session](/v1.3/agents/guides/create-a-response/multi-turn-sessions) to extend the story: "Create episode 2, continuing from this ending."
* **Change the domain**: Replace the ingestion configuration, instructions, and schema to build a training curriculum designer, compliance auditor, or content planner.

# Jupyter notebook

[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/twelvelabs-io/twelvelabs-developer-experience/blob/main/quickstarts/jockey/recipes/build_content_agent.ipynb)

# See also

* [Structured output](/v1.3/agents/guides/create-a-response/structured-output) - more on JSON Schema responses
* [Multi-turn sessions](/v1.3/agents/guides/create-a-response/multi-turn-sessions) - more on session-based conversations
* [Configure ingestion](/v1.3/agents/guides/create-a-knowledge-store/configure-ingestion) - control what the platform extracts during processing
* [Create a response](/v1.3/api-reference/responses/create) - API reference for the Responses endpoint