> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.twelvelabs.io/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.twelvelabs.io/_mcp/server.

# Organize a video library

> Discover the contents of a video collection, find the best organization strategy, and categorize every video in a multi-turn session.

This recipe shows you how to organize an entire video library in a multi-turn session. The final output is a complete library catalog as structured JSON. Every video is assigned to a group, ready for your CMS or DAM system.

**Use cases**:

* **Library management**: Categorize a new or growing video collection
* **Content audit**: Discover natural groupings before restructuring a library
* **Pipeline automation**: Generate a machine-readable catalog for a CMS or DAM system

# Key concepts

* **Knowledge store**: A persistent store of your videos and images plus the understanding the platform derives from them - spatiotemporal context, a typed ontology, and embeddings - that together enable corpus-level reasoning.
* **Multi-turn session**: A conversation where each request builds on previous turns. The first request creates a session, and follow-up requests reference that session to continue the conversation. For details, see the [Multi-turn sessions](/v1.3/agents/guides/create-a-response/multi-turn-sessions) page.
* **Structured output**: A JSON schema you provide with your request. Jockey constrains its output to match your schema, so you retrieve machine-readable results instead of plain text. For a guide on designing schemas and parsing responses, see the [Structured output](/v1.3/agents/guides/create-a-response/structured-output) page.

# Workflow

This recipe uses a multi-turn session in which each call builds on the previous ones. The first call requests a plain-text overview of the collection. The second call passes a JSON schema and asks [Jockey](/v1.3/agents/concepts/jockey) to discover and rank categorization strategies. The third call takes the top-ranked strategy and categorizes every video. Jockey refines its analysis as the session progresses. You can adapt this recipe to organize by different criteria or target a specific domain.

# Prerequisites

* You've already created a knowledge store with at least one item in `ready` status. See the [Create a knowledge store](/v1.3/agents/guides/create-a-knowledge-store) and [Add assets to a knowledge store](/v1.3/agents/guides/add-assets) guides for details.
* You're familiar with the request and response format. See the [Create a response](/v1.3/agents/guides/create-a-response) page for details.
* For best results, use a knowledge store with 10 or more videos.

# Complete example

Copy and paste the code below, replacing the placeholders surrounded by `<>` with your values.

**`Python`**

```python Python maxlines=50
import json
from twelvelabs import TwelveLabs, TextParam
from twelvelabs.types.text_param_format import TextParamFormat_JsonSchema

client = TwelveLabs(api_key="<YOUR_API_KEY>")
STORE_ID = "<YOUR_KNOWLEDGE_STORE_ID>"

# Step 1: Request a collection overview
print("Understanding your collection...")
response = client.responses.create(
    knowledge_store_id=STORE_ID,
    input=[{"type": "message", "role": "user", "content": "Give me a high-level overview of this video collection. What themes, subjects, and patterns do you see?"}],
)

# Read the overview and store the session ID
session_id = response.session_id
for output in response.output:
    if output.type == "message":
        for content in output.content:
            print(content.text)

# Define the organization axes schema
axes_schema = {
    "type": "object",
    "properties": {
        "recommended_axes": {
            "type": "array",
            "items": {
                "type": "object",
                "properties": {
                    "axis": {"type": "string"},                  # Name of the categorization dimension
                    "reason": {"type": "string"},                # Why this axis works well
                    "expected_categories": {                      # Sample category names
                        "type": "array",
                        "items": {"type": "string"},
                    },
                    "score": {"type": "number"},                 # 0-1 effectiveness score
                },
            },
        },
        "best_axis": {"type": "string"},                         # Top-ranked axis name
        "reasoning": {"type": "string"},                         # Overall recommendation rationale
    },
}

# Step 2: Discover organization strategies
print("\nFinding the best organization strategy...")
response = client.responses.create(
    knowledge_store_id=STORE_ID,
    session_id=session_id,
    input=[{"type": "message", "role": "user", "content": "What are the best ways to organize this collection? Rank the top 3 axes by how well they'd separate the content into useful groups."}],
    text=TextParam(format=TextParamFormat_JsonSchema(name="organization_axes", schema_=axes_schema)),
)

# Step 3: Parse the organization strategies
for output in response.output:
    if output.type == "message":
        for content in output.content:
            axes = json.loads(content.text)
            print(f"Best axis: {axes['best_axis']}")
            print(f"Reasoning: {axes['reasoning']}")
            for ax in axes["recommended_axes"]:
                print(f"  {ax['axis']} (score: {ax['score']}): {ax['reason']}")

# Define the catalog schema
catalog_schema = {
    "type": "object",
    "properties": {
        "organization_axis": {"type": "string"},             # The axis used for categorization
        "categories": {
            "type": "array",
            "items": {
                "type": "object",
                "properties": {
                    "name": {"type": "string"},              # Category name
                    "description": {"type": "string"},       # What this category contains
                    "videos": {
                        "type": "array",
                        "items": {
                            "type": "object",
                            "properties": {
                                "reference": {"type": "string"},  # Source video identifier
                                "title": {"type": "string"},      # Video title
                                "summary": {"type": "string"},    # Brief description
                            },
                        },
                    },
                },
            },
        },
        "total_videos": {"type": "integer"},                 # Total number of videos categorized
    },
}

# Step 4: Categorize videos
print(f"\nOrganizing by '{axes['best_axis']}'...")
response = client.responses.create(
    knowledge_store_id=STORE_ID,
    session_id=session_id,
    input=[{"type": "message", "role": "user", "content": f"Now organize every video by '{axes['best_axis']}'. Include all videos."}],
    text=TextParam(format=TextParamFormat_JsonSchema(name="video_catalog", schema_=catalog_schema)),
)

# Step 5: Parse the catalog
for output in response.output:
    if output.type == "message":
        for content in output.content:
            catalog = json.loads(content.text)
            print(f"\nLibrary Catalog ({catalog['total_videos']} videos)")
            print(f"Organized by: {catalog['organization_axis']}")
            for cat in catalog["categories"]:
                print(f"\n  [{cat['name']}] - {cat['description']}")
                for v in cat["videos"]:
                    print(f"    {v['title']}: {v['summary']}")
```

**`Node.js`**

```javascript Node.js maxlines=50
import { TwelveLabs } from "twelvelabs-js";

const client = new TwelveLabs({ apiKey: "<YOUR_API_KEY>" });
const storeId = "<YOUR_KNOWLEDGE_STORE_ID>";

// Step 1: Request a collection overview
console.log("Understanding your collection...");
let response = await client.responses.create({
  knowledgeStoreId: storeId,
  input: [{ type: "message", role: "user", content: "Give me a high-level overview of this video collection. What themes, subjects, and patterns do you see?" }],
});

// Read the overview and store the session ID
const sessionId = response.sessionId;
for (const output of response.output ?? []) {
  if (output.type === "message") {
    for (const content of output.content ?? []) {
      console.log(content.text);
    }
  }
}

// Define the organization axes schema
const axesSchema = {
  type: "object",
  properties: {
    recommended_axes: {
      type: "array",
      items: {
        type: "object",
        properties: {
          axis: { type: "string" }, // Name of the categorization dimension
          reason: { type: "string" }, // Why this axis works well
          expected_categories: { // Sample category names
            type: "array",
            items: { type: "string" },
          },
          score: { type: "number" }, // 0-1 effectiveness score
        },
      },
    },
    best_axis: { type: "string" }, // Top-ranked axis name
    reasoning: { type: "string" }, // Overall recommendation rationale
  },
};

// Step 2: Discover organization strategies
console.log("\nFinding the best organization strategy...");
response = await client.responses.create({
  knowledgeStoreId: storeId,
  sessionId,
  input: [{ type: "message", role: "user", content: "What are the best ways to organize this collection? Rank the top 3 axes by how well they'd separate the content into useful groups." }],
  text: { format: { type: "json_schema", name: "organization_axes", schema: axesSchema } },
});

// Step 3: Parse the organization strategies
let axes;
for (const output of response.output ?? []) {
  if (output.type === "message") {
    for (const content of output.content ?? []) {
      axes = JSON.parse(content.text);
      console.log(`Best axis: ${axes.best_axis}`);
      console.log(`Reasoning: ${axes.reasoning}`);
      for (const ax of axes.recommended_axes) {
        console.log(`  ${ax.axis} (score: ${ax.score}): ${ax.reason}`);
      }
    }
  }
}

// Define the catalog schema
const catalogSchema = {
  type: "object",
  properties: {
    organization_axis: { type: "string" }, // The axis used for categorization
    categories: {
      type: "array",
      items: {
        type: "object",
        properties: {
          name: { type: "string" }, // Category name
          description: { type: "string" }, // What this category contains
          videos: {
            type: "array",
            items: {
              type: "object",
              properties: {
                reference: { type: "string" }, // Source video identifier
                title: { type: "string" }, // Video title
                summary: { type: "string" }, // Brief description
              },
            },
          },
        },
      },
    },
    total_videos: { type: "integer" }, // Total number of videos categorized
  },
};

// Step 4: Categorize videos
console.log(`\nOrganizing by '${axes.best_axis}'...`);
response = await client.responses.create({
  knowledgeStoreId: storeId,
  sessionId,
  input: [{ type: "message", role: "user", content: `Now organize every video by '${axes.best_axis}'. Include all videos.` }],
  text: { format: { type: "json_schema", name: "video_catalog", schema: catalogSchema } },
});

// Step 5: Parse the catalog
for (const output of response.output ?? []) {
  if (output.type === "message") {
    for (const content of output.content ?? []) {
      const catalog = JSON.parse(content.text);
      console.log(`\nLibrary Catalog (${catalog.total_videos} videos)`);
      console.log(`Organized by: ${catalog.organization_axis}`);
      for (const cat of catalog.categories) {
        console.log(`\n  [${cat.name}] - ${cat.description}`);
        for (const v of cat.videos) {
          console.log(`    ${v.title}: ${v.summary}`);
        }
      }
    }
  }
}
```

# Code explanation

#### Python

#### Request a collection overview

Ask for a high-level overview. This first request creates the session that the later turns build on.\

**Function call**: You call the [`responses.create`](/v1.3/sdk-reference/python/responses#create-a-response) method with a prompt that requests an overview.\

**Parameters**:

* `knowledge_store_id`: The unique identifier of the knowledge store to reason over.
* `input`: An array of input items. Each item is a message you send to Jockey. This example requests a high-level overview of themes, subjects, and patterns.\


**Return value**: An object of type `ResponseObject`. Read the overview text from the `output` items, and keep the `session_id` for the follow-up turns. Jockey retains this overview as session context, so later turns build on it. The response also includes `id`, `status`, and `usage`.

#### Discover organization strategies

Continue the session with a prompt that ranks categorization strategies. Jockey uses the collection context from the previous turn.\

**Function call**: You call the [`responses.create`](/v1.3/sdk-reference/python/responses#create-a-response) method with the `session_id` and a `text` parameter that describes your axes schema.\

**Parameters**:

* `knowledge_store_id`: The unique identifier of the knowledge store. Must match the store used to create the session.
* `session_id`: The session identifier from the first response.
* `input`: An array of input items. Each item is a message you send to Jockey. This example asks Jockey to rank the top three categorization strategies by effectiveness.
* `text`: An object that specifies the response format. To return structured JSON, set the `format` field. Within it, the `schema_` field defines the structure the output must match, and the `name` field identifies the schema. Design the schema to fit your use case; the inline comments in the example describe each field. For the schema rules, see the [Structured output](/v1.3/agents/guides/create-a-response/structured-output#json-schema-requirements) page.\


**Return value**: An object of type `ResponseObject`. The `text` field of each content part in a `message` item is a JSON string that matches your axes schema.

#### Process the strategies

Parse the `text` field of each content part in a `message` item with the `json.loads()` method, then iterate the `recommended_axes` array to read each ranked strategy. Keep the `best_axis` value; the next step uses it to categorize the videos.

#### Categorize videos

Continue the session with a prompt that organizes every video by the top-ranked axis.\

**Function call**: You call the [`responses.create`](/v1.3/sdk-reference/python/responses#create-a-response) method with the `session_id` and a `text` parameter that describes your catalog schema.\

**Parameters**:

* `knowledge_store_id`: The unique identifier of the knowledge store. Must match the store used to create the session.
* `session_id`: The session identifier from the first response.
* `input`: An array of input items. Each item is a message you send to Jockey. This example inserts the `best_axis` value from the previous step into the prompt, so Jockey organizes every video by that axis.
* `text`: The catalog schema, in the same form as the previous step.\


**Return value**: An object of type `ResponseObject`. The `text` field of each content part in a `message` item is a JSON string that matches your catalog schema.

#### Process the catalog

Parse the `text` field of each content part in a `message` item with the `json.loads()` method, then iterate the `categories` array, and the `videos` within each category, to read the final library catalog.

#### Node.js

#### Request a collection overview

Ask for a high-level overview. This first request creates the session that the later turns build on.\

**Function call**: You call the [`responses.create`](/v1.3/sdk-reference/node-js/responses#create-a-response) method with a prompt that requests an overview. You pass all parameters as properties of a single object.\

**Parameters**:

* `knowledgeStoreId`: The unique identifier of the knowledge store to reason over.
* `input`: An array of input items. Each item is a message you send to Jockey. This example requests a high-level overview of themes, subjects, and patterns.\


**Return value**: An `HttpResponsePromise` that resolves to an object of type `ResponseObject`. Read the overview text from the `output` items, and keep the `sessionId` for the follow-up turns. Jockey retains this overview as session context, so later turns build on it. The response also includes `id`, `status`, and `usage`.

#### Discover organization strategies

Continue the session with a prompt that ranks categorization strategies. Jockey uses the collection context from the previous turn.\

**Function call**: You call the [`responses.create`](/v1.3/sdk-reference/node-js/responses#create-a-response) method with the `sessionId` and a `text` parameter that describes your axes schema.\

**Parameters**:

* `knowledgeStoreId`: The unique identifier of the knowledge store. Must match the store used to create the session.
* `sessionId`: The session identifier from the first response.
* `input`: An array of input items. Each item is a message you send to Jockey. This example asks Jockey to rank the top three categorization strategies by effectiveness.
* `text`: An object that specifies the response format. To return structured JSON, set the `format` field. Within it, the `type` field must be `"json_schema"`, the `schema` field defines the structure the output must match, and the `name` field identifies the schema. Design the schema to fit your use case; the inline comments in the example describe each field. For the schema rules, see the [Structured output](/v1.3/agents/guides/create-a-response/structured-output#json-schema-requirements) page.\


**Return value**: An `HttpResponsePromise` that resolves to an object of type `ResponseObject`. The `text` field of each content part in a `message` item is a JSON string that matches your axes schema.

#### Process the strategies

Parse the `text` field of each content part in a `message` item with the `JSON.parse()` method, then iterate the `recommended_axes` array to read each ranked strategy. Keep the `best_axis` value; the next step uses it to categorize the videos.

#### Categorize videos

Continue the session with a prompt that organizes every video by the top-ranked axis.\

**Function call**: You call the [`responses.create`](/v1.3/sdk-reference/node-js/responses#create-a-response) method with the `sessionId` and a `text` parameter that describes your catalog schema.\

**Parameters**:

* `knowledgeStoreId`: The unique identifier of the knowledge store. Must match the store used to create the session.
* `sessionId`: The session identifier from the first response.
* `input`: An array of input items. Each item is a message you send to Jockey. This example inserts the `best_axis` value from the previous step into the prompt, so Jockey organizes every video by that axis.
* `text`: The catalog schema, in the same `format` object form as the previous step.\


**Return value**: An `HttpResponsePromise` that resolves to an object of type `ResponseObject`. The `text` field of each content part in a `message` item is a JSON string that matches your catalog schema.

#### Process the catalog

Parse the `text` field of each content part in a `message` item with the `JSON.parse()` method, then iterate the `categories` array, and the `videos` within each category, to read the final library catalog.

# Example response

The `text` field inside each content part is a JSON string that matches your schema. After parsing, a typical result from the final categorization step looks like this:

```json
{
  "organization_axis": "Content Type",
  "categories": [
    {
      "name": "Tutorials",
      "description": "Step-by-step instructional videos covering product features",
      "videos": [
        {
          "reference": "069eb4e8-aeb0-7e83-8000-86413fcc296a",
          "title": "Getting Started with the Dashboard",
          "summary": "Walks through initial setup, navigation, and key workflows"
        },
        {
          "reference": "069e1e97-27f8-7a8e-8000-bf32ecd7fc8c",
          "title": "API Integration Tutorial",
          "summary": "Covers authentication, first API call, and error handling"
        }
      ]
    },
    {
      "name": "Product Demos",
      "description": "Feature demonstrations and product showcases",
      "videos": [
        {
          "reference": "069ea183-ed43-7d69-8000-e2e3ab3ea1d1",
          "title": "Analytics Dashboard Demo",
          "summary": "Live walkthrough of the analytics dashboard and reporting tools"
        }
      ]
    }
  ],
  "total_videos": 12
}
```

> **Notes**
>
> * The `reference` field in each video entry contains the asset identifier of the knowledge store item.
> * Jockey generates the `title` field from the content of the video, not from a filename or metadata field.
> * Jockey evaluates the actual content in your knowledge store. Results change depending on the videos you have indexed.

# Variations

Change the prompts or add the [`instructions`](/v1.3/api-reference/responses/create#request.body.instructions) parameter to adapt this recipe for different organizational needs.

* **Multi-axis catalog**: Run step 4 multiple times with different axes from step 2 to produce catalogs along different dimensions.
* **Hierarchical organization**: After the first categorization, send a follow-up prompt: "Now sub-organize the \[category] group by difficulty level."
* **Export**: Pipe the JSON output to a file for use in your CMS or DAM system.
* **Known criteria**: Skip the discovery steps and send a single request. Set the `instructions` parameter to a librarian role (for example, "You are a content librarian. Organize videos into clear, mutually exclusive categories.") and specify the categories directly in the prompt.

  Example:

  | Criteria    | Prompt                                                           |
  | ----------- | ---------------------------------------------------------------- |
  | By topic    | "Organize by main subject matter"                                |
  | By mood     | "Organize by emotional tone: upbeat, serious, neutral"           |
  | By audience | "Organize by target audience: beginners, intermediate, advanced" |
  | By format   | "Organize by format: tutorial, interview, demo, vlog"            |
  | By speaker  | "Group videos by who appears on screen"                          |

# Jupyter notebook

[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/twelvelabs-io/twelvelabs-developer-experience/blob/main/quickstarts/jockey/recipes/organize_video_library.ipynb)

# See also

* [Generate a corpus overview](/v1.3/agents/recipes/get-a-corpus-overview) - the standalone version of step 1
* [Find organization axes](/v1.3/agents/recipes/find-organization-axes) - the standalone version of steps 2–3
* [Multi-turn sessions](/v1.3/agents/guides/create-a-response/multi-turn-sessions) - more on session-based conversations
* [Structured output](/v1.3/agents/guides/create-a-response/structured-output) - more on JSON Schema responses
* [Create a response](/v1.3/api-reference/responses/create) - API reference for the Responses endpoint
* [Sports Jockey](https://www.twelvelabs.io/blog/sports-semantic-jockey) - similar workflow applied end-to-end in a complete sample application