Organize a video library

Discover what’s in a video collection, find the best organization strategy, then categorize everything - in a single multi-turn session.

What you’ll build

A three-step workflow that produces a complete library catalog: overview, organization axes, then categorized videos. Each step builds on the previous one using session context.

Prerequisites

  • Complete the Quickstart to create a knowledge store with at least one item in ready status.
  • Read Create a response to understand the request and response format.
  • Best with 10+ videos for meaningful organization.

Step 1: Get a collection overview

Start a new session by requesting a high-level overview. Jockey creates a session and returns its ID for follow-up turns.

1import json
2import requests
3
4API_KEY = "YOUR_API_KEY"
5BASE_URL = "https://api.twelvelabs.io/v1.3"
6HEADERS = {"x-api-key": API_KEY, "Content-Type": "application/json"}
7STORE_ID = "your_knowledge_store_id"
8
9print("Step 1: Understanding your collection...")
10response = requests.post(
11 f"{BASE_URL}/responses",
12 headers=HEADERS,
13 json={
14 "model": "jockey1.0",
15 "input": [
16 {"type": "message", "role": "user", "content": "Give me a high-level overview of this video collection. What themes, subjects, and patterns do you see?"}
17 ],
18 "tools": [
19 {"type": "knowledge_store", "knowledge_store_id": STORE_ID}
20 ]
21 }
22)
23result = response.json()
24session_id = result["session_id"]
25
26for output in result["output"]:
27 if output["type"] == "message":
28 for content in output["content"]:
29 print(content["text"])

Step 2: Discover organization axes

Continue the session to ask for organization recommendations. Jockey uses the overview context from step 1 to suggest strategies scored by effectiveness.

1print("\nStep 2: Finding best organization strategy...")
2axes_schema = {
3 "type": "object",
4 "properties": {
5 "recommended_axes": {
6 "type": "array",
7 "items": {
8 "type": "object",
9 "properties": {
10 "axis": {"type": "string"},
11 "reason": {"type": "string"},
12 "expected_categories": {"type": "array", "items": {"type": "string"}},
13 "score": {"type": "number"}
14 }
15 }
16 },
17 "best_axis": {"type": "string"},
18 "reasoning": {"type": "string"}
19 }
20}
21
22response = requests.post(
23 f"{BASE_URL}/responses",
24 headers=HEADERS,
25 json={
26 "model": "jockey1.0",
27 "session_id": session_id,
28 "input": [
29 {"type": "message", "role": "user", "content": "What are the best ways to organize this collection? Rank the top 3 axes by how well they'd separate the content into useful groups."}
30 ],
31 "tools": [
32 {"type": "knowledge_store", "knowledge_store_id": STORE_ID}
33 ],
34 "text": {"format": "json_schema", "json_schema": axes_schema}
35 }
36)
37result = response.json()
38
39for output in result["output"]:
40 if output["type"] == "message":
41 for content in output["content"]:
42 axes = json.loads(content["text"])
43 print(f"Best axis: {axes['best_axis']}")
44 print(f"Reasoning: {axes['reasoning']}")
45 for ax in axes["recommended_axes"]:
46 print(f" {ax['axis']} (score: {ax['score']}): {ax['reason']}")

Step 3: Categorize by the best axis

Use the recommended axis from step 2 to organize every video into categories. The session context carries forward, so Jockey knows which axis to apply.

1print(f"\nStep 3: Organizing by '{axes['best_axis']}'...")
2catalog_schema = {
3 "type": "object",
4 "properties": {
5 "organization_axis": {"type": "string"},
6 "categories": {
7 "type": "array",
8 "items": {
9 "type": "object",
10 "properties": {
11 "name": {"type": "string"},
12 "description": {"type": "string"},
13 "videos": {
14 "type": "array",
15 "items": {
16 "type": "object",
17 "properties": {
18 "reference": {"type": "string"},
19 "title": {"type": "string"},
20 "summary": {"type": "string"}
21 }
22 }
23 }
24 }
25 }
26 },
27 "total_videos": {"type": "integer"}
28 }
29}
30
31response = requests.post(
32 f"{BASE_URL}/responses",
33 headers=HEADERS,
34 json={
35 "model": "jockey1.0",
36 "session_id": session_id,
37 "input": [
38 {"type": "message", "role": "user", "content": f"Now organize every video by '{axes['best_axis']}'. Include all videos."}
39 ],
40 "tools": [
41 {"type": "knowledge_store", "knowledge_store_id": STORE_ID}
42 ],
43 "text": {"format": "json_schema", "json_schema": catalog_schema}
44 }
45)
46
47result = response.json()
48for output in result["output"]:
49 if output["type"] == "message":
50 for content in output["content"]:
51 catalog = json.loads(content["text"])
52 print(f"\nLibrary Catalog ({catalog['total_videos']} videos)")
53 print(f"Organized by: {catalog['organization_axis']}")
54 for cat in catalog["categories"]:
55 print(f"\n [{cat['name']}] - {cat['description']}")
56 for v in cat["videos"]:
57 print(f" {v['title']}: {v['summary']}")

Shortcut: organize by known criteria

If you already know your criteria, skip steps 1-2 and organize directly. Use instructions to set the librarian role and specify categories in the prompt.

1org_schema = {
2 "type": "object",
3 "properties": {
4 "criteria": {"type": "string"},
5 "categories": {
6 "type": "array",
7 "items": {
8 "type": "object",
9 "properties": {
10 "name": {"type": "string"},
11 "description": {"type": "string"},
12 "video_count": {"type": "integer"},
13 "videos": {
14 "type": "array",
15 "items": {
16 "type": "object",
17 "properties": {
18 "reference": {"type": "string"},
19 "reason": {"type": "string"}
20 }
21 }
22 }
23 }
24 }
25 },
26 "uncategorized": {
27 "type": "array",
28 "items": {"type": "string"}
29 }
30 }
31}
32
33response = requests.post(
34 f"{BASE_URL}/responses",
35 headers=HEADERS,
36 json={
37 "model": "jockey1.0",
38 "instructions": "You are a content librarian. Organize videos into clear, mutually exclusive categories. Every video should appear in exactly one category.",
39 "input": [
40 {
41 "type": "message",
42 "role": "user",
43 "content": "Organize these videos by content type: tutorials, interviews, product demos, and announcements."
44 }
45 ],
46 "tools": [
47 {"type": "knowledge_store", "knowledge_store_id": STORE_ID}
48 ],
49 "text": {"format": "json_schema", "json_schema": org_schema}
50 }
51)

Example criteria you can use:

CriteriaPrompt
By topic”Organize by main subject matter”
By mood”Organize by emotional tone: upbeat, serious, neutral”
By audience”Organize by target audience: beginners, intermediate, advanced”
By format”Organize by format: tutorial, interview, demo, vlog”
By speaker”Group videos by who appears on screen”

Variations

  • Multi-axis catalog: Run step 3 multiple times with different axes from step 2
  • Hierarchical: After the first organization, follow up: “Now sub-organize the [category] group by difficulty level”
  • Export: Pipe the JSON output to a file for use in your CMS or DAM system

Jupyter notebook

Download the notebook to run this recipe interactively.

See also