> For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.twelvelabs.io/v1.3/agents/recipes/assemble-highlight-reels/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.twelvelabs.io/_mcp/server. # Assemble highlight reels > Find and sequence relevant clips from a video collection based on a theme, topic, or criteria. This recipe shows you how to find and sequence clips from a video collection based on a theme, topic, or set of criteria. The response includes clip references with video identifiers, timestamps, and assembly notes, ready for your editing tool or pipeline. **Use cases**: * **Marketing highlight reels**: Select product demos and customer reactions for promotional videos * **Event recaps**: Pull the best moments from conference or event footage * **Training compilations**: Assemble instructional clips by topic or skill level * **Content repurposing**: Find and sequence clips for social media or internal communications # Key concepts * **Knowledge store**: A persistent store of your videos and images plus the understanding the platform derives from them - spatiotemporal context, a typed ontology, and embeddings - that together enable corpus-level reasoning. * **Instructions**: Additional guidance that shapes Jockey's behavior for a specific domain or task. * **Structured output**: A JSON schema you provide with your request. Jockey constrains its output to match your schema, so you retrieve machine-readable results instead of plain text. For a guide on designing schemas and parsing responses, see the [Structured output](/v1.3/agents/guides/create-a-response/structured-output) page. # Workflow This recipe combines three elements: a JSON schema that defines the clip data structure, instructions that give [Jockey](/v1.3/agents/concepts/jockey) a specific role and domain guidance, and a prompt that describes what you want in the highlight reel. Jockey reasons across your knowledge store, finds the best matching moments, and returns structured clip references in the order it recommends. You can adapt this recipe to target different editorial styles or content themes. # Prerequisites * You've already created a knowledge store with at least one item in `ready` status. See the [Create a knowledge store](/v1.3/agents/guides/create-a-knowledge-store) and [Add assets to a knowledge store](/v1.3/agents/guides/add-assets) guides for details. * You're familiar with the request and response format. See the [Create a response](/v1.3/agents/guides/create-a-response) page for details. # Complete example Copy and paste the code below, replacing the placeholders surrounded by `<>` with your values. **`Python`** ```python Python maxlines=40 import json from twelvelabs import TwelveLabs, TextParam from twelvelabs.types.text_param_format import TextParamFormat_JsonSchema client = TwelveLabs(api_key="") STORE_ID = "" # Define a JSON schema for the clip references clip_schema = { "type": "object", "properties": { "assembly_title": {"type": "string"}, # Title for the highlight reel "clips": { "type": "array", "items": { "type": "object", "properties": { "video_reference": {"type": "string"}, # Source video identifier "start_time": {"type": "string"}, # Clip start timestamp "end_time": {"type": "string"}, # Clip end timestamp "description": {"type": "string"}, # What happens in the clip "relevance_reason": {"type": "string"}, # Why Jockey selected this clip }, }, }, "total_estimated_duration": {"type": "string"}, # Combined duration of all clips "assembly_notes": {"type": "string"}, # Sequencing and editorial guidance }, } # Assemble the reel response = client.responses.create( knowledge_store_id=STORE_ID, instructions="You are a video editor assembling a highlight reel. Select clips that flow well together with good pacing and variety.", input=[{"type": "message", "role": "user", "content": "Find the best clips showing product demos and customer reactions. I need a 2-minute highlight reel."}], text=TextParam(format=TextParamFormat_JsonSchema(name="highlight_reel", schema_=clip_schema)), ) # Parse and read the clips for output in response.output: if output.type == "message": for content in output.content: assembly = json.loads(content.text) print(f"Assembly: {assembly['assembly_title']}") print(f"Duration: {assembly['total_estimated_duration']}") print(f"Notes: {assembly['assembly_notes']}") for i, clip in enumerate(assembly["clips"], 1): print(f" {i}. [{clip['start_time']}-{clip['end_time']}] {clip['description']}") print(f" Why: {clip['relevance_reason']}") ``` **`Node.js`** ```javascript Node.js maxlines=40 import { TwelveLabs } from "twelvelabs-js"; const client = new TwelveLabs({ apiKey: "" }); const storeId = ""; // Define a JSON schema for the clip references const clipSchema = { type: "object", properties: { assembly_title: { type: "string" }, // Title for the highlight reel clips: { type: "array", items: { type: "object", properties: { video_reference: { type: "string" }, // Source video identifier start_time: { type: "string" }, // Clip start timestamp end_time: { type: "string" }, // Clip end timestamp description: { type: "string" }, // What happens in the clip relevance_reason: { type: "string" }, // Why Jockey selected this clip }, }, }, total_estimated_duration: { type: "string" }, // Combined duration of all clips assembly_notes: { type: "string" }, // Sequencing and editorial guidance }, }; // Assemble the reel const response = await client.responses.create({ knowledgeStoreId: storeId, instructions: "You are a video editor assembling a highlight reel. Select clips that flow well together with good pacing and variety.", input: [{ type: "message", role: "user", content: "Find the best clips showing product demos and customer reactions. I need a 2-minute highlight reel." }], text: { format: { type: "json_schema", name: "highlight_reel", schema: clipSchema } }, }); // Parse and read the clips for (const output of response.output ?? []) { if (output.type === "message") { for (const content of output.content ?? []) { const assembly = JSON.parse(content.text); console.log(`Assembly: ${assembly.assembly_title}`); console.log(`Duration: ${assembly.total_estimated_duration}`); console.log(`Notes: ${assembly.assembly_notes}`); assembly.clips.forEach((clip, i) => { console.log(` ${i + 1}. [${clip.start_time}-${clip.end_time}] ${clip.description}`); console.log(` Why: ${clip.relevance_reason}`); }); } } } ``` # Code explanation #### Python To assemble a highlight reel, call the [`responses.create`](/v1.3/sdk-reference/python/responses#create-a-response) method with editorial `instructions`, a prompt, and a `text` parameter that describes your result schema.\ **Parameters**: * `knowledge_store_id`: The unique identifier of the knowledge store to reason over. * `input`: An array of input items. Each item is a message you send to Jockey. This example describes the highlight reel: what clips to find, how long the reel should be, and the thematic criteria. * `instructions`: A per-request system prompt that shapes Jockey's behavior for a domain or task. This example uses a video editor role that prioritizes pacing and variety. * `text`: An object that specifies the response format. To return structured JSON, set the `format` field. Within it, the `schema_` field defines the structure the output must match, and the `name` field identifies the schema. Design the schema to fit your use case; the inline comments in the example describe each field. For the schema rules, see the [Structured output](/v1.3/agents/guides/create-a-response/structured-output#json-schema-requirements) page.\ **Return value**: An object of type `ResponseObject` containing, among other information, the following fields: * `id`: The unique identifier of the response. * `knowledge_store_id`: The knowledge store this response was generated against. * `session_id`: The session identifier. Pass it in a follow-up request to continue the conversation. * `status`: The status of the response. The possible values are `completed`, `failed`, `in_progress`, and `incomplete`. * `output`: The response output items. The `text` field of each content part in a `message` item is a JSON string that matches your schema. Parse it with the `json.loads()` method, then iterate the `clips` array to read each selection in order. * `usage`: Token usage statistics, including the `input_tokens` and `output_tokens` fields. #### Node.js To assemble a highlight reel, call the [`responses.create`](/v1.3/sdk-reference/node-js/responses#create-a-response) method with editorial `instructions`, a prompt, and a `text` parameter that describes your result schema. You pass all parameters as properties of a single object.\ **Parameters**: * `knowledgeStoreId`: The unique identifier of the knowledge store to reason over. * `input`: An array of input items. Each item is a message you send to Jockey. This example describes the highlight reel: what clips to find, how long the reel should be, and the thematic criteria. * `instructions`: A per-request system prompt that shapes Jockey's behavior for a domain or task. This example uses a video editor role that prioritizes pacing and variety. * `text`: An object that specifies the response format. To return structured JSON, set the `format` field. Within it, the `type` field must be `"json_schema"`, the `schema` field defines the structure the output must match, and the `name` field identifies the schema. Design the schema to fit your use case; the inline comments in the example describe each field. For the schema rules, see the [Structured output](/v1.3/agents/guides/create-a-response/structured-output#json-schema-requirements) page.\ **Return value**: An `HttpResponsePromise` that resolves to an object of type `ResponseObject` containing, among other information, the following fields: * `id`: The unique identifier of the response. * `knowledgeStoreId`: The knowledge store this response was generated against. * `sessionId`: The session identifier. Pass it in a follow-up request to continue the conversation. * `status`: The status of the response. The possible values are `completed`, `failed`, `in_progress`, and `incomplete`. * `output`: The response output items. The `text` field of each content part in a `message` item is a JSON string that matches your schema. Parse it with the `JSON.parse()` method, then iterate the `clips` array to read each selection in order. * `usage`: Token usage statistics, including the `inputTokens` and `outputTokens` fields. # Example response The `text` field inside each content part is a JSON string that matches your schema. After parsing, a typical result looks like this: ```json { "assembly_title": "Product Demo Highlights Q1", "clips": [ { "video_reference": "069eb4e8-aeb0-7e83-8000-86413fcc296a", "start_time": "01:22", "end_time": "01:45", "description": "Close-up product walkthrough with feature callouts", "relevance_reason": "Strong visual demonstration of core feature" }, { "video_reference": "069e1e97-27f8-7a8e-8000-bf32ecd7fc8c", "start_time": "03:10", "end_time": "03:38", "description": "Customer describing their positive experience", "relevance_reason": "Authentic testimonial with emotional impact" } ], "total_estimated_duration": "01:51", "assembly_notes": "Opens with product demo for context, transitions to customer reactions for social proof" } ``` > **Notes** > > * The `video_reference` field contains the asset identifier of the knowledge store item. > * The `start_time` and `end_time` fields use `MM:SS` format for videos under one hour and `HH:MM:SS` for longer content. > * Jockey returns clip references (video identifiers and timestamps), not rendered videos. Use these references with your video editing tool or pipeline to assemble the final reel. # Variations Change the instructions and prompt to adapt this recipe for different editorial styles and content focuses. * **Change the editorial style**: Set the `instructions` parameter to "documentary editor", "social media creator", or "training video producer." * **Change the content focus**: Change the prompt to request "funny moments", "technical deep dives", or "executive summaries." * **Refine with follow-up turns**: Use a [multi-turn session](/v1.3/agents/guides/create-a-response/multi-turn-sessions) to adjust the results: "Replace the second clip with something more energetic." # Jupyter notebook [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/twelvelabs-io/twelvelabs-developer-experience/blob/main/quickstarts/jockey/recipes/assemble_highlight_reels.ipynb) # See also * [Search a knowledge store](/v1.3/agents/guides/search-a-knowledge-store) - the single-step way to find clips * [Structured output](/v1.3/agents/guides/create-a-response/structured-output) - more on JSON Schema responses * [Create a response](/v1.3/api-reference/responses/create) - API reference for the Responses endpoint * [Sports Jockey](https://www.twelvelabs.io/blog/sports-semantic-jockey) - similar workflow applied end-to-end in a complete sample application > Find and sequence relevant clips from a video collection based on a theme, topic, or criteria.