Retrieve predictable, typed JSON responses by providing a JSON Schema with your request.

Key concepts

• JSON Schema: A standard for describing the structure of JSON data. You define the expected shape (properties, types, arrays), and the platform constrains its output to match.

Prerequisites

• You’ve already uploaded your videos and images, and the assets have reached the ready status. See the Upload content page for details.
• You’ve already created a knowledge store. See the Create a knowledge store page for details.
• You’ve already added at least one asset to the knowledge store, and the item has reached the ready status. See the Add assets to a knowledge store page for details.
• You’ve already read the Create a response page and understand the basic request and response format.

Example

Define a JSON Schema describing the output you want, then pass it in the text parameter. This example extracts themes and entities from your videos and images. Replace the placeholders surrounded by <> with your values.

1
import json
2
from twelvelabs import TwelveLabs, TextParam
3
from twelvelabs.types.text_param_format import TextParamFormat_JsonSchema
4
5
client = TwelveLabs(api_key="<YOUR_API_KEY>")
6
7
response = client.responses.create(
8
    knowledge_store_id="<YOUR_KNOWLEDGE_STORE_ID>",
9
    input=[{"type": "message", "role": "user", "content": "List the main themes and key entities in these videos and images"}],
10
    text=TextParam(
11
        format=TextParamFormat_JsonSchema(
12
            name="themes_and_entities",
13
            schema_={
14
                "type": "object",
15
                "properties": {
16
                    "themes": {"type": "array", "items": {"type": "string"}},
17
                    "entities": {
18
                        "type": "array",
19
                        "items": {
20
                            "type": "object",
21
                            "properties": {
22
                                "name": {"type": "string"},
23
                                "type": {"type": "string"},
24
                                "frequency": {"type": "string"},
25
                            },
26
                        },
27
                    },
28
                },
29
            },
30
        )
31
    ),
32
)
33
34
for output in response.output:
35
    if output.type == "message":
36
        for content in output.content:
37
            structured = json.loads(content.text)
38
            print(f"Themes: {structured['themes']}")
39
            for entity in structured["entities"]:
40
                print(f"  {entity['name']} ({entity['type']})")

Code explanation

Example response

The response text is a JSON string that matches your schema:

1
{
2
  "themes": ["outdoor recreation", "team building", "nature conservation"],
3
  "entities": [
4
    {"name": "John Rivera", "type": "person", "frequency": "3 items"},
5
    {"name": "Yellowstone", "type": "location", "frequency": "2 items"}
6
  ]
7
}

Combine with instructions

Use the instructions field to apply a domain-specific lens, and a schema to capture the results in a structured format. This example enriches videos and images through a brand strategy perspective.

1
enrichment_schema = {
2
    "type": "object",
3
    "properties": {
4
        "enrichments": {
5
            "type": "array",
6
            "items": {
7
                "type": "object",
8
                "properties": {
9
                    "item_reference": {"type": "string"},
10
                    "original_summary": {"type": "string"},
11
                    "enriched_analysis": {"type": "string"},
12
                    "new_insights": {"type": "array", "items": {"type": "string"}},
13
                    "tags": {"type": "array", "items": {"type": "string"}},
14
                },
15
            },
16
        }
17
    },
18
}
19
20
response = client.responses.create(
21
    knowledge_store_id="<YOUR_KNOWLEDGE_STORE_ID>",
22
    instructions="You are a brand strategist. Analyze videos and images through the lens of brand perception and marketing effectiveness.",
23
    input=[{"type": "message", "role": "user", "content": "For each item, provide enriched analysis focusing on brand messaging effectiveness, audience engagement signals, and production quality."}],
24
    text=TextParam(format=TextParamFormat_JsonSchema(name="enrichment", schema_=enrichment_schema)),
25
)

Swap the instructions field to apply a different analysis lens. The schema stays the same:

JSON schema requirements

Your schema must adhere to the JSON Schema Draft 2020-12 specification and must meet the requirements below:

• Supported data types: array, boolean, integer, null, number, object, and string.
• Automatic schema changes: The platform adds all object properties to required and sets additionalProperties to false on every object. You do not need to include required or additionalProperties in your schema.
• Unsupported keywords: Do not use oneOf, default, if/then/else, not, patternProperties, contains, or prefixItems. They may produce incomplete or malformed output without returning an error. Use anyOf instead of oneOf, and properties instead of patternProperties. Omit the others.
• Subschema references: You can reference subschemas using $ref. Define subschemas within $defs at the root of the schema. External URIs and relative-path references are not supported. For details, see the JSON Schema documentation on $defs.

For complete specifications, see the schema field in the API Reference section.

Common pitfalls

• Large schemas can reduce output quality. Keep nesting to 5 levels of objects or fewer. Keep the total number of properties across all objects to 100 or fewer. These are best-practice guidelines, not enforced limits.
• Response text is a JSON string. You still need to parse the text content with json.loads() (Python) or JSON.parse() (Node.js) into a native object.
• Truncated responses may fail to parse. Even with a valid schema, the platform may truncate the response at the token limit. If the status field is incomplete, simplify your schema or break the request into smaller queries.

Next steps

• Recipes - complete examples that combine structured output with real tasks like entity extraction, content organization, and enrichment
• Streaming - combine streaming with structured output for real-time typed responses
• Multi-turn sessions - use structured output across a multi-turn conversation
• Create a response - review the basic request and response format

Jupyter notebook