Recipes

Extract entities

List all entities - people, places, objects, brands, and concepts - found across your video collection. Returns each entity with its type, frequency, and the videos it appears in.

Prerequisites

  • Complete the Quickstart to create a knowledge store with at least one item in ready status.
  • Read Create a response to understand the request and response format.

When to use this

  • Building a content index or searchable catalog
  • Understanding who and what appears across your videos
  • Feeding entity lists to downstream classification or tagging systems

Extract and categorize entities

Ask Jockey to list every distinct entity across all videos. The schema captures the entity name, type, how often it appears, and which videos contain it.

1import json
2import requests
3
4API_KEY = "YOUR_API_KEY"
5BASE_URL = "https://api.twelvelabs.io/v1.3"
6HEADERS = {"x-api-key": API_KEY, "Content-Type": "application/json"}
7STORE_ID = "your_knowledge_store_id"
8
9entity_schema = {
10    "type": "object",
11    "properties": {
12        "entities": {
13            "type": "array",
14            "items": {
15                "type": "object",
16                "properties": {
17                    "name": {"type": "string"},
18                    "type": {"type": "string"},
19                    "frequency": {"type": "string"},
20                    "appears_in": {"type": "array", "items": {"type": "string"}}
21                }
22            }
23        },
24        "entity_count": {"type": "integer"}
25    }
26}
27
28response = requests.post(
29    f"{BASE_URL}/responses",
30    headers=HEADERS,
31    json={
32        "model": "jockey1.0",
33        "input": [
34            {"type": "message", "role": "user", "content": "List every distinct entity across all videos - people, places, objects, brands, and concepts. Include how frequently each appears."}
35        ],
36        "tools": [
37            {"type": "knowledge_store", "knowledge_store_id": STORE_ID}
38        ],
39        "text": {"format": "json_schema", "json_schema": entity_schema}
40    }
41)
42
43result = response.json()
44for output in result["output"]:
45    if output["type"] == "message":
46        for content in output["content"]:
47            data = json.loads(content["text"])
48            print(f"Found {data['entity_count']} entities:\n")
49            for entity in data["entities"]:
50                print(f"  [{entity['type']}] {entity['name']} - {entity['frequency']}")
51                if entity.get("appears_in"):
52                    print(f"    Videos: {', '.join(entity['appears_in'])}")

Variations

  • Filter by type: “List only the people who appear in these videos”
  • Cross-video presence: “Which entities appear in more than one video?”
  • With relationships: “List entities and how they relate to each other”

Jupyter notebook

Download the notebook to run this recipe interactively.

See also