> This page is for version v1.3 (default).
> For other versions, use one of these documentation indexes:
> - v1.3 (default): https://docs.twelvelabs.io/v1.3/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.twelvelabs.io/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.twelvelabs.io/_mcp/server.

# Vespa - Multivector video retrieval

> This integration combines the TwelveLabs Analyze and Embed APIs with Vespa to create an efficient solution for semantic video search.

![](/_fern-img/0382040250cff304739560e6b1b7b90912a1fccc127305d848253bc2885ca703.webp)

**Summary**: This integration combines TwelveLabs' Analyze and Embed APIs with [Vespa](https://vespa.ai/) to create an efficient solution for semantic video search. It captures rich video content as multimodal embeddings and utilizes Vespa's robust indexing and hybrid ranking capabilities to deliver precise and relevant search results.

**Description**: The process of performing a semantic video search using TwelveLabs and Vespa involves three main steps:

1. Create summaries and keywords using the [Analyze](/docs/guides/analyze-videos-and-images) API.
2. Create multimodal embeddings for your video content using the [Embed API](/docs/guides/create-embeddings).
3. Deploy a Vespa application to index these embeddings.
4. Use the embeddings to perform vector searches with hybrid ranking.
5. Review the results.

**Code explanation**: Our blog post, [Multivector Video Retrieval with TwelveLabs and Vespa](https://www.twelvelabs.io/blog/twelve-labs-and-vespa), guides you through the process of building a semantic video search solution.

**Colab Notebook**: [video\_search\_twelvelabs\_cloud](https://drive.google.com/file/d/1CKX476xAF_JMr16Xg5XeDW1fAsYsLtfe/view?usp=sharing)

# Integration with TwelveLabs

This section shows how to use the Generate and Embed APIs for creating video embeddings and metadata, which facilitate the efficient retrieval of relevant video segments.

## Generate summaries and keywords

The code below uploads videos to an index and monitors the processing status:

**`Python`**

```python Python
def on_task_update(task: EmbeddingsTask):
    print(f"  Status={task.status}")

for video_url in VIDEO_URLs:
    task = client.task.create(index_id=index.id, url=video_url, language="en")
    status = task.wait_for_done(sleep_interval=10, callback=on_task_update)
    if task.status != "ready":
        raise RuntimeError(f"Indexing failed with status {task.status}")

```

Once the videos are processed, you can generate rich metadata using the `/summarize` and `/analyze` endpoints. This code creates summaries and lists of keywords for each video to enhance search capabilities:

**`Python`**

```python Python
summaries = []
keywords_array = []
titles = [
    "Mr. Bean the Animated Series Holiday for Teddy",
    "Twas the night before Christmas",
    "Hide and Seek with Giant Jenny",
]

videos = client.index.video.list(index_id)
for video in videos:
    # Generate summary
    res = client.generate.summarize(
        video_id=video.id,
        type="summary",
        prompt="Generate an abstract of the video serving as metadata on the video, up to five sentences."
    )
    summaries.append(res.summary)
    
    # Generate keywords
    keywords = client.generate.text(
        video_id=video.id,
        prompt="Based on this video, I want to generate five keywords for SEO. Provide just the keywords as a comma delimited list."
    )
    keywords_array.append(keywords.data)
```

## Create video embeddings

The code below creates multimodal embeddings for each video. These embeddings capture the temporal and contextual nuances of the video content:

**`Python`**

```python Python
task_ids = []

for url in VIDEO_URLs:
    task = client.embed.task.create(model_name="Marengo-retrieval-2.7", video_url=url)
    task_ids.append(str(task.id))
    status = task.wait_for_done(sleep_interval=10, callback=on_task_update)
    if task.status != "ready":
        raise RuntimeError(f"Embedding failed with status {task.status}")

tasks = []
for task_id in task_ids:
    task = client.embed.task.retrieve(task_id)
    tasks.append(task)
```

See the [Video embeddings](/docs/guides/create-embeddings/at-scale/video) section for details.

## Create text emeddings

The code below generates an embedding for your text query:

**`Python`**

```python Python
client = TwelveLabs(api_key=TL_API_KEY)
user_query = "Santa Claus on his sleigh"

# Generate embedding for the query
res = client.embed.create(
    model_name="Marengo-retrieval-2.7",
    text=user_query,
)

print("Created a text embedding")
print(f" Model: {res.model_name}")
if res.text_embedding is not None and res.text_embedding.segments is not None:
    q_embedding = res.text_embedding.segments[0].embeddings_float
    print(f" Embedding Dimension: {len(q_embedding)}")
    print(f" Sample 5 values from array: {q_embedding[:5]}")
```

See the [Embed a query](/docs/guides/create-embeddings/query) section for details.

## Perform hybrid searches

The code below uses Vespa's approximate nearest neighbor (ANN) search capabilities to combine lexical search (BM25) with vector similarity ranking. The query retrieves the top hit based on hybrid ranking:

**`Python`**

```python Python
with app.syncio(connections=1) as session:
    response: VespaQueryResponse = session.query(
        yql="select * from videos where userQuery() OR ({targetHits:100}nearestNeighbor(embeddings,q))",
        query=user_query,
        ranking="hybrid",
        hits=1,
        body={"input.query(q)": q_embedding},
    )
    assert response.is_successful()

# Print the top hit
for hit in response.hits:
    print(json.dumps(hit, indent=4))

# Get full response JSON
response.get_json()
```

# Next steps

After reading this page, you have the following options:

* **Customize and use the example**: Use the [video\_search\_twelvelabs\_cloud](https://drive.google.com/file/d/1CKX476xAF_JMr16Xg5XeDW1fAsYsLtfe/view?usp=sharing) notebook to understand how the integration works. You can make changes and add functionalities to suit your specific use case.
* **Explore further**: Try the [applications built by the community](/docs/resources/from-the-community) or our [sample applications](/docs/resources/sample-applications) to get more insights into the TwelveLabs Video Understanding Platform's diverse capabilities and learn more about integrating the platform into your applications.