Summary: This integration combines the Pegasus 1.2 video understanding model with VideoDB’s real-time streaming infrastructure to create intelligent video monitoring systems. It transforms live video feeds into actionable alerts, enabling proactive responses to critical events such as security breaches, safety hazards, and emergencies.

Description: Integrating Pegasus with VideoDB addresses key challenges in real-time video monitoring, such as processing live streams at scale, detecting specific events with high accuracy, and delivering instant notifications. The process involves the following main steps:

1. Connect live video streams using VideoDB’s RTStream infrastructure.
2. Index video scenes in real-time using TwelveLabs’ Pegasus 1.2 model for intelligent understanding.
3. Define custom events with natural language prompts to specify what to monitor.
4. Configure webhook alerts to receive immediate notifications when events occur.
5. Access stream segments for instant review through generated HLS manifests.

Step-by-step guide: Our blog post, Unlock Real-Time Video Understanding with VideoDB and TwelveLabs, guides you through setting up real-time monitoring systems, configuring scene indexing, and implementing two practical use cases: flash flood detection and baby crib monitoring.

GitHub: VideoDB + TwelveLabs Integration

Integration with TwelveLabs

This section describes how you can use Pegasus 1.2 within VideoDB to create real-time video monitoring applications. The integration provides seamless access to advanced video understanding without additional API keys or account setup.

RTStream connection

Description: Establishes live video stream connections from RTSP sources, IP cameras, or other streaming protocols.

Inputs: Stream name, RTSP URL or video source, optional connection parameters.

Output: RTStream object with connection status, stream metadata, and unique stream identifier for monitoring operations.

Scene indexing

Description: Processes live video streams using Pegasus 1.2 to generate intelligent scene descriptions and enable event detection.

Inputs: RTStream object, extraction type (time-based or scene-based), extraction configuration (time intervals, frame counts), custom prompt for guidance, and model specification.

Output: Scene index with unique identifier, processing status, and AI-generated descriptions of video content segments.

Event definition

Description: Creates custom event types using natural language prompts to specify what conditions trigger alerts.

Inputs: Event prompt describing target conditions, event label for identification, optional confidence thresholds. Confidence is a value between 0 and 1 that shows how certain the VideoDB platform is about the decision to trigger the alert, based on the description of the video segment.

Output: Event definition with unique event ID, configured detection parameters, and activation status for monitoring workflows.

Alert configuration

Description: Connects event definitions to webhook endpoints for real-time notification delivery when events occur.

Inputs: Event ID from defined events, callback URL for webhook delivery, optional alert parameters and filters.

Output: Alert configuration with unique alert ID, webhook settings, and delivery status for notification management.

Stream segment access

Description: Generates time-limited HLS manifest URLs for immediate review of detected events or specific time ranges.

Inputs: Stream identifier, start time, end time, optional playback parameters.

Output: Short-lived HLS manifest URL with embedded player compatibility and configurable validity period.

Webhook payload processing

Description: Receives structured alert data containing event details, confidence scores, explanations, and stream access links.

Inputs: Webhook endpoint configuration and payload parsing requirements.

Output: Event metadata including label, confidence score, timestamp, explanation text, and stream segment URL for immediate review.

Next steps

After reading this page, you have the following options:

• Deploy your first monitor: Start with any of the examples to understand stream connection, scene indexing, and alert configuration, then adapt the pattern for your specific monitoring needs.
• Implement custom events: Create specialized event definitions using natural language prompts, configure confidence thresholds, and test detection accuracy with your video content.
• Optimize for production: Configure processing settings for multiple streams, implement webhook retry logic, and monitor system performance metrics for large-scale deployments.