The example projects on this page utilize the TwelveLabs Video Understanding Platform to transform education and training experiences. These projects demonstrate how multimodal AI can enhance learning outcomes and instructional effectiveness, exemplifying its transformative power in creating more engaging and personalized educational content.

Coach AI

Summary: Coach AI is an AI-powered sports analysis platform that aims to make professional coaching accessible to underrepresented athletes through personalized video feedback.

Description: The platform integrates the Pegasus 1.2 and Marengo 2.7 video understanding models with Gemini AI to deliver a comprehensive SWOT analysis of athletic performance. This analysis includes timestamped feedback and interactive video playback. Coach AI addresses a significant challenge faced by 44% of Canadian parents, who struggle to afford sports coaching, by providing professional-level analysis that is accessible through AI technology.

GitHub repo: KaranChawlaD/hackthe6ix

Devpost: Coach.AI

Dual Wield

Summary: Dual Wield transforms downtime during gaming into productive learning opportunities by automatically detecting moments when players are idle and providing bite-sized study content.

Description: The application extensively utilizes the TwelveLabs API for gameplay analysis. It invokes the client.analyze() method to detect periods of inactivity and the client.summarize() method for processing video content across multiple gaming titles, including Valorant, League of Legends, and CS:GO. This innovative productivity tool showcases advanced prompt engineering with timestamp extraction and represents a unique intersection of gaming and educational technology.

GitHub repo: WhyILived/DualWield

Devpost: Dual Wield

SocialSim

Summary: SocialSim is an AI-powered social skills training platform designed for professionals in healthcare, law enforcement, education, and social work.

Description: The application utilizes the TwelveLabs API for video indexing and summarization (using Pegasus 1.2), as well as embeddings (with Marengo 2.7). It features an advanced scoring system that integrates real-time avatar conversations through Tavus. This innovative training platform showcases deep integration with TwelveLabs and implements RAG (Retrieval-Augmented Generation), effectively addressing the essential skill development needs in high-stakes professional environments.

GitHub repo: hack-the-6ix-2025/SocialSim

Devpost: SocialSim

Tonalysis

Summary: Tonalysis provides real-time analysis and feedback on non-verbal communication using webcam and microphone inputs for speech therapy applications.

Description: The platform incorporates the TwelveLabs Pegasus 1.2 video understanding model for thorough video analysis, featuring sophisticated prompts that address speech clarity, body language, and actionable recommendations. It is further enhanced by MediaPipe facial landmark detection. This innovative communication coaching system meets the genuine needs of introverted and neurodivergent individuals by offering evidence-based feedback and professional presentation training.

GitHub repo: arnavmalhotra/Tonalysis-backend

Devpost: Tonalysis