The example projects on this page demonstrate using the Twelve Labs Video Understanding Platform to enhance educational experiences. These projects address complex topics in education and pave the way for the future of AI-powered e-learning solutions.

NeuroLearn

Summary: NeuroLearn is an AI-based learning platform that integrates neurofeedback to personalize education by creating.

Description: The application analyzes brainwave data to select and emphasize parts of lectures that match the student's educational objectives. This approach keeps students engaged and improves their learning results. Developed by Akhil Dhavala, Ayush Khandelwal, Jackson Mowatt Gok, and Jacky Wong, NeuroLearn won 1st Place at the TEDAI Multimodal Hackathon (23 Labs) in San Francisco.

GitHub repo: NeuroLearn

Integration with Twelve Labs

NeuroLearn uses the Twelve Labs Video Understanding Platform to create custom snippets from lectures tailored to each student's knowledge, background, and interests.

The code below indexes videos from YouTube and for each video generates a summary and highlights:

    def __init__(self):
        self.api_url = os.getenv("TWELVE_LABS_BASE_URL")
        self.headers = {"x-api-key": os.getenv("TWELVE_LABS_API_KEY")}

    def index_youtube_video(self, index_id: str, youtube_url: str):
        task_url = f"{self.api_url}/tasks/external-provider"
        data = {
            "index_id": index_id,
            "url": youtube_url,
        }
        response = requests.post(task_url, headers=self.headers, json=data)
        # you can get the video ID from video_id
        return response.json()

    def highlight_video(self, body: HighlightVideoBody) -> HighlightVideoResponse:
        url = f"{self.api_url}/summarize"
        data = body.dict()
        response = requests.post(url, headers=self.headers, json=data)
        return HighlightVideoResponse(**response.json())

    def summarize(self, body: HighlightVideoBody) -> HighlightVideoResponse:
        url = f"{self.api_url}/summarize"
        data = body.dict()
        response = requests.post(url, headers=self.headers, json=data)
        return response.json()

42Labs - Personalized Podcast Builder

Summary: The 42 Labs Personalized Podcast Builder transforms how you learn on the go by synthesizing high-quality content from diverse sources like TED talks, podcasts, and articles into customized podcasts.

Description: The application analyzes your preferences, including topics of interest and proficiency levels, to curate content in various languages. You can interactively refine your learning experience by selecting subtopics and providing feedback, enhancing the application's ability to offer relevant material. The application not only overcomes language and learning barriers but also holds the potential to evolve into a tool for creating personalized educational videos. The application was developed by Shivani Poddar, David Salib, Varun Theja, and Everett Knag.

GitHub:

Frontend: TEDAI_2023.
Backend: 42labs-answer-to-everything.

Integration with Twelve Labs

The application uses the /classify/bulk endpoint to identify videos relevant to a specified topic:

def classify_videos(index_id, sub_topic, api_key):
    CLASSIFY_BULK_URL = f"{url}/classify/bulk"

    data =  {
    "options": ["conversation", "text_in_video"],
    "index_id": index_id,
    "classes": [{"name": sub_topic,
        "prompts": [
            sub_topic,
        ]}]
    }
    headers = {
        "accept": "application/json",
        "Content-Type": "application/json",
        "x-api-key": api_key
    }

    response = requests.post(CLASSIFY_BULK_URL, headers=headers, json=data)
    print (f'Status code: {response.status_code}')
    print(response.json())
    return response.json()["data"]

For each relevant video, the application invokes the /summarize endpoint to summarize videos as lists of chapters:

def summarize_video(video_id, api_key):
    payload = {
        "type": "chapter",
        "video_id": video_id,
    }
    headers = {
        "accept": "application/json",
        "x-api-key": api_key,
        "Content-Type": "application/json"
    }

    url = "https://api.twelvelabs.io/v1.2/summarize"
    response = requests.post(url, json=payload, headers=headers)

    print(response.text)
    chapter_summaries = [x["chapter_summary"] for x in response.json()["chapters"]]
    return chapter_summaries