Generate social media posts for your videos

The Generate API suite generates open-ended text from videos. You can customize the generated text using prompts. Unlike conventional models limited to unimodal interpretations that, for example, summarize videos relying solely on transcriptions, the Generate API suite uses a multimodal approach that analyzes the whole context of a video, including visuals, sounds, spoken words, and texts and their relationship with one another. This method ensures a holistic understanding of your videos, capturing nuances that an unimodal interpretation might miss. Consequently, utilizing its video-to-text generative capabilities, the platform translates this understanding into your desired textual representation.

Simplify cross-platform video promotion

Creating unique posts for each social media platform can be time-consuming and challenging. The "Generate social media posts for your videos" application solves this problem. You can tailor the formats for specific social media platforms, such as Instagram, Facebook, X, etc.

Overview

This guide is designed for developers with intermediate knowledge of Node.js and React. After completing this guide, you will learn to integrate the Twelve Labs Video Understanding Platform's text generation functionalities into your web application.

The application consists of a Node.js backend and a React frontend:

  • The backend is an intermediary between the Twelve Labs Video Understanding platform and the frontend. It uses the Express web framework.

  • The frontend is structured as a typical React application, and it uses React Query, a library for fetching, caching, and updating data.

Prerequisites

  • You're familiar with the concepts described on the Platform overview page.
  • Before you begin, sign up for a free account, or if you already have one, sign in.
  • You’ve already created an index, and the Pegasus video understanding engine is enabled for this index.
  • You have a basic understanding of React (components, state, and props) and Node.js.
  • You're familiar with using command line interfaces and GitHub.
  • Node.js is installed on your system.
  • Nodemon is installed on your system.

Run the application

  1. Clone the Generate Social Posts for Your Video GitHub repository to your computer.

  2. Retrieve your API key by going to the Dashboard page and selecting the Copy icon at the right of your API key.

  3. Retrieve the base URL and the latest version of the API. Using these values, construct the API URL. For instructions, see the Call an endpoint page.

  4. Create a file named .env with the key-value pairs shown below in the root directory of your project, replacing the placeholders surrounded by <> with your values:

    REACT_APP_API_URL=<BASE_URL>/<VERSION>
    REACT_APP_API_KEY=<YOUR_API_KEY>
    REACT_APP_SERVER_URL=<SERVER_URL>
    REACT_APP_PORT_NUMBER=<SERVER_PORT>
    REACT_APP_INDEX_ID=<YOUR_INDEX_ID>
    

    Example:

    REACT_APP_API_URL=https://api.twelvelabs.io/v1.2
    REACT_APP_API_KEY=tlk_1KM4EKT25JWQV722BNZZ30GAD1JA
    REACT_APP_SERVER_URL=http://localhost
    REACT_APP_PORT_NUMBER=4000
    REACT_APP_INDEX_ID=660a9fc12ae59d128f133713
    
  5. To install the dependencies, open a terminal window, navigate to the root directory of your project, and run the following command:

    npm install
    
  6. To start the backend, enter the following command:

    nodemon server.js
    

    You should see an output similar to this:

    [nodemon] 3.0.1
    [nodemon] to restart at any time, enter `rs`
    [nodemon] watching path(s): *.*
    [nodemon] watching extensions: js,mjs,cjs,json
    [nodemon] starting `node server.js`
    
  7. To start the frontend, open a new terminal window, navigate to the root directory of your project, and enter the following command:

    npm start
    

    This will open a new browser window displaying the application:

  8. Typically, using the application is a two-step process:

    1. Upload a video.
    2. Generate a social media post.

Integration with Twelve Labs

This section provides a concise overview of integrating the "Generate social media posts for your videos" application with the Twelve Labs Video Understanding Platform.

List the videos within an index

To list the videos within an index, the application invokes the GET method of the /indexes/{index-id}/videos endpoint with the following parameters and values:

Parameter and valuePurpose
page_limit = request.query.page_limitThe platform returns the results one page at a time, with a default limit of 10 videos on each page. This parameter overrides the default value, specifying the desired number of videos to return on each page.

Below is the code for reference:

app.get("/indexes/:indexId/videos", async (request, response, next) => {
  const params = {
    page_limit: request.query.page_limit,
  };

  try {
    const options = {
      method: "GET",
      url: `${API_BASE_URL}/indexes/${request.params.indexId}/videos`,
      headers: { ...HEADERS },
      data: { params },
    };
    const apiResponse = await axios.request(options);
    response.json(apiResponse.data);
  } catch (error) {
    const status = error.response?.status || 500;
    const message = error.response?.data?.message || "Error Getting Videos";
    return next({ status, message });
  }
});

For a description of each field in the request and response, see the API Reference > List videos page.

Retrieve information about a video

To retrieve information about a video uploaded to the platform, the application invokes GET method of the /indexes/{index-id}/videos/{video-id} endpoint.

Below is the code for reference:

app.get(
  "/indexes/:indexId/videos/:videoId",
  async (request, response, next) => {
    const indexId = request.params.indexId;
    const videoId = request.params.videoId;

    try {
      const options = {
        method: "GET",
        url: `${API_BASE_URL}/indexes/${indexId}/videos/${videoId}`,
        headers: { ...HEADERS },
      };
      const apiResponse = await axios.request(options);
      response.json(apiResponse.data);
    } catch (error) {
      const status = error.response?.status || 500;
      const message = error.response?.data?.message || "Error Getting a Video";
      return next({ status, message });
    }
  }
);

For a description of each field in the request and response, see the API Reference > Retrieve video information page.

Upload a video

To upload videos, the application invokes the POST method of the /task endpoint with the following parameters and values:

Parameter and valuePurpose
data = formDataRepresents a set of key-value pairs containing, among other information, the unique identifier of the video to which you wish to upload your video and the content of the video file stored as raw binary data.

Below is the code for reference:

app.post(
  "/index",
  upload.single("video_file"),
  async (request, response, next) => {
    const formData = new FormData();

    // Append data from request.body
    Object.entries(request.body).forEach(([key, value]) => {
      formData.append(key, value);
    });

    const blob = new Blob([request.file.buffer], {
      type: request.file.mimetype,
    });

    formData.append("video_file", blob, request.file.originalname);

    const options = {
      method: "POST",
      url: `${API_BASE_URL}/tasks`,
      headers: {
        "x-api-key": TWELVE_LABS_API_KEY,
        accept: "application/json",
        "Content-Type":
          "multipart/form-data; boundary=---011000010111000001101001",
      },
      data: formData,
    };
    try {
      const apiResponse = await axios.request(options);
      response.json(apiResponse.data);
    } catch (error) {
      const status = error.response?.status || 500;
      const message =
        error.response?.data?.message || "Error indexing a YouTube Video";
      return next({ status, message });
    }
  }
);

For a description of each field in the request and response, see the API Reference > Create a video indexing task page.

Check the status of a video indexing task

A video indexing task represents a request to upload and index a video. To check the status of a video indexing task, the application invokes the GET method of the /tasks/{task_id} endpoint.

Below is the code for reference:

app.get("/tasks/:taskId", async (request, response, next) => {
  const taskId = request.params.taskId;

  try {
    const options = {
      method: "GET",
      url: `${API_BASE_URL}/tasks/${taskId}`,
      headers: { ...HEADERS },
    };
    const apiResponse = await axios.request(options);
    response.json(apiResponse.data);
  } catch (error) {
    const status = error.response?.status || 500;
    const message = error.response?.data?.message || "Error getting a task";
    return next({ status, message });
  }
});

For a description of each field in the request and response, see the API Reference > Retrieve a video indexing task page.

Generate social media posts

To generate social media posts based on a video, the application invokes the POST method of the /generate endpoint with the following parameters and values:

Parameter and valuePurpose
prompt = request.body.dataA prompt that guides the model on the desired format or content. The prompts for each platform are defined in the handleSubmit function
video_id = videoIdThe unique identifier of the video.
temperature = 0.3Controls the randomness of the text output generated by the model. The default value is 0.7. A higher value generates more creative text, while a lower value produces more deterministic text output.

Below is the code for reference:

app.post("/videos/:videoId/generate", async (request, response, next) => {
  const videoId = request.params.videoId;
  let prompt = request.body.data;
  try {
    const options = {
      method: "POST",
      url: `${API_BASE_URL}/generate`,
      headers: { ...HEADERS, accept: "application/json" },
      data: { ...prompt, video_id: videoId, temperature: 0.3 },
    };
    const apiResponse = await axios.request(options);
    response.json(apiResponse.data);
  } catch (error) {
    const status = error.response?.status || 500;
    const message = error.response?.data?.message || "Error Generating Text";
    return next({ status, message });
  }
});

For a description of each field in the request and response, see the API Reference > Generate open-ended texts page.

Next steps

After you have completed this tutorial, you have several options:

  • Try out the deployed application: Visit the live demo to interact with the application.
  • Use the application as-is: Inspect the source code to understand the platform's features and start using the application immediately.
  • Customize and enhance the application: This application is a foundation for various projects. Modify the code to meet your specific needs, from adding new features to integrating more endpoints.
  • Explore further: Try the applications built by the community or the partner integrations to get more insights into the Twelve Labs Video Understanding Platform's diverse capabilities and learn more about integrating the platform into your applications.