Skip to content
English
Create Support Ticket
  • There are no suggestions because the search field is empty.

What is the Subtitle Detection?

Locate and Extract Subtitles with Advanced Heatmap Visualization

Module Description:

The Subtitle Detection module automatically detects and extracts subtitles from a video or an image. It generates a heatmap to pinpoint the specific region where the subtitles appear. This helps in recognizing and analyzing subtitle content, positioning, and patterns, which is crucial for video indexing, accessibility, and content analysis.

How does it work?

  1. Select the Media File: Choose the media file you want to analyze.
  2. Activate the Subtitle Detection Module: In the left column, select the "Subtitle Detection" module.
  3. Define the Model & Parameters: Choose the model for analysis from the available options, set the parameters, and click the yellow "Add Module" button.
  4. Start the Analysis: You can either add more modules or begin the analysis immediately by clicking "Start Analysis"

What Parameters are available?

  • Text Similarity Threshold (0-1):
    This threshold helps compare the text across multiple frames. A lower value will increase sensitivity and detect text duplicates, while a higher value will ensure only unique text snippets are considered.
  • Text Region Buffer Threshold (0-1):
    This threshold controls the extraction of text outside the main text region. By adjusting it, you can capture content that appears beyond the typical subtitle box, such as extra annotations.
  • Dump Heatmap (Checkbox):
    If this parameter is enabled, the system will generate a heatmap as an artifact, which visually shows where the subtitle regions are located within the video or image.

Displaying the Results:

Timeline:

The timeline, located below the player, displays the entire video runtime and the results from each module as gray bars.
  • By clicking on any of the grey result bars, you will see details such as:
    • Label Name
    • Confidence Value (0%-100%)
    • Timecode (TC)
    • Exact frame numbers
    • Runtime/Duration
  • Clicking on a result moves the playhead to the beginning of that result.
  • These results are identical to those provided by the API, but in a more user-friendly, graphical format. If there are multiple results, use your mouse wheel to scroll through the timeline.

Module Section

On the right side of the player, you’ll see a section with detailed results for each module used in the analysis. Clicking on the module name opens a dropdown with specific parameters, useful for troubleshooting or viewing metadata.

Subtitle List:

On the right side, you will see the list of detected subtitles. By clicking on a specific subtitle in this list, the video player will jump to the exact timecode where that subtitle appears in the video.
Each subtitle is shown in chronological order, with details such as:
      • Confidence Level: The system provides a confidence score (e.g., "Confidence: 1.00") for each subtitle, indicating how confident the system is in its accuracy.
      • Exact Subtitles: The detected text is displayed, showing both spoken words and any text found on-screen, such as captions or labels.
      • Language: The detected language of the subtitles is shown at the top (e.g., "Main Language: English").

Artifacts Section:

  • Heatmap Artifact:
    At the bottom of the viewer, you will find the Artifacts section, which contains the heatmap generated during the analysis (shown as "heatmap.png"). The heatmap visually represents the subtitle regions in the video, helping you understand where subtitles are most concentrated. You can download this artifact by clicking the download icon.
    • Expiration: Next to the artifact, an expiration time is listed (e.g., "Expiry: in 7 days"), indicating how long the heatmap file will be available for download.

Heatmap:
Here is an example of a heatmap, for more details see the description below.
The heatmap uses the actual pixel sizes on the Y and X axis for location the subtitels in space.

subtitle-heatmap

Heatmap Colors:

The colors represent the intensity of subtitle detection. The warmer the color (yellow or red), the more frequently the subtitles appear in that specific region. Cooler colors like blue or purple indicate areas where subtitles were detected less frequently or not at all.

Location:

In the example, subtitles are concentrated near the bottom center of the image, which is typical in most videos or films where subtitles are placed.

Intensity:

The heatmap shows dense subtitle activity in the lower-middle section, suggesting that most subtitles appeared in this region throughout the video.

This visualization helps you understand where subtitles are located and allows you to extract and work with that data effectively.

Practical Use of Subtitle Detection:

The Subtitle Detection module can be particularly useful for:

  • Video indexing: Automatically identifying the subtitle region makes it easier to retrieve specific information from videos.
  • Content analysis: By recognizing subtitle patterns, the module helps analyze dialogue structure and timing, making it valuable for media companies, streaming platforms, and content curators.
  • Accessibility improvements: The extracted subtitles can be processed to ensure better accessibility for viewers with hearing impairments.