And how do different translation strategies impact their cognitive experience? A study by Qihang Jiang and Stephen Doherty, titled An eye-tracking study on the cognitive processing, reception, and perception of swearword translation in films, sought to answer these questions by investigating the cognitive processing, reception, and perception of subtitled swearwords in audiovisual content using RealEye.
It provided a novel way to analyze viewers' engagement with subtitled swearwords, offering insights into how translation strategies - like omission, de-swearing, softening, and literal translation - shape cognitive effort and viewing experience.
The study's primary goal was to understand how viewers cognitively process subtitled swearwords presented under different translation strategies. Instead of using traditional in-lab eye-tracking systems, the researchers opted for RealEye, a webcam-based system that allowed participants to complete the experiment remotely from their own homes. This decision was partly driven by the need for flexibility during the COVID-19 pandemic, but it also allowed for a larger, more diverse participant pool.
Participants were exposed to four film clips. The clips were selected based on their use of swearwords, and each clip was presented with subtitles produced using one of several translation strategies. Participants were randomly assigned to one of five groups. Four of these groups were exposed to subtitles using different strategies for swearword translation, while a control group viewed the original English subtitles without translation.
Once the participants began the study, RealEye tracked their gaze as they watched each film clip. The software recorded how long participants focused on key elements of the screen, such as the subtitle area, areas containing swearwords, and the overall film scene. After watching each clip, participants completed a questionnaire on their experience, answering questions about their immersion, enjoyment, satisfaction, and perceived offensiveness of the subtitles.
By capturing gaze data and attention shifts, RealEye allowed researchers to assess cognitive load and processing effort in response to different swearword translation strategies. This remote, real-time tracking system provided a unique opportunity to study natural viewing behavior in a way that would have been more difficult to achieve with traditional lab-based methods.
To assess participants’ cognitive processing of subtitles, the study relied on several key eye-tracking metrics, each of which provided unique insights into how viewers engaged with subtitled swearwords. These metrics were calculated using gaze data collected by RealEye.
The study revealed several notable patterns in cognitive processing and audience perception of subtitled swearwords. These findings shed light on how translation strategies impact viewers’ attention and cognitive load.
One key finding was that participants spent more time processing subtitles containing swearwords than they did for subtitles without them. The Total Fixation Duration (TFD) was longer in subtitle areas where swearwords were present, indicating that swearwords required more cognitive effort to process. Additionally, participants’ Mean Fixation Duration (MFD) was longer for swearword subtitles, which implies that processing swearwords may be cognitively demanding, especially when non-literal translation strategies (like omission) were used.
Translation strategy also affected attention patterns. For instance, the omission strategy (where swearwords are removed entirely) produced a longer fixation duration in the general subtitle area. This may be because viewers noticed the absence of swearwords and spent more time searching for context or meaning. In contrast, literal translations (where swearwords were translated with their most direct equivalents) did not require as much cognitive effort, as evidenced by shorter fixation durations and fewer total fixations.
Interestingly, time to first fixation (TTFF) on subtitles was shorter when swearwords were present in the subtitle. This suggests that swearwords attract attention more quickly than non-swearword text, reflecting their emotional salience. Viewers are naturally drawn to swearwords, which could explain why these words capture attention faster than other parts of the subtitle.
Viewers' subjective responses were also measured. Through questionnaires, participants reported their satisfaction with the subtitles, their enjoyment of the film, their comprehension, and their perception of offensiveness. The results showed no significant differences in these measures across the five translation strategies. However, participants generally preferred literal translations of swearwords, indicating that viewers value fidelity in subtitle translation.
This study highlights how RealEye's remote eye-tracking technology was used to investigate how viewers process subtitles containing swearwords. By tracking gaze behavior, researchers identified key differences in cognitive processing across translation strategies. Viewers spent more time and attention on subtitles when swearwords were removed or sanitized, indicating that non-literal translation strategies may require more cognitive effort to process.
This study demonstrates how new eye-tracking tools like RealEye enable remote, large-scale research into audiovisual content. By offering precise, real-time gaze-tracking without the need for a physical lab, RealEye makes it possible to explore how viewers interact with subtitles in natural, everyday viewing environments. As the demand for accessible media grows, studies like this offer valuable guidance for translators, content creators, and streaming platforms looking to provide more effective and engaging subtitles.
Follow the steps below to start your own experiment with RealEye:
Ready to set up your own study? Visit RealEye Support page to learn more and keep us posted on your results! 🚀
.jpg)
