Comparing the impact of professional and automatic closed captions on video-watching experience

Closed captions (CC) have gained popularity in recent years. While professional subtitles are commonly used, there has been a trend toward using automatic speech recognition technology to generate closed captions. However, there is limited research on the impact of these subtitles on viewers. To address this gap, we conducted an experiment to investigate how professional and automatic CC influence people’s video-watching experiences. We developed to measure viewers’ narrative engagement. We found that people showed better narrative understanding with the professional compared to the automatic one, while no significant difference in their emotional engagement and narrative presence. These findings suggest that automatic CC has the potential to be a rival professional one, but more research is needed to improve its effectiveness. Additionally, it provides insight into the motivations of viewers without hearing impairments or language translation needs, who use CC to enhance their video-watching experiences.

Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems