On March 23rd, YouTube launched its improved automatic captioning system that now includes sound effects. This first release of their new system includes the following sounds: [APPLAUSE], [MUSIC] and [LAUGHTER].
This is a fabulous announcement! Not only the transcription of the spoken word is important for people with hearing impairment to have complete information of audiovisual material. Sound effects are also an important element of the information.
For example, if you can’t hear and you are watching a talent show video on YouTube, you can miss relevant elements of the program if the camera does not show the excited applause of the audience to a performance.
Including sound effects on captions is not something new; the novelty is to have them detected and included automatically in the captions with no human intervention. Technically the automatic identification of sounds in a great variety of situations is a very complex job. YouTube and the teams Sound Understanding and Accessibility, from Google, have developed a technology that can identify sound effects. This first release includes only 3 sounds, but in the future they expect to include sounds like ringing, barking, knocking, piano music, etc.
Go to the following video, click on the CC button and see the sound effects captioning system in action.
This is, without a question, a great achievement in improved access to information for people with hearing impairment. It is also an achievement that will find many uses in other areas.
