Google Cloud innovator and responsible AI researcher Denis Krasnikov presented a new technological solution in the field of artificial intelligence at the Society of Motion Picture and Television Engineers (SMPTE) summit. This innovation offers a unique approach to video translation and dubbing, prioritizing emotional accuracy to provide a more culturally adaptive and resonant viewing experience. The presentation took place at the SMPTE summit, the Hollywood media industry’s premier annual event, organized by this leading global standards organization.
SMPTE plays an important role in developing technical approaches and standards that ensure consistency, quality, and innovation in film, television, and digital media, including shaping Hollywood’s adaptation to technological advances such as artificial intelligence.
Denis Krasnikov touched upon the gap in current AI-based translations, which are currently limited mainly to content where emotional nuances are not so important. The new technological solution determines not only the meaning of phrases, but also intonation, identifies emotions on the screen, and collects this data into a new database for synthesizing emotions in new languages with additional parameters.
This allows for emotionally accurate translations that preserve the intent and mood of the content creator while making it understandable and engaging to people across cultural divides. “Emotions are expressed differently across cultures and generations, so capturing and conveying these feelings is important for authentic storytelling. Moreover, it is not a static but a dynamic characteristic of such diverse patterns of behavior in different countries,” said Denis Krasnikov.
“Our technology allows translated content to deeply resonate with diverse audiences around the world with different interpretations of emotions. With the advancement of technology, films localized by artificial intelligence will soon become even more captivating than the best versions translated by human actors today.”
During the SMPTE summit and the subsequent Production Summit, Krasnikov presented a demonstration of emotionally translated videos, emphasizing the scalability of the technology across different languages. This new solution complements the technologies created by the team of Alexander Konovalov, Evgeny von Rubinberg, and Denis Krasnikov, and their previous achievements, including Vidby, an AI-powered video localization tool recognized as YouTube’s recommended technology for multilingual audio tracks, as well as a real-time voice translation solution for Google Meet and Zoom calls.
In addition to his academic and business achievements, Denis Krasnikov announced a humanitarian initiative at the Star Independent Film Festival, organized by renowned Miami-based producer Elena Yaremchuk. This initiative provides free real-time translation services on Google Meet and Zoom to non-profit organizations supporting humanitarian efforts in Ukraine, demonstrating the team’s commitment to addressing some of the most pressing issues facing global society today.
