Cyberpunk 2077 uses advanced AI for its lip-sync performances, localized in 10 languages

Cyberpunk 2077 Street Kid Screenshot
Cyberpunk 2077 Street Kid Screenshot (Image credit: CD Projekt Red)

What you need to know

  • Cyberpunk 2077 is a much-hyped open world RPG from CD Projekt RED, and promises to be huge.
  • Its size means there will be an absurd number of characters, dialogue, languages, and more that the studio needs to animate.
  • With the pandemic, traditional facial motion capture became an impossibility, so CD Projekt RED looked for other solutions.
  • They found JALI, an innovative AI that can fully animate facial expressions and vocals, with little-to-no animator input.

Cyberpunk 2077 is one of the most anticipated games of the year, and is setting out to deliver by being one of the biggest games of the year. However, this means Cyberpunk 2077 will have thousands of lines of dialogues, dozens of unique voiced characters, and a wide range of personalities and speech types. More than that, CD Projekt RED reportedly set out with the goal to surpass the Witcher 3 with lip-sync quality, as well as support at least 10 languages with native localization. Traditional facial motion capture wasn't a good solution, so they looked for an alternative patch. Enter: JALI, an AI program that can procedurally analyze and animate lip-sync and facial animations.

JALI is an "animator-centric viseme model for expressive lip synchronization," meaning it breaks down facial animation into base components including speech, speech style, eyes, brows, emotion, and head and neck motion. It can handle lip-sync as well as full emotional expression, including the movement of various parts of the face, making performances feel authentic and powerful without ever needing to be animated. It looks incredibly impressive, and will be used by every single voiced character in the entire world.

Even better, JALI can analyze and understand multiple languages and dictations, so Cyberpunk 2077 will have native language localization for 10 languages, including French, Spanish, Japanese, German, and more, just by utilizing the innovated power of JALI. The video above is a little long, but it includes a ton of fascinating information about the tech being used here, as well as examples to show it off. If JALI is consistent across the game, lip-sync in Cyberpunk 2077 should be very impressive, and rightfully blend into the background.

We're officially in the month-long lead-up to Cyberpunk 2077, with the game having recently gone gold before its release on November 19, 2020.

Zachary Boddy
Zachary Boddy is the Minecraft Expert and a News Writer for Windows Central, Android Central, and iMore. They have been gaming and writing for most of their life, and have been freelancing for Android Central and its sister sites since 2019, with a focus on Xbox and PC gaming. You can find Zachary on Twitter [@BoddyZachary](https://twitter.com/BoddyZachary).