The application of artificial intelligence (AI) is inarguably broad, and in the context of digital content, that can range from anything to restoring or reconstructing images and videos, to enhancing and upscaling content in real-time. For Polish studio CD Projekt Red (CDPR), it used AI for something a little less conventional: lip-syncing different languages to all the game’s characters.
To be precise, CDPR used the medium to help it lip-sync a total of 10 languages for all characters in-game with a voice. The AI technology that the studio used, by the way, comes from a Canada-based company called Jali. To break it down, Jali essentially created the model that enabled studios like CDPR to implement the different languages into the game. Without having to resort to the use of motion-capture technology for the face.
How JALI does this is, naturally, more technical than it sounds or looks, but if you’re interested in learning about how the company managed to do this, there’s also a technical video that shows you how it is done. Suffice to say, the AI technology does a seemingly decent job in accurately matching the words coming from a character directly with movement of their lips. This also includes the different facial expressions that a human face can pull off, including blinking, brow and neck movements, and emotions.
Cyberpunk 2077 is already on track on being CD Projekt Red’s (CDPR) most ambitious titles yet, to be played by gamers around the world and in their native languages. The studio’s attention to such detail isn’t common, but it should come as no surprise that this isn’t the only detail the studio has focused on, as evidenced by its obsession with getting the different vehicle sounds in-game.
