NVIDIA is a company that’s no stranger to generative AI, and probably has the most unique contribution to the field, with AI-generated NPCs for games. A lot of this includes generating voices for said non-player characters, and it looks like the company is taking the audio generative AI to the next level with a new AI model that it calls Fugatto.
As is usual, Fugatto (which stands for Foundational Generative Audio Transformer Opus 1) works by using text prompts, but you can also add in audio bases for it to work with. You can do the usual music creation based on said text prompt, or get it to add or remove instruments from a clip you provide. You can also get it to generate a voice line, and then modify it to having an angry or calm tone, for instance.
In the blog post, NVIDIA calls Fugatto the Swiss Army knife for sound, capable of even producing “sounds never heard before”. Which, while impressive, is still quite the claim. In a YouTube video, the company demos some of these generations which are perhaps unique permutations of sounds, though one can argue that they are combinations of sounds they’ve definitely heard before.
All that being said though, it certainly sounds more capable than other audio generation tools currently out there. Though it remains to be seen if it’s something that anyone can get to use, as the NVIDIA omitted the mention of availability in said blog post.
(Source: NVIDIA)
