It’s safe to say that, as a company, NVIDIA has been in the AI space for quite awhile. Earlier in the week, it announced ACE for Games, essentially tech that gives NPCs in games generative AI capabilities. In a separate announcement, the company has unveiled what it calls Neuralangelo. The name being a play on the famous Italian sculptor is fitting, as the this AI model’s main feature is creating 3D replica models from a 2D video.
The way this works is simply having the AI model take a look at a video that shows an object from several angles. Neuralangelo then uses information on shape, size and depth from the video to generate a rough 3D representation of the object. Then, it goes through a round of optimisation, sharpening details to have the 3D model look as close as possible to the original object. There’s a lot of analogies being drawn to an actual sculpture chiseling a physical sculpture to go with the theme that the name brings up.
NVIDIA says that Neuralangelo was built by adopting the tech of its own NeRF AI, announced last year, which worked using the same principle for the most part. The key difference is that the AI model from last year primarily involves images as opposed to video. With the new one, the tech is claimed to be so good that it can generate virtual objects from videos recorded by smartphones. If you’re looking to create entire scenes, this can be done within two hours.
Naturally, NVIDIA says that Neuralangelo can benefit anyone looking to “recreate the real world in the digital world”. Which would naturally be a handy tool for game development, be they of the more conventional kind, or even those of the VR variety.
(Source: NVIDIA [1], [2])
