Nvidia Hints At Replacing Rasterization and Ray Tracing With Full Neural Rendering

Mark Tyson writes via Tom's Hardware: A future version of [Deep Learning Super Sampling (DLSS) technology] is likely to include full neural rendering, hinted Bryan Catanzaro, a Nvidia VP of Applied Deep Learning Research. In a round table discussion organized by Digital Foundry (video), various video game industry experts talked about the future of AI in the business. During the discussion, Nvidia's Catanzaro raised a few eyebrows with his openness to predict some key features of a hypothetical "DLSS 10." [...] We've seen significant developments in Nvidia's DLSS technology over the years. First launched with the RTX 20-series GPUs, many wondered about the true value of technologies like the Tensor cores being included in gaming GPUs. The first ray tracing games, and the first version of DLSS, were of questionable merit. However, DLSS 2.X improved the tech and made it more useful, leading to it being more widely utilized -- and copied, first via FSR2 and later with XeSS. DLSS 3 debuted with the RTX 40-series graphics cards, adding Frame Generation technology. With 4x upscaling and frame generation, neural rendering potentially allows a game to only fully render 1/8 (12.5%) of the pixels. Most recently, DLSS 3.5 offered improved denoising algorithms for ray tracing games with the introduction of Ray Reconstruction technology. The above timeline raises questions about where Nvidia might go next with future versions of DLSS. And of course, "Deep Learning Super Sampling" no longer really applies, as the last two additions have targeted other aspects of rendering. Digital Foundry asked that question to the group: "Where do you see DLSS in the future? What other problem areas could machine learning tackle in a good way?" Bryan Catanzaro immediately brought up the topic of full neural rendering. This idea isn't quite as far out as it may seem. Catanzaro reminded the panel that, at the NeurIPS conference in 2018, Nvidia researchers showed an open-world demo of a world being rendered in real-time using a neural network. During that demo the UE4 game engine provided data about what objects were in a scene, where they were, and so on, and the neural rendering provided all the on-screen graphics. "DLSS 10 (in the far far future) is going to be a completely neural rendering system," Catanzaro added. The result will be "more immersive and more beautiful" games than most can imagine today.

Read more of this story at Slashdot.