Oppenheimer’s legacy: How the national labs forged the age of hyperscale AI
We know a lot about Robert Oppenheimer and Los Alamos national laboratory from the movie.Many AI researchers today read avidly the classic work by Richard Rhodes, the making of the atomic bomb. But there's a stronger connection between Oppenheimers work and current AI research that's sometimes not noticed.
By the end of World War II early computing systems were coming online that had some capability for modeling nuclear reactions. Los Alamos was one of the labs at the forefront.
Fast-forward 55 years(vacuum tubes, transistors, integrated circuits, micro processors, multicore CPUs). Super computers played an increasing role in nuclear modeling because of the asci program in particular.
In the early 2000s was a growing concern that something radically different was needed to take computing for the next level. In fact, Denard scaling would hit a wall within a few short years.
A team at Los Alamos undertook an ambitious project to build a system from gaming processors to run science applications. This project eventuated in the roadrunner system, the first computer to break the petascale barrier.
On the other side of the country, Oak Ridge national laboratory, also involved in a Manhattan project in World War II, had launched several iterations of Jaguar, a conventional multicore system. There was likewise a feeling in 2009 of uncertainty as to how to bring computing to the next level. This led to a partnership with nvidia that would lead to the launch of the ORNL Titan system in 2012 (to read about the blood sweat and tears to prepare applications for this entirely new system, see here).
At the time some thought the approach was crazy, but today all the major US Department of energy labs have flagship GPU powered systems.
And In the same year of Titans launch, the landmark image net paper came out, based on Nvidia GPUs. Within a few short years Jensen Huong would make a big bet and go all in on GPU for AI. ORNL launched Summit in 2018 in frontier in 2022, both powered by GPU's.
Today the hunger for massive GPU clusters for AI training is insatiable.People are talking about million GPU clusters powered by nuclear or fusion reactors. Sizes like this are staggering and difficult to conceive. But it all has its roots in the pioneering efforts at Los Alamos in Oak Ridge, the heirs of Oppenheimer's mission, that Brought Us RoadRunner and Titan.
The post Oppenheimer's legacy: How the national labs forged the age of hyperscale AI first appeared on John D. Cook.