Will Approximation Drive Post-Moore's Law HPC Gains?

Will Approximation Drive Post-Moore's Law HPC Gains?:

Hardware-based improvements are going to get more and more difficult," said Neil Thompson, an innovation scholar at MIT's Computer Science and Artificial Intelligence Lab (CSAIL). [...] Thompson, speaking at Supercomputing Frontiers Europe 2021, likely wasn't wrong: the proximate death of Moore's law has been a hot topic in the HPC community for a long time.

[...] Thompson opened with a graph of computing power utilized by the National Oceanic and Atmospheric Administration (NOAA) over time. Since the 1950s, there has been about a one trillion-fold increase in the amount of computing power being used in these models," he said. But there was a problem: tracking a weather forecasting metric called mean absolute error (When you make a prediction, how far off are you on that prediction?"), Thompson pointed out that you actually need exponentially more computing power to get that [improved] performance." Without those exponential gains in computing power, the steady gains in accuracy would slow, as well.

Enter, of course, Moore's law, and the flattening of CPU clock frequencies in the mid-2000s. But then we have this division, right?" Thompson said. We start getting into multicore chips, and we're starting to get computing power in that very specific way, which is not as useful unless you have that amount of parallelism." Separating out parallelism, he explained, progress had dramatically slowed. This might worry us if we want to, say, improve weather prediction at the same speed going forward," he said.

So in 2020, Thompson and others wrote a paper examining ways to improve performance over time in a post-Moore's law world. The authors landed on three main categories of promise: software-level improvements; algorithmic improvements; and new hardware architectures.

Read more of this story at SoylentNews.