Article 79TNG Nvidia claims its Vera CPU can gain up to 39% performance boost via software optimization at SC26 — architectural fine-tuning is needed to fully unleash custom Olympus cores

Nvidia claims its Vera CPU can gain up to 39% performance boost via software optimization at SC26 — architectural fine-tuning is needed to fully unleash custom Olympus cores

by
ashilov@gmail.com (Anton Shilov)
from Latest from Tom's Hardware on (#79TNG)

Nvidia's Vera CPU, designed as both a standalone data-center part and a host for Nvidia's Vera Rubin NVL72 servers, offers impressive performance thanks to substantially higher execution resources per core than its predecessor, Grace. However, Nvidia admits at Supercomputing 2026 (SC26) that existing software cannot always take full advantage of Vera's resources without adaptation for the architecture.

The company shared that architecture-specific software optimizations can boost Vera's performance by nearly 39% in certain computationally intensive workloads. But despite offering unbeatable single-thread performance, the CPU is not positioned for general-purpose workloads.

Get the Tom's Hardware App

wy2jRCvahmJNbu4bh6m86M-1920-80.png

(Image credit: Tom's Hardware)

Get the full Tom's Hardware experience on your device with our brand-new app. Bookmark articles, use our smart search feature, personalize your feed, and get push notifications so you never miss a beat.

Unlike Grace, which relies on off-the-shelf Neoverse V2 CPU cores developed by Arm, Vera uses Nvidia's proprietary Olympus cores specifically designed to maximize per-core performance as well as to ensure high aggregate throughput. To that end, Nvidia equipped Olympus with a wide front end that features a 10-wide decoder, a neural branch predictor, a deep out-of-order execution engine, six vector execution units, four load pipelines, two store pipelines, all backed with large private caches and advanced prefetch mechanisms.

tNinGuat4fBcaghWKj5mcP-1920-80.png

(Image credit: Nvidia)

In fact, Nvidia deliberately opted for 88 high-performance general-purpose cores with support for its proprietary spatial multithreading technology (176 threads in total) rather than maximizing the number of cores per socket, as the maximum possible single-thread performance was one of the key design goals for Vera.

"CPU design is really just a game of trade-offs," said Hannah Coutand, who leads Vera CPU marketing at Nvidia, in an interview with Tom's Hardware Premium. "You could maximize for a lot of cores, but literally when you maximize the die for lots of cores, you do not actually have space to maximize for IPC, like instructions per clock."

Although Nvidia primarily positions Vera as a CPU for agentic AI and other AI-related workloads, the company originally conceived its Olympus microarchitecture with data processing applications in mind. As it turned out, the key attributes of Vera - high per-core performance, high memory bandwidth, and low memory access latency - are precisely what agentic AI workloads need as well, according to Coutand.

"We are very clear in our strategy with Vera and our positioning with Vera that it is not a general-purpose CPU," Coutand said. " If you want a web server for your cloud data center, there are probably other choices that are better suited for that than Vera. [Vera is made for] workloads [...] that require that high per-core speed, high memory bandwidth, low latency. [...] It has great fungibility for lots of workloads as long as those workloads value the things that Vera is good for."

Software optimizations make good things better

To assess the computational capabilities of Nvidia's custom Olympus microarchitecture and its advantages over Neoverse V2, Nvidia's researchers used a double-precision general matrix multiplication (DGEMM) workload based on an assembly kernel from OpenBLAS, an open-source linear algebra library. While OpenBLAS features a kernel tailored for the Arm Neoverse V2 cores that power Nvidia's Grace processor, it lacks one specifically designed for Olympus.

To examine the efficiency of the two microarchitectures, the researchers used a standalone benchmark called miniKernelGEMM with a 128 * 128 matrix multiplication problem that fits into the L1 cache of both processors.

WzHRg3hA8ru96QppKsfFfP-1920-80.png

(Image credit: Nvidia)

The Grace CPU achieved 4.51 instructions per clock (IPC) and 13.03 FP64 operations per cycle with the Neoverse V2 kernel, whereas Vera executed 5.43 instructions per clock and 15.69 FP64 operations per cycle with the very same code, an improvement of approximately 20%.

However, when the researchers used a kernel specifically optimized for Olympus, Vera's performance increased to 7.53 instructions per clock and 21.75 FP64 operations per cycle. As a result, Vera outperformed its predecessor by nearly 67% on a per-core, per-clock basis in this particular workload.

Grace's Neoverse V2 microarchitecture has four 128-bit SIMD pipelines that can perform a total of 16 FP64 operations per clock, whereas Nvidia's Olympus has six such pipelines that can execute up to 24 FP64 operations per cycle, a 50% improvement.

But where the Neoverse V2-optimized kernel uses 81.4% of Grace's peak FP64 throughput, it uses only about 66.5% of Vera's computational capabilities. With changes to matrix tile organization, register allocation, and instruction scheduling that better suit the Olympus microarchitecture, the researchers managed to utilize 90.6% of its peak FP64 throughput, according to Nvidia's tutorial published at SC26.

YJxJdu5bzLRhUHQreJZEAP-1920-80.png

(Image credit: Nvidia)

While software developed and optimized for existing Arm-based data center processors can still benefit from the capabilities of Nvidia's Olympus cores without any specific optimizations, the company's SC26 benchmark results demonstrate that at least some applications will need to be fine-tuned for the new microarchitecture to take full advantage of its execution resources. Of course, it remains to be seen how much performance can be gained through Olympus-specific optimizations in workloads other than matrix multiplication.

As Nvidia expands beyond CPUs that primarily serve as companions to its data center GPUs and chases the standalone server CPU market, the company will have to ensure that performance-critical software takes advantage of the architectural peculiarities of its processors, much like Intel and AMD have done for decades.

But the company is already deeply experienced with partner and ecosystem development for platforms like CUDA, so as Vera is more widely adopted, that experience could prove useful in spurring optimizations for these CPUs going forward.

External Content
Source RSS or Atom Feed
Feed Location https://www.tomshardware.com/feeds/all
Feed Title Latest from Tom's Hardware
Feed Link https://www.tomshardware.com/feeds.xml
Reply 0 comments