TSMC: Shortage Of Nvidia's AI GPUs To Persist For 1.5 Years
Arthur T Knackerbracket has processed the following story:
TSMC warns AI chip crunch will last another 18 monthsInsufficient packaging capacity is to blame.
The chairman ofTSMCadmitted that the ongoing short supply of compute GPUs for artificial intelligence (AI) and high-performance computing (HPC) applications is caused byconstraintsof its chip-on-wafer-on-substrate (CoWoS) packaging capacity.This shortage is expected to persist for around 18 monthsdue torising demand for generative AI applicationsand relatively slow expansion of CoWoS capacity at TSMC.
"It is not the shortage of AI chips, it is the shortage of our CoWoS capacity,"said Mark Liu, the chairman of TSMC, in a conversation with Nikkeiat Semicon Taiwan. "Currently, we cannot fulfill 100% of our customers' needs, but we try to support about 80%. We think this is a temporary phenomenon. After our expansion of [advanced chip packaging capacity], it should be alleviated in one and a half years."
TSMC istheproducerof the majority of AI processors, including Nvidia'sA100 andH100compute GPUs thatare integral to AI tools like ChatGPT and are predominantly used in AI data centers.These processors, just like solutions from other players like AMD, AWS, and Google, use HBM memory (which isessential forhigh bandwidthandproperfunctioning of extensive AI language models) and CoWoS packaging, which puts additional strain on TSMC's advanced packaging facilities.
Liu said that demand for CoWoS surged unexpectedly earlier this year, tripling year-over-year, leading to the current supply constraints. TSMC recognizes thatdemand forgenerative AI services is growing and so is demand for appropriate hardware, so it is speeding up expansion of CoWoS capacity to meet demand for compute GPUs as well as specialized AI accelerators and processors.
At present, the company is installing additional tools for CoWoS at its existing advanced packaging facilities, but this takes time and the company expects its CoWoS capacity to double only by the end of 2024.
Arthur T Knackerbracket has also processed the following story:
Until TSMC can bring additional capacity online, Nvidia's H100 and older A100 - which power many popular generative AI models, such as GPT-4 - are at the heart of this shortage. However, it's not just Nvidia. AMD's upcoming Instinct MI300-series accelerators - which it showed off during its Datacenter and AI event in June - make extensive use of CoWoS packaging technology.
AMD's MI300A APU is currently sampling with customers and is slated to power Lawrence Livermore National Laboratory's El Capitan system, while the MI300X GPU is due to start making its way into customers' hands in Q3.
We've reached out to AMD for comment on whether the shortage of CoWoS packaging capacity could impact availability of the chip and we'll let you know if we hear anything back.
It's worth noting that TSMC's CoWoS isn't the only packaging tech out there. Samsung, which is rumored to pick up some of the slack for the production of Nvidia GPUs, has I-Cube and H-Cube for 2.5D packaging and X-Cube for 3D packaging.
Intel, meanwhile, packages several of the chiplets used in its Ponte Vecchio GPU Max cards, but doesn't rely on CoWoS tech to stitch them together. Chipzilla has developed its own advanced packaging tech, which can work with chips from different fabs or process nodes. It's called embedded multi-die interconnect bridge (EMIB) for 2.5D packaging and Foveros for vertically stacking chiplets on top of one another.
Read more of this story at SoylentNews.