Waymo has designed a robocar chip to stay ahead of Tesla

To reach their destinations safely, autonomous vehicles have just milliseconds to ingest and process streaming data from more than a dozen cameras. It's a job that's been handled with off-the-shelf AI components thus far, but Waymo has begun rolling its own AI ASICs to optimize the process. It's not alone. Revealed in a blog post Thursday, the Alphabet-backed robo-taxi startup's first custom silicon is designed to convert raw sensor data into driver responses as quickly as possible. Built on Taiwanese foundry giant TSMC's 5 nm process tech, the chip is specifically optimized to run both more traditional machine learning algorithms like convolutional neural networks and modern transformer models similar to those used to run AI chatbots or image generation models. According to Waymo, the chip's design incorporates more than 200 million miles worth of autonomous driving data, and is tuned to maximize responsiveness, reliability, and redundancy. Prior to this, Waymo had employed Intel FPGAs for sensor processing. FPGAs are ideal in low latency applications, which is one of the reasons why high frequency trading often takes place on them. However, compared to dedicated silicon, FPGAs are notoriously difficult to program for and lack the compute density achievable using application specific hardware. Accidents can unfold in a fraction of a second, far too quickly for a remote operator to take over. So Waymo designed the chip with a major focus on minimizing latency. Within those critical milliseconds, advanced ML models build a high-fidelity understanding of the environment to evaluate the safest path forward," the company explained. This includes performing temporal noise reduction to improve low light visibility in real time. All of that requires a considerable amount of computation. The robo-taxi startup claims its ASICs are capable of churning out more than 1,000 TOPS of AI performance. But without knowing the precision and power levels the chips are operating at, it's difficult to draw comparisons to existing autonomous vehicle and robotics platforms. We've reached out seeking clarification, and will let you know if we hear back. But considering that Waymo is specifically advertising TOPS, we're probably looking at INT8 performance, which would put it in the same ballpark as Nvidia's Drive AGX Thor platform. In addition to offering plenty of compute, the chip also needs to be reliable. Vehicles are exposed to a near-constant stream of vibrations, shock, and extreme temperature swings, unlike anything you'd see in a datacenter. In order to combat this, Waymo has employed multiple layers of redundancy. The chips themselves are liquid cooled by the same coolant system used by the vehicle itself, ensuring that the silicon maintains optimal temps regardless of the weather. Meanwhile, to ensure a hardware fault doesn't put passengers at risk, Waymo says each vehicle is equipped with a pair of ASICs. Under normal conditions, the chips behave as a single unit. But in the event one fails or produces an erroneous result, the other can take over. Air and space craft often include a third system to serve as a tiebreaker in the event of an upset. So, it's not surprising to see this level of redundancy employed for a vehicle that's going to be hurtling down roads filled with other vehicles, pedestrians, and obstacles. To be clear, Waymo's custom ASIC isn't responsible for all of the vehicle's functions. The company says that it is working on several other custom chips and systems, but for now, non-ML tasks like orchestration, data movement, and logging are handled by components provided by its partners, which include AMD, Micron, Samsung, Sandisk, and Nvidia. Waymo is far from the only autonomous vehicle vendor rolling its own custom silicon. Tesla, which launched a limited Robotaxi service in Austin after years of missed deadlines, has been developing custom chips for its vehicles for years. Waymo will share more detail on its ML accelerators next week during the annual Hot Chips conference at Stanford. (R)