Chips to Compute With Encrypted Data Are Coming

Trust no one. It's not just a throwaway line from TV thrillers. It's becoming the goal of computer security, and a technology that can make it a reality has arrived. Called fully homomorphic encryption, or FHE, it allows software to compute on encrypted data without ever decrypting it.

The possibilities are enormous: huge leaps in medical research and patient care without exposing patient data, more effective tools against money laundering without regulators actually seeing anyone's bank-account information, self-driving cars that can learn from each other without snitching on their drivers, analytics about your business without poking into your customer's business," and much more.

This article is part of our special report Top Tech 2024.

Although FHE software has made some inroads in protecting financial and health care data, it's been held back by the fact that it can take as much as a millionfold more effort on today's computers. But in 2024, at least six companies will be testing or even commercializing the first chips that accelerate FHE to the point where computing on encrypted data is nearly as quick as computing on unencrypted data. And when that's the case, why would you do it any other way?

I think this is the coolest technology of the last 20 years," says Todd Austin, a hardware security expert at the University of Michigan, whose startup Agita Labs does a different form of secure computing in the Amazon and Microsoft clouds. It breaks the cardinal rule of computer security-that everything is hackable-because you deny the programmer the ability to see the data."

Data Protection Regulations Aren't Enough

Regulatory efforts to protect data are making strides globally. Patient data is protected by law in the United States and elsewhere. In Europe the General Data Protection Regulation (GDPR) guards personal data and recently led to a US $1.3 billion fine for Meta. You can even think of Apple's App Store policies against data sharing as a kind of data-protection regulation.

These are good constraints. These are constraints society wants," says Michael Gao, founder and CEO of Fabric Cryptography, one of the startups developing FHE-accelerating chips. But privacy and confidentiality come at a cost: They can make it more difficult to track disease and do medical research, they potentially let some bad guys bank, and they can prevent the use of data needed to improve AI.

Fully homomorphic encryption is an automated solution to get around legal and regulatory issues while still protecting privacy," says Kurt Rohloff, CEO of Duality Technologies, in Hoboken, N.J., one of the companies developing FHE accelerator chips. His company's FHE software is already helping financial firms check for fraud and preserving patient privacy in health care research.

Despite the relatively slow pace of today's unaccelerated FHE, it works because we address use cases where it's not really a computation bottleneck, use cases where there is a human in the loop," such as lawyers negotiating data-use agreements, Rohloff says. Adding a new kind of hardware to his company's software won't just speed FHE, it will let it tackle bigger human-in-the-loop problems as well, he says.

How Fully Homomorphic Encryption Works

At first glance, it might seem impossible to do meaningful computation on data that looks like gibberish. But the idea goes back decades, and was finally made possible in 2009 by Craig Gentry, then a Stanford graduate student. Gentry found a way to do both addition and multiplication without calculation-killing noise accumulating, making it possible to do any form of encrypted computation.

One comparison you can use to understand FHE is that it's analogous to a Fourier transform. For those of you who don't remember your college signal processing, a Fourier transform is a mathematical tool that turns a signal in time, such as the oscillation of voltage in a circuit, into a signal in frequency. One of the key side effects is that any math you can do in the time domain has its equivalent in the frequency domain. So you can compute in either time or frequency and come up with the same answer.

The genius of fully homomorphic encryption is that it uses lattice cryptography- a form of quantum-computer-proof encoding-as the mathematical transformation. The problem with this approach is that the transformation leads to a big change in the type and amount of data and in the sorts of operations needed to compute. That's where the new chips come in.

It's a new chapter in the history of computing." -Ro Cammarota, Intel

Computing with FHE means doing transforms, addition, and multiplication on a very long list of numbers, and each number in itself is very large," explains Rohloff. Computing with numbers that might require more than a hundred bits to describe is not something today's CPUs and GPUs are inherently good at. If anything, GPUs have been going in the opposite direction, focusing on less precise math done using smaller and smaller floating-point numbers. The FHE accelerator chips, by contrast, can stream huge volumes of data through hardware that does integer math on numbers that are thousands of bits long to accommodate encryption's precision needs.

Each accelerator has its own way of dealing with these streams of huge numbers. But they're all after the same goal-making FHE as fast as today's unencrypted computing.

DARPA Drives FHE

The quest for hardware that can accelerate FHE got its biggest boost in 2021, when the U.S. Defense Advanced Research Projects Agency (DARPA) began a project called DPRIVE. The goal was to build hardware that could radically reduce the time it took for FHE computing tasks, from weeks to just seconds or even milliseconds. Three participating teams-led by Duality Technologies, Galois, and Intel-are on track to deliver chips designed to make FHE perform within a factor of 10 of traditional computing or even better in 2024.

These chips will be crucial if FHE is to break out of its current niche. While algorithm and software development has taken us far, it's not nearly far enough for FHE to be practical in any but a small and narrow set of applications," says Galois's David Archer. A distinction of the Galois hardware, called Basalisc, is the use of asynchronous clocking so that the various types of circuits used to do FHE operations can run at their own speed.

For the Intel team's chip, Heracles, they came up with a way to decompose FHE's huge numbers into short data words that are just 32 bits. The smaller words lead to a lower computing latency. They also mean Intel can squeeze in more computational units and more pathways for data to reach those units, explains Ro Cammarota, chief scientist for privacy-enhanced computing research at Intel.

The Duality team, whose chip is called Trebuchet, sees its advantage as having a design that's made to support and accelerate the FHE software the startup has already commercialized. We started from applications to drive our software and then have that software drive our hardware," says Rohloff.

FHE Startups Smell Opportunity

At least three other companies went after FHE hardware independently of DARPA's DPRIVE.

Gao founded Fabric Cryptography after leaving his previous startup, an optical computing company called Luminous that sought to accelerate AI. Impressed and a little concerned with the amount of data his customers had, Gao wanted to see what encrypted computing could do about maintaining people's privacy while still helping businesses benefit from the information. The result is a chip that Fabric expects to be in mass production within the year.

For Campbell, Calif.-based Cornami, FHE was an opportunity to repurpose a new type of parallel computing architecture. The architecture was originally designed to speed computing by allowing programs to be broken up into completely independent streams of instructions, which could then flow through the processor's many cores without the delays of having to share resources.

When chip-industry veteran Walden C. Wally" Rhines came across Cornami in 2019, the company was planning to apply the architecture to machine learning, but the field was already too crowded, he says. Instead, fresh off some work for DARPA on FHE, he steered the startup in that direction. Rhines, who is now CEO, says Cornami will have a product ready in 2024 that will let FHE match plain-text computation speeds.

Optalysys, in Leeds, England, is looking to take advantage of optical computing's inherent agility with Fourier transforms. It's long been known that a fairly straightforward optical system can instantly produce the Fourier transform of a two-dimensional image. Optalysys was founded more than a decade ago to exploit this phenomenon, and it has built systems over the years for defense-related tasks like finding patterns in cluttered images.

With the increasing availability of silicon photonics tech, the company has been able to adapt its transform-powered technology for encryption and FHE, CEO Nick New says. FHE is an area that is absolutely dominated by" transforms that can be done in optics, he says. The startup plans to have a product ready in the second half of 2024.

FHE's Road Ahead

Ultimately, if it's fast enough and cost effective enough, there's no reason not to use FHE," says New. But there's a long way to go to get to that point."

Intel's Cammarota sees the accelerator chips as just the starting point. FHE will also need software development tools to make programming easier as well as standardization. The two are in progress even without chips in hand, but there are many ways to do FHE and standardization work is in its early stages.

Once industry has all three ingredients-software, standards, and hardware-researchers can begin to see what else these accelerator chips can do. It's a new chapter in the history of computing," says Cammarota.

This article appears in the January 2024 print issue.