These minuscule pixels are poised to take augmented reality by storm

Google Glass, a prototype augmented-reality headset released in April 2013, had the makings of a hit. It promised intuitive, hands-free access to a smartphone's most important features-video recording, navigation, and even email. Forget touch screens and buttons: the future of computing was on your face.

It was a disaster.

Though beautiful in concept, Glass was awkward to wear and struggled to deliver a sharp, bright image outdoors. Then came the glasshole" backlash. The size of the display made wearers easy to spot in a crowd and, on at least two occasions, led to physical altercations.

The implications were clear. Hands-free augmented reality (AR) was fun on paper, but with tensions over Big Tech's influence mounting, it couldn't overcome the stigma of making people look like extras in a cyberpunk flick.

Now, more than a decade later, the future Google envisioned-and much more-is on the brink of becoming reality. Tiny new displays, some small enough to fit on the tip of your finger, will contain micro-LEDs and micro-OLEDs (organic LEDs). They are set to deliver a wave of headsets that may convert even the most ardent AR skeptics.

Apple's Vision Pro, slated for release in 2024, will lead this change-though it might not shake the cyberpunk aesthetic. The fully enclosed headset, vaguely reminiscent of ski goggles, is intended for a mixture of AR and virtual reality (VR) that Apple calls spatial computing."

The Vision Pro avoids some of the problems Google Glass faced by narrowing the product's scope. Apple hopes the headset might replace a computer, tablet, and TV-though only within the confines of your home or office.

The real innovation is inside: a pair of micro-OLED displays no larger than a postage stamp that pack 4K resolution into a screen just 1.3 inches square. Each display contains more than 11 million pixels spaced only 6.3 micrometers apart-less than the diameter of a human red blood cell.

It's a spectacular upgrade. Apple's Vision Pro, like the Meta Quest 3 and the HTC Vive XR Elite, uses cameras to replicate the outside world on internal displays, a technique known as pass-through mixed reality. But its competitors use liquid crystal displays that lack the sharpness to faithfully reproduce the world around you, so tasks that should be simple, like glancing at a handwritten note, can prove difficult.

I think overall they've achieved something impressive," says Anshel Sag, principal analyst at Moor Insights & Strategy. This is the headset that you build if you want people to really, fully understand what the maximum potential of AR and VR is." Sag believes the individual pixels on Vision Pro displays will be invisible to most people, unless you have extremely impressive visual acuity, like 20/10."

The Vision Pro's pixel-dense displays are widely believed to be the culmination of years of work from Sony's Semiconductor Solutions Group. The division's micro-OLED adventures were originally focused on colorful high-resolution digital viewfinders for cameras like the Sony SLT-A77. The group also built them for a head-mounted device, the HMZ-T1 Personal 3D Viewer, that Sony launched in 2011, pitching it as a movie-theater-like experience for watching video.

The HMZ-T1 headset performed best with 3D films, which proved to be a fad. But Sony didn't give up on micro-OLEDs, and in 2018 it announced a 0.5-inch micro-OLED display that reduced the distance between pixels from 7.8 to 6.3 micrometers (the same as the larger displays found in the Vision Pro), an innovation made possible by a breakthrough that placed the color filter closer to the OLED's light-emitting organic material. With a display this small, any subtle change in the angle of light emitted from red, blue, and green subpixels can hurt color performance. Moving the color filter improves the viewing angle of each pixel, which makes a smaller display possible without compromising image quality.

Micro-OLEDs benefit from some of the traditional strengths of light-emitting diodes made with organic films. Each pixel is self-emissive, which means its brightness is zero when it's off." The LCDs in most headsets can't achieve this, and as a result, darker scenes have a hazy, gray glow. And when micro-OLEDs are on, they're on. The Vision Pro's displays are quoted at a peak brightness of 5,000 nits, the industry's go-to measure of brightness. It's a 50-fold improvement over Meta's Quest 2, which hits just 100 nits. (Meta hasn't revealed the Quest 3's brightness, but it's likely similar.)

The Vision Pro is likely to quicken the adoption of micro-OLED technology. But despite its many strengths, those miniature OLEDs still have some shortcomings. Michael Murray, CEO of Kopin, a display company in Westborough, Massachusetts, notes that micro-OLED displays are excellent for moving images, such as movies, but sometimes less so for static text-a reason, he says, why Meta's Quest headsets have stuck with LCD. While micro-OLED displays can be bright, the organic molecules inside them can degrade over time, a phenomenon known as burn-in. Micro-OLED also fails to entirely resolve the design issues of Google Glass: the display is improved, but the headset is even more conspicuous.

Fortunately, micro-LEDs offer a solution.

Truly microscopic

Micro-OLED and micro-LED displays differ in the details, but their production shares broad similarities. Both pair a silicon backplane," which provides structure and power, with a display frontplane" that creates visible light. Each is named for the type of frontplane used: a layer of organic material that emits light in response to an electric current in the case of micro-OLED displays, and a very small array of electronic diodes made from semiconductors in the case of micro-LEDs.

Micro-LED display technology is not as mature as micro-OLED, but the possibilities are alluring. Micro-LED happens to be the best of all worlds," says Murray. It has the best display quality, it has longevity, doesn't have burn-in issues, has high brightness that you can control ... that's where the future is going."

Mojo Vision, a display technology company based in Saratoga, California, was among the first companies to realize the LED's potential in tiny devices. It made waves in 2020 with a contact lens with a flexible, transparent AR display. The company has since abandoned the contact lens to focus just on the display, and in 2023, Mojo Vision demonstrated micro-LED displays with an astounding 28,000 pixels per inch. That works out to a pixel pitch-the distance between the centers of two adjacent pixels-of just 1.87 micrometers, smaller than some bacteria and a third the size of what you'll find in the Apple Vision Pro.

Mojo Vision hopes to have a color micro-LED prototype ready in early 2024.COURTESY OF MOJO VISION

Such extreme pixel density is the result of a fundamental shift in micro-LED design. The first micro-LED displays were built with a technique called mass transfer." Red, blue, and green LEDs were produced on wafers and transferred one by one to a display substrate (a technique that is still used to make larger displays). But small micro-LED arrays, like those produced by Mojo Vision, take a monolithic approach: the micro-LEDs and the silicon backplane are bonded in a production pipeline like that used to manufacturer cutting-edge computer chips.

Most monolithic micro-LED displays are currently monochrome, meaning they display a single color (usually red, blue, or green). But full-color micro-LED displays are right around the corner. Mojo Vision hopes to have a color micro-LED prototype ready in early 2024, and one of its competitors, Shanghai-based Jade Bird Displays (often referred to by its initials, JBD), has demonstrated a functional color micro-LED prototype with a pixel pitch of five micrometers-larger than what Mojo Vision hopes to achieve, but smaller than Apple's Vision Pro.

The key benefit of smaller, denser pixels is the reduction of display size at any given resolution, which in turn reduces the size and weight of an AR headset. JBD's monochromatic AmuLED series, for example, achieves 640 x 480 resolution on a display a carpenter ant could carry on its back-with room to spare.

Micro-LEDs also score a massive win in brightness. The range is from 1.8 million up to 3 million nits, Murray says: It will literally tear the retina out of your eye and blind you for life." The brightest OLED displays, by comparison, currently peak at around 15,000 nits.

The possibility of permanent eye damage might seem an odd perk, but not to worry-no one will be looking at the micro-LEDs directly. Placing a display directly in front would block the wearer's view of the real world, so many AR devices place the display to the side. Waveguides then redirect the light from the offset display to make it visible. This process can prove tremendously inefficient, especially for modern AR glasses like the Magic Leap 2 and Vuzix Blade 2, which focus and redirect light through multiple waveguides arranged like mirrors in a fun house.

[The efficiency] is something like 5% to 10%," says Michael Miller, augmented-reality hardware lead at Niantic. If you have a display of 3,000 nits, you will get 300 nits out. You can put a dark lens on top of it so you can maybe use it outdoors, but it's not good enough."

Displays built from micro-LEDs should be able to make it through a gauntlet of waveguides and still be bright enough to be viewed on transparent lenses that look just like prescription ones.

Awesome performance, awesome cost

Headsets with cutting-edge displays, like the upcoming Vision Pro, thrash the performance of mass-market VR headsets. They're also more expensive: the Vision Pro will retail for $3,499.

The displays deserve some of the blame.

Each micro-OLED display can cost $400 to manufacture, says Murray. If you're building a Meta Quest, or something like it, you need two of them," he says, and your bottom-line cost is already $800."

The big price tag attached to such small displays might seem strange. After all, OLED displays are a mature technology found in hundreds of millions of smartphones, tablets, and televisions worldwide. LEDs are even more ubiquitous: just flip a nearby light switch to see one in action. These are well-understood technologies found in many affordable devices.

HMDmd's Model CR3 headset is designed specifically for surgeons.Like other AR glasses, the Vuzix Blade 2 redirects light from the side to the front through multiple waveguides.

On this diminutive scale, however, building a display is no longer a job for a factory. It requires a foundry-a specialized chip-manufacturing facility.

Costs could come down as manufacturers shift to building the displays on larger silicon wafers. Larger wafers are more expensive, but each one can pack more displays, which lowers the cost of each display. Micro-OLED makers are in the midst of a shift from eight-inch wafers to 12-inch wafers, which is the standard in high-volume, cutting-edge silicon manufacturing. Micro-LED production is less mature, with some companies relying on inefficient four-inch wafers.

Producing usable displays with the extreme pixel densities that micro-OLEDs and micro-LEDs can achieve is a challenge. The fundamental problem is a defect you've likely witnessed more than once: the dead" pixel. A dead pixel displays one color-often perfect black or a bright, blinding white-and refuses to respond to display signals. Avoiding this defect is already difficult for smartphone displays, where pixels might be separated by 500 micrometers. With monolithic micro-LEDs, the smallest, most densely packed displays ever produced, the slightest flaw in the silicon, or the slimmest sliver of debris, can render a display useless.

Here's the scary math," says Murray. The amount of usable displays after you're done is probably a tenth of the usable silicon you started with." In other words, more than 90% of a silicon wafer could be wasted. Yet the company producing the micro-LED displays still pays for the entire wafer-adding huge costs to each display.

Micro-LED pioneers are investing in tools and processes that reduce the steps involved in production. That's critical, because the more complex the production process, the higher the risk of introducing a defect.

Soeren Steudel, CTO of the Belgian display developer Micledi, is hyper-focused on this problem. The company has partnered with the semiconductor manufacturer GlobalFoundries and plans to move production there to reduce costs. Micro-LED is not yet a mature product. It was a wild dream 10 years ago, and now the first companies have demos," says Steudel. The question now is, how can you manufacture that in volume without defects?"

Augmented reality finally goes mainstream

The difficulty of producing micro-OLED and micro-LED displays is high, but the problems are worth solving. These displays could make AR a virtual space people can easily and quickly access in their daily lives-not only because the displays appear more lifelike, but also because small, thin, high-quality displays give engineers more freedom to tailor a headset's look and feel.

The impact of micro-OLEDs is already apparent. Kopin produces displays for HMDmd's Model CR3, a headset designed for surgeons, and defense projects, such as an AR weapon sight for the M1 Abrams main battle tank. XReal, another AR pioneer, recently released its Air 2 headset, which packs Sony micro-OLED displays.

The possibilities for the future could be even more dramatic. The displays' extreme brightness, diminutive size, and low power consumption could unlock the dream of light, attractive, fully transparent AR glasses that don't immediately stand out from conventional eyewear.

People want to go to consumer augmented reality. And consumer AR means that you have lightweight glasses, maybe 50 grams, and that you don't look like Darth Vader," says Steudel. Vuzix, a leader in lightweight VR headsets, has achieved this with the Ultralite, a prototype platform revealed in 2023 that uses micro-LED technology to provide sleek, slim spectacles that weigh just 38 grams.

Augmented reality still needs its iPhone moment"-the debut of an easy-to-use device that offers irresistible benefits. Better displays will make AR-if it ever gets widely adopted-bright, sharp, convincing, and-most important of all-pleasant to use.

Matthew S. Smith is a freelance technology journalist based in Portland, Oregon.