The Biggest New AI Features Coming to Google's Pixel 10 Phones

The new Pixel 10 lineup, as you'd expect, is all about AI. But it's taking a more subtle approach. Gemini has become a staple for many Android smartphones, not just Pixel, so the focus is on what Pixel can do differently: Using its new AI-tailored chip to provide smaller, on-device AI features that no one else can. Here are all the important AI features that will be coming exclusively to Pixel 10 series smartphones (including the Pro and Fold) later this year.

A New AI-tailored Chip for on-device processing

The bedrock of all these new changes is the new Tensor G5 chip. It's Google's biggest silicon upgrade to date. Google has shifted to using TSMC's latest 3 nm process, instead of Samsung's foundries, meaning it's fully custom. This has, according to Google, lead to approximately 34% faster performance overall, but more importantly, the on-device Tensor Processing Unit that handles AI tasks is 60% faster than before.

This means that Google can run its lightweight Gemini Nano model in a more efficient and dynamic fashion, efficiently switching between sub-models depending on the use case.

Now, almost all the new AI features coming to Pixel 10 series are processed on-device, which lets them work offline and gives them more reign to use your data without privacy concerns.

Magic Cue Credit: Google

Magic Cue is going to be everywhere on Pixel 10, but only when it's needed. It's Google's version of contextual quiet computing", where it surfaces important information and actionable steps exactly when needed. When not, it just doesn't come up.

For example, say your friend messages you to ask about dinner reservations. Magic Cue can understand this, pull up relevant information from your Gmail or Google Calendar, and bring it up as a suggested text right there. All you have to do is tap to send it. This can work for flight details as well, which will display on screen during phone calls with the airline. This is built into most of Google's core apps, like Gmail, Calendar, Screenshots, Messages, and more.

Image Editing with Gemini Credit: Google

In Pixel 10, you won't need to use buttons and sliders to edit images anymore. Google is integrating Gemini right inside the Photos image editor. It's coming first to the Pixel 10 in the U.S.

With Gemini, you'll be able to use your voice or text to edit your images, and you'll get responses back in real-time. You can give commands like remove the cars in the background" or restore this old photo". If you don't know exactly what you want, you can also just say something like make it better".

Image editing using AI also raises some concerns about whether what you're seeing is real. To get around this, Google is adding C2PA Content Credentials in Google Photos, which lets users view a photo's metadata to check all the ways AI was used in its creation.

Camera Coach Credit: Google

There's a new AI mode built-in to the Pixel camera that isn't purely about editing or enhancing images. It's about helping you, the user, take better images from the get-go. Quick note: Google told Lifehacker that this one does send a photo to the cloud, but also said it gets deleted after you're done using the feature.

Essentially, Camera Coach analyzes a photo and gives you prompts or suggestions on how to take a better shot. It will tell you to maybe try moving a couple of steps back, or try inverting your phone to make the subject look taller. It can also suggest you shift the angle if the lighting is too harsh, and use AI to ideate scenarios that weren't in your original shot.

AI voice translation that sounds like you

AI live translation is coming to Pixel's Phone app. Pixel 10 can translate calls in real time, via speech-to-speech, and even matches the translated AI voice to sound just like the speaker's voice. This means two people can carry on a conversation in their own native tongues. There's no in-depth training period required for the AI voice clone, either. As you speak, the voice will adapt to your cadence and nuances. Again, all of this happens on device, too, so it's fast. This works with English to and from Spanish, German, Japanese, French, Hindi, Italian, Portuguese, Swedish, Russian, and Indonesian.

Highlights in Gemini Live camera view Credit: Google

A couple of smaller Gemini Live features are coming to Pixel phones on August 28th (along with other supported Android smartphones). The major new feature is Gemini's ability to highlight things on your screen. It makes a little box around what it wants you to focus on, and dims the rest of the screen. Gemini Live will also be able to interact with more apps, including Messages, Phone, and Clock.

The AI Treatment for missed calls

Google is adding AI features to your missed calls as well. The Take a Message feature will now provide real-time transcripts of incoming call, but it goes beyond that. You'll get a summary, and actionable steps based on the voicemail.