OpenAI’s New GPT-4o Can Provide Realtime Language Translation With a Simple Verbal Request

OpenAI launched GPT-4o, their new flagship model that can reason across audio, vision, and text in real time", which includes the very useful realtime language translation feature. One of the users proposes to the AI the languages that need to be translated and the translation kicks in.

Hey Chat GPT, I'm here with my co-worker today. We'd like you to act as a translator for us so every time I say something in English can you repeat it back in Spanish and every time he says something in Spanish can you repeat it back in English?

A Live Demo of the Translation Feature

OpenAI CTO Mira Murati demonstrated the realtime translation live while also providing an overall explanation of GPT-4o.

Introducing GPT-4o

With GPT-4o, we trained a single new model end-to-end across text, vision, and audio, meaning that all inputs and outputs are processed by the same neural network. Because GPT-4o is our first model combining all of these modalities, we are still just scratching the surface of exploring what the model can do and its limitations.

Other GPT-4o Features

Some other GPT-4o features include accessibility, singing, vocal inflection, teaching a language, and lots of personality.