Apple Researchers Unveil Keyframer, an AI Tool That Animates Still Images Using LLMs

Apple researchers have unveiled a new AI tool called "Keyframer," (PDF) which harnesses the power of large language models (LLMs) to animate static images through natural language prompts. From a report: This novel application, detailed in a new research paper published on arxiv.org, represents a giant leap in the integration of artificial intelligence into the creative process -- and it may also hint at what's to come in newer generations of Apple products such as the iPad Pro and Vision Pro. The research paper, titled "Keyframer: Empowering Animation Design using Large Language Models," explores uncharted territory in the application of LLMs to the animation industry, presenting unique challenges such as how to effectively describe motion in natural language. Imagine this: You're an animator with an idea that you want to explore. You've got static images and a story to tell, but the thought of countless hours bending over an iPad to breathe life into your creations is, well, exhausting. Enter Keyframer. With just a few sentences, those images can begin to dance across the screen, as if they've read your mind. Or rather, as if Apple's large language models (LLMs) have. Keyframer is powered by a large language model (in the study, they use GPT-4) that can generate CSS animation code from a static SVG image and prompt. "Large language models have the potential to impact a wide range of creative domains, but the application of LLMs to animation is under-explored and presents novel challenges such as how users might effectively describe motion in natural language," the researchers explain.

Read more of this story at Slashdot.