Article 6VN6S AI firms follow DeepSeek’s lead, create cheaper models with “distillation”

AI firms follow DeepSeek’s lead, create cheaper models with “distillation”

by
Cristina Criddle and Melissa Heikkilä, Financial
from Ars Technica - All content on (#6VN6S)
Story Image

Leading artificial intelligence firms including OpenAI, Microsoft, and Meta are turning to a process called distillation" in the global race to create AI models that are cheaper for consumers and businesses to adopt.

The technique caught widespread attention after China's DeepSeek used it to build powerful and efficient AI models based on open source systems released by competitors Meta and Alibaba. The breakthrough rocked confidence in Silicon Valley's AI leadership, leading Wall Street investors to wipe billions of dollars of value from US Big Tech stocks.

Through distillation, companies take a large language model-dubbed a teacher" model-which generates the next likely word in a sentence. The teacher model generates data which then trains a smaller student" model, helping to quickly transfer knowledge and predictions of the bigger model to the smaller one.

Read full article

Comments

External Content
Source RSS or Atom Feed
Feed Location http://feeds.arstechnica.com/arstechnica/index
Feed Title Ars Technica - All content
Feed Link https://arstechnica.com/
Reply 0 comments