It Takes a Lot of Energy for Machines to Learn – Here's Why AI is So Power-Hungry
upstart writes in with an IRC submission for SoyCow7066:
It takes a lot of energy for machines to learn - here's why AI is so power-hungry:
Researchers at the University of Massachusetts Amherst estimated the energy cost of developing AI language models by measuring the power consumption of common hardware used during training. They found that training BERT once has the carbon footprint of a passenger flying a round trip between New York and San Francisco. However, by searching using different structures - that is, by training the algorithm multiple times on the data with slightly different numbers of neurons, connections and other parameters - the cost became the equivalent of 315 passengers, or an entire 747 jet.
[...] All of this means that developing advanced AI models is adding up to a large carbon footprint. Unless we switch to 100% renewable energy sources, AI progress may stand at odds with the goals of cutting greenhouse emissions and slowing down climate change. The financial cost of development is also becoming so high that only a few select labs can afford to do it, and they will be the ones to set the agenda for what kinds of AI models get developed.
[...] What does this mean for the future of AI research? Things may not be as bleak as they look. The cost of training might come down as more efficient training methods are invented. Similarly, while data center energy use was predicted to explode in recent years, this has not happened due to improvements in data center efficiency, more efficient hardware and cooling.
[...] Looking forward, the AI community should invest more in developing energy-efficient training schemes. Otherwise, it risks having AI become dominated by a select few who can afford to set the agenda, including what kinds of models are developed, what kinds of data are used to train them and what the models are used for.
Reference:
Emma Strubell, Ananya Ganesh, Andrew McCallum. Energy and Policy Considerations for Deep Learning in NLP (arXiv:1906.02243v1 [cs.CL])
Read more of this story at SoylentNews.