Meta’s AI leaders want you to know fears over AI existential risk are “ridiculous”

It's a really weird time in AI. In just six months, the public discourse around the technology has gone from Chatbots generate funny sea shanties" to AI systems could cause human extinction." Who else is feeling whiplash?

My colleague Will Douglas Heaven asked AI experts why exactly people are talking about existential risk, and why now. Meredith Whittaker, president of the Signal Foundation (which is behind the private messaging app Signal) and a former Google researcher, sums it up nicely: Ghost stories are contagious. It's really exciting and stimulating to be afraid."

We've been here before, of course: AI doom follows AI hype. But this time feels different. The Overton window has shifted in discussions around AI risks and policy. What was once an extreme view is now a mainstream talking point, grabbing not only headlines but the attention of world leaders.

Read more from Will here.

Whittaker is not the only one who thinks this. While influential people in Big Tech companies such as Google and Microsoft, and AI startups like OpenAI, have gone all in on warning people about extreme AI risks and closing up their AI models from public scrutiny, Meta is going the other way.

Last week, on one of the hottest days of the year so far, I went to Meta's Paris HQ to hear about the company's recent AI work. As we sipped champagne on a rooftop with views to the Eiffel Tower, Meta's chief AI scientist, Yann LeCun, a Turing Award winner, told us about his hobbies, which include building electronic wind instruments. But he was really there to talk about why he thinks the idea that a superintelligent AI system will take over the world is preposterously ridiculous."

People are worried about AI systems that are going to be able to recruit all the resources in the world to transform the universe into paper clips," LeCun said. That's just insane." (He was referring to the paper clip maximizer problem," a thought experiment in which an AI asked to make as many paper clips as possible does so in ways that ultimately harms humans, while still fulfilling its main objective.)

He is in stark opposition to Geoffrey Hinton and Yoshua Bengio, two pioneering AI researchers (and the two other godfathers of AI"), who shared the Turing prize with LeCun. Both have recently become outspoken about existential AI risk.

Joelle Pineau, Meta's vice president of AI research, agrees with LeCun. She calls the conversation "unhinged." The extreme focus on future risks does not leave much bandwidth to talk about current AI harms, she says.

When you start looking at ways to have a rational discussion about risk, you usually look at the probability of an outcome and you multiply it by the cost of that outcome. [The existential-risk crowd] have essentially put an infinite cost on that outcome," says Pineau.

When you put an infinite cost, you can't have any rational discussions about any other outcomes. And that takes the oxygen out of the room for any other discussion, which I think is too bad."

While talking about existential risk is a signal that tech people are aware of AI risks, tech doomers have a bigger ulterior motive, LeCun and Pineau say: influencing the laws that govern tech.

At the moment, OpenAI is in a position where they are ahead, so the right thing to do is to slam the door behind you," says LeCun. Do we want a future in which AI systems are essentially transparent in their functioning or are ... proprietary and owned by a small number of tech companies on the West Coast of the US?"

What was clear from my conversations with Pineau and LeCun was that Meta, which has been slower than competitors to roll out cutting-edge models and generative AI in products, is banking on its open-source approach to give it an edge in an increasingly competitive AI market. Meta is, for example, open-sourcing its first model in keeping with LeCun's vision of how to build AI systems with human-level intelligence.

Open-sourcing technology sets a high bar, as it lets outsiders find faults and hold companies accountable, Pineau says. But it also helps Meta's technologies become a more integral part of the infrastructure of the internet.

When you actually share your technology, you have the ability to drive the way in which technology will then be done," she says.

Deeper Learning

Five big takeaways from Europe's AI Act

It's crunch time for the AI Act. Last week, the European Parliament voted to approve its draft rules. My colleague Tate Ryan-Mosley has five takeaways from the proposal. The parliament would like the AI Act to include a total ban on real-time biometrics and predictive policing in public spaces, transparency obligations for large AI models, and a ban on the scraping of copyrighted material. It also classifies recommendation algorithms as high risk" AI that requires stricter regulation.

What happens next? This does not mean the EU is going to adopt these policies outright. Next, members of the European Parliament will have to thrash out details with the Council of the European Union and the EU's executive arm, the European Commission, before the draft rules become law. The final legislation will be a compromise between three different drafts from the three institutions. European lawmakers are aiming to get the AI Act in final shape by December, and the regulation should be in force by 2026.

You can read my previous piece on the AI Act here.

Bits and Bytes

A fight over facial recognition will make or break the AI Act
Whether to ban the use of facial recognition software in public places will be the biggest fight in the final negotiations for the AI Act. Members of the European Parliament want a complete ban on the technology, while EU countries want the freedom to use it in policing. (Politico)

AI researchers sign a letter calling for focus on current AI harms
Another open letter! This one comes from AI researchers at the ACM conference on Fairness, Accountability, and Transparency (FAccT), calling on policymakers to use existing tools to design, audit, or resist AI systems to protect democracy, social justice, and human rights." Signatories include Alondra Nelson and Suresh Venkatasubramanian, who wrote the White House's AI Bill of Rights.

The UK wants to be a global hub for AI regulation
The UK's prime minister, Rishi Sunak, pitched his country as the global home of artificial-intelligence regulation. Sunak's hope is that the UK could offer a third way" between the EU's AI Act and the US's Wild West. Sunak is hosting a AI regulation summit in London in the fall. I'm skeptical. The UK can try, but ultimately its AI companies will be forced to comply with the EU's AI Act if they want to do business in the influential trading bloc. (Time)

YouTube could give Google an edge in AI
Google has been tapping into the rich video repository of its video site YouTube to train its next large language model. This material could help Google train a model that can generate not only text but audio and video too. Apparently this is not lost on OpenAI, which has been secretly using YouTube data to train its AI models. (The Information)

A four-week-old AI startup raised 105 million
Talk about AI hype. Mistral, a brand-new French AI startup with no products and barely any employees, has managed to raise 105 million in Europe's largest-ever seed round. The founders of the company previously worked at DeepMind and Meta. Two of them were behind the team that developed Meta's open-source Llama language model. (Financial Times)