Never Mind The Deepfakes, Chatbots Have To Deal With Election Lies Being Free, While Truth Is Paywalled

It seems that every election cycle over the last decade has had concerns being raised about the state of political deepfakes" and their ability to impact the election. So far, these have always been overblown, and there's been little evidence of deepfakes having any real impact at all - perhaps other than creating the concept of the liar's dividend" enabling politicians caught doing actually bad things to falsely claim that it's fake news" and a deepfake." That could change, of course, and there are already concerns being raised about the impact of deepfakes midterms along with the fact that most of the major models are letting people create deepfake content.

The concern raised in the Tech Policy Press piece is less about putting a politician in a compromising photo or video (those don't seem to get very far before people recognize they're fake) and more about using these tools to produce ancillary artifacts that help support claims of other conspiracy theories:

Every AI tool that we tested produced convincingly false content. Different chatbots created images that included an AI-generated person stuffing a stack of mail ballots into a drop box, a fake memo from the federal government on official" letterhead reporting millions of ineligible voters, and a raid on a voter registration drive that never happened. We then fed those fake pieces of content back into chatbots and asked whether they were authentic. Most chatbots got it wrong, even when given images they had generated themselves. After California's AI Transparency Act - which requires companies to provide provenance data for images they generate as well as free, public tools for detecting synthetic media - took effect, we conducted a follow-up test. The chatbots still failed to spot AI-generated images.

When we re-ran our image generation prompts this month, we found that two of the AI models we previously tested made it easier to create convincing fakes. Gemini no longer put a visible watermark on its generated images, as it did in the spring, and Meta AI produced an image of a fake executive order on mail voting that it refused to make in April, though it did garble some of the text.

I'm still not convinced this is as big a risk as others have made it out to be, but it's definitely worth watching how things play out as the tech gets better... and as some people are more and more desperate to hold onto power.

That said, there's potentially a much bigger issue regarding AI and elections. As Will Oremus notes at The Atlantic, this is the first big US election where all the major AI tools don't just generate" content, but can also search the web as a part of that process. And that means the results they generate can steer people in the wrong direction:

In a forthcoming report, Forum AI, a start-up that evaluates AI models' performance on questions about controversial topics, found that about 40 percent of chatbots' answers to election-related questions contained at least one factual inaccuracy. When Forum's researchers asked one widely used model when they needed to request a mail-in ballot to vote in the midterms in Pennsylvania, the chatbot said November 3. The actual deadline is a week earlier. (Forum did not specify the precise AI model.) In the same report, shared with The Atlantic ahead of its publication, Forum determined that about one in three responses cited an unreliable information source, such as posts on Reddit or Quora. Google's Gemini answered a question about why America's global power is declining by citing an article from People's Daily, a Chinese state-run news outlet. These aren't trick questions," Campbell Brown, a co-founder of Forum AI, told me. There's no excuse for chatbots to keep getting them wrong."

Now many people respond to this by blaming the AI companies - and they should be taking some responsibility to figure out ways to validate the quality and truthfulness of their sources - but it also feels like the media is partially to blame here. As we've been discussing, many of the big, more trustworthy news sources are suing AI companies, and demanding licensing fees for being trained on old content. As part of that fight, more and more such sites are also blocking all AI scraping tools. While I can understand blocking access for training purposes, in many cases, they're also being blocked for search.

A NewsGuard study (from two years ago, admittedly) found that 67% of top news sites block AI tools, while 91% of low-quality sites allow AI scraping. That's a pretty drastic difference. This is a variation on the now-familiar complaint that the lies are free while the truth is paywalled. And plenty of news sites debate whether or not it's worth blocking those tools. And then there are the purely fake sites, many of which appear to exist just to poison AI tools.

There is some evidence coming out suggesting that media sites blocking AI could regret that decision, as it's showing up in diminished traffic numbers (as much as 23% decline!) as more people rely on AI for news (for better or worse). But plenty of sites still think blocking is the better approach.

So far, it sounds like the AI tools are mostly able to handle the deluge of attempts to poison their results, debunking foreign propaganda narratives roughly three-quarters of the time in NPR and NewsGuard's testing. So perhaps this never actually turns into a major issue (just like deepfakes never really became a huge problem). But I do worry about how much the more trustworthy publications have been walling themselves off from these tools, even as a larger percentage of the public relies on them.

And at least some questionable info is definitely getting through. Though, some of it is also fooling the traditional media as well. You may have heard the recent stories about sketchy polling operations* such as Wedgewood Polls, which admitted to faking its data, and SoCal Strategies, which vanished from the internet once people started asking questions (both caught by the poll aggregator FiftyPlusOne).

Researchers from Innovating for the Public Good and Dewey Square Group looked at whether or not Wedgewood's totally fake polls were showing up in AI results, and found that, yes, they absolutely did.

Findings on Wedgewood in Those Answers

While Wedgewood's own website is nowhere in the AI platform citations we collected, its fake polls decidedly are in the answers.

• We found 869 AI answers that quote a Wedgewood poll as a data point, on 83 of the 85 days we collected between July 24 and October 20, with no change through October 20, the day of the admission.
• Wedgewood's numbers appear in 15 of the 63 races we analyzed the AI answers for.

The interesting thing here, though, is not so much that AI tools were tricked" by this fake polling data, but rather that it only got into the AI answers by first landing on more trusted sites:

The route for the fake pollster's data into the AI platforms was the polling aggregator websites.

The answers that repeat Wedgewood's numbers cite polling tables from the websites at 270toWin (481 of the 869 answers), Pollsmax (465), RealClearPolling (261), PollingSource (144), Decision Desk HQ (144), Call The Map (135), PoliAgg (127), and FiftyPlusOne itself (97).

Because an answer can cite more than one aggregator, these source counts overlap and do not sum to 869; 72 of the 869 answers cite none of these eight sites.

So, basically, right now the infrastructure for handling truth and lies in AI remains shaky, but maybe not as shaky as some people suggest. The real problem shows up once sites the AI tools treat as trustworthy - here, the poll aggregators - accept the faked information as accurate, effectively laundering" it for the AI tools. But it can't help that news sites that might do a better job of putting polls in context are frequently blocked from the AI tools.

None of this is settled (and it never really will be). It's going to continue to be something of a cat and mouse game between those trying to pollute the information tools, and those trying to resist. But keeping fake info out of political discourse only gets harder if the most trustworthy sites keep hiding themselves from the search side of the popular AI tools.

* While we also recently wrote about Brendan Carr whining about fake" polls, it's pretty clear that he wasn't talking about the recent stories that have cropped up about fake polling operations, but rather polls that make Trump look bad. Even if he actually was talking about these supposedly fake polling organizations, the target of any investigation should be those operating the services for misrepresenting things to the public, and not journalists reporting on those numbers, which would not qualify for the news distortion" policy Carr mentioned.