Pay up or stop scraping: Cloudflare program charges bots for each crawl
by Ashley Belanger from Ars Technica - All content on (#6YBKB)
Cloudflare is now experimenting with tools that will allow content creators to charge a fee to AI crawlers to scrape their websites.
In a blog Tuesday, Cloudflare explained that its "pay-per-crawl" feature is currently in a private beta. A small number of publishers and content creators will participate in the experiment. Each publisher will be able to set their own prices that bots must pay before scraping content, Cloudflare said.
Matthew Prince, CEO of Cloudflare, said the feature would ensure that the Internet as we know it will survive "the age of AI."