Pay-per-output? AI firms blindsided by beefed up robots.txt instructions.
by Ashley Belanger from Ars Technica - All content on (#6ZY9P)
Leading Internet companies and publishers-including Reddit, Yahoo, Quora, Medium, The Daily Beast, Fastly, and more-think there may finally be a solution to end AI crawlers hammering websites to scrape content without permission or compensation.
Announced Wednesday morning, the "Really Simply Licensing" (RSL) standard evolves robots.txt instructions by adding an automated licensing layer that's designed to block bots that don't fairly compensate creators for content.
Free for any publisher to use starting today, the RSL standard is an open, decentralized protocol that makes clear to AI crawlers and agents the terms for licensing, usage, and compensation of any content used to train AI, a press release noted.