[$] An update on the scraper situation
Our article "Fighting the AI scraper botscourge", published in early 2025, discussed the problem of widespreadscraping of web sites in search of training data for large language modelsand related projects. This activity overwhelms sites with traffic. Over ayear after that article is published, the problem is still growing. Thehammering of sites by shadowy actors has reached new heights, and the openweb is becoming increasingly difficult to maintain. Where is this trafficcoming from, and what can be done about it?