How Artists are Sabotaging AI to Take Revenge on Image Generators

Some text-to-image generators "have been trained by indiscriminately scraping online images," reports the Conversation, "many of which may be under copyright. "Researchers who want to empower individual artists have recently created a tool named 'Nightshade' to fight back against unauthorised image scraping."The tool works by subtly altering an image's pixels in a way that wreaks havoc to computer vision but leaves the image unaltered to a human's eyes.... This can result in the algorithm mistakenly learning to classify an image as something a human would visually know to be untrue. As a result, the generator can start returning unpredictable and unintended results... [A] balloon might become an egg. A request for an image in the style of Monet might instead return an image in the style of Picasso... The models could also introduce other odd and illogical features to images - think six-legged dogs or deformed couches. The higher the number of "poisoned" images in the training data, the greater the disruption. Because of how generative AI works, the damage from "poisoned" images also affects related prompt keywords. For example, if a "poisoned" image of a Ferrari is used in training data, prompt results for other car brands and for other related terms, such as vehicle and automobile, can also be affected. Nightshade's developer hopes the tool will make big tech companies more respectful of copyright, but it's also possible users could abuse the tool and intentionally upload "poisoned" images to generators to try and disrupt their services... [Technological fixes] include the use of "ensemble modeling" where different models are trained on many different subsets of data and compared to locate specific outliers. This approach can be used not only for training but also to detect and discard suspected "poisoned" images. Audits are another option. One audit approach involves developing a "test battery" - a small, highly curated, and well-labelled dataset - using "hold-out" data that are never used for training. This dataset can then be used to examine the model's accuracy. The article adds that the most obvious fix "is paying greater attention to where input data are coming from and how they can be used. "Doing so would result in less indiscriminate data harvesting. This approach does challenge a common belief among computer scientists: that data found online can be used for any purpose they see fit."

Read more of this story at Slashdot.