A new tool could protect artists by sabotaging AI image generators

Artists may soon have a new weapon helping to protect their work from becoming machine learning fodder. Called Nightshade, the tool makes tiny changes to the pixels in a digital artwork to effectively "poison" it, rendering the image useless for the purposes of training AI.

MIT Technology Review reports that a team led by University of Chicago professor Ben Zhao submitted Nightshade for peer review at the USENIX computer security conference. The software works by making small edits to an image which, while invisible to the human eye, cause AI algorithms to completely misidentify them.

For example, an artist may paint a picture of a cat that can clearly be identified as a feline by any human or AI that examines it. However, upon applying Nightshade, humans will still see the same image while AI will incorrectly believe it's a dog. 

Flood the AI with enough bad training material like this, and soon a request for an image of a cat will cause it to generate a dog instead.

Mashable Light Speed Want more out-of-this world tech, space and science stories? Sign up for Mashable's weekly Light Speed newsletter. By clicking Sign Me Up, you confirm you are 16+ and agree to our Terms of Use and Privacy Policy. Thanks for signing up!

SEE ALSO: OpenAI is working on a tool to detect DALL-E 3 AI-generated images

Of course, just one poisoned image is unlikely to have a significant effect on an AI image generator's algorithm. Its training data would need to be tainted by thousands of altered images before a real impact is seen. 

However, AI image generators are known to indiscriminately scoop up thousands of new samples from the internet in order to refine their algorithm. If enough artists upload their images with Nightshade applied, it could eventually make such AI tools unusable. 

---

Related Stories

---

• Move over Bing Image Creator! Google announces new AI image generation tool
• Bing Image Creator users are doing some... interesting things with new DALL-E 3 upgrade
• OpenAI just revealed DALL-E 3, its newest image generator
• Artists on Tumblr created adorable avatars for new users
• MIDNATT is HYBE's first AI-powered artist. Here's how the project was brought to life.

It would also be incredibly difficult for AI companies to fix the issue, as each poisoned image must be individually identified and removed from their training pool. This could create a powerful incentive for such companies to think twice before dragging a trawl net through the internet and using artists' work without their explicit consent.

This isn't the first AI-disrupting tool Zhao's team has created. The group previously released Glaze, a tool that disguises an artist's personal style in a similar manner. Nightshade will eventually be integrated into Glaze, as well as made open source in order to allow others to build on their work protecting artists.