Good news, everyone: “open-weight” AI models that are available to anyone to download and run are comically easy to poison.
Katie Paxton-Fear, a cybersecurity researcher at Semgrep, demonstrated this in an attack that took less than an hour and cost less than $100 to carry out, The Register reports, successfully manipulating the AI’s behavior by feeding it malicious data.
By training the model on just ten examples of poisoned material, the model started churning out new code that’s exposed to remote code execution, a vulnerability that allows hackers to run code on a person’s machine.
“I did a proper backdoor,” she triumphantly shared on social media.
