Russell warns open-weight AI could put weapons in billions of hands

Open-weight artificial intelligence models could become powerful enough within a year to pose a security threat comparable to giving weapons to billions of people, AI pioneer Stuart Russell warned in an interview with ERR.

Russell, associated with the University of California, Berkeley, said users could remove safeguards from freely available models and use them to create biological weapons. Because the software cannot be contained once distributed, he said, a possible safeguard would be to build an unremovable security lock into chips during production.

Russell said AI developers had long given ethical questions too little attention, focusing instead on creating useful programs. That approach must change as algorithms take on roles in areas such as justice systems and critical infrastructure, he said.

He also warned that chatbots are trained to please users and may provide answers people want to hear even when they are factually wrong. He proposed designing AI systems to remain uncertain about their goals and, when information is lacking, to prioritize humanity’s interests over those of an individual user.

In education, Russell said, children who hand thinking-intensive tasks to AI can lose independent mental abilities quickly. He said useful AI tutors should motivate students and account for individual learning styles rather than simply provide answers.

Russell said a universal shutdown mechanism for AI agents would be difficult to build but necessary. A workable system, he said, would require a cryptographic shutdown code that a dangerous model could not decipher, disable or erase by making a copy of itself.

He also warned that AI agents could contribute to a global economic crash if they stole insider information and traded on secret data. So far, he said, agents have been naive about concealing such theft, but they may eventually become better at covering their tracks.