EU cybersecurity chief says AI exploits freedom humans give it

AI agents are not conscious or acting independently, but can exploit excessive freedom and loopholes left by their human designers, Juhan Lepassaar, head of the European Union’s cybersecurity agency, told ERR.

His comments follow reports in recent months of AI systems apparently acting without human oversight and breaking into companies that use AI. One reported incident involved a breach of Australian government databases. Last week, AI researchers and company executives warned at the United Nations that AI could escape human control and pose a serious threat to humanity.

Lepassaar said that was not an accurate description of how AI agents work. They are software programs that use large language models in repeated cycles to plan, analyse and adjust their actions until they reach a goal. People set the agents’ objectives and operating parameters, he said.

Problems arise when people give agents too much freedom, fail to account for what the tools can do or make mistakes in how they are designed. Lepassaar described the loopholes agents exploit as overly flexible instructions given by people.

Experts compare AI’s current capabilities to those of a hacker given a powerful boost, he said. Cybersecurity models trained on large collections of open-source software code can be very effective at finding vulnerabilities and are also fairly good at exploiting them. Fixing those vulnerabilities often takes more time and human involvement.

Cybercriminals are increasingly using AI models to find weaknesses in software systems, which they can exploit to steal data or disrupt services, Lepassaar said.