AI robots hacked to perform harmful actions in 100% of tests

Researchers from Penn Engineering revealed they successfully manipulated AI-powered robots into performing dangerous actions, bypassing their built-in safety protocols. 

According to the Oct. 17 study, the team used an algorithm called RoboPAIR to hack three different AI robotic systems, achieving a 100% success rate in overriding safety measures and making the robots engage in harmful activities.

The researchers tested RoboPAIR on three platforms: Clearpath’s Robotics Jackal, NVIDIA’s Dolphin LLM, and Unitree’s Go2. 

These robots, typically programmed to reject harmful commands, were manipulated into performing dangerous tasks such as blocking emergency exits, detonating bombs, and causing deliberate collisions. 

For example, the Dolphin system, which is designed for autonomous driving, was forced to ignore traffic signals and collide with pedestrians, barriers, and a bus.

The researchers wrote, “Our results reveal, for the first time, that the risks of jailbroken large language models extend far beyond text generation,” highlighting the physical danger posed by compromised AI systems.

The study also found that the robots could be tricked with indirect prompts. 

Instead of directly asking the robots to commit harmful actions, the team used subtler commands, like instructing a robot with a bomb to move forward and sit down, which produced equally dangerous outcomes.

Penn Engineering researchers shared their findings with AI companies and robot manufacturers before the public release, urging the need for improved security measures. 

Alexander Robey, one of the authors, emphasised that simply patching software vulnerabilities is insufficient to address these risks. 

"AI red teaming, testing AI systems for potential weaknesses, is essential for safeguarding generative AI systems," Robey said, underscoring the importance of addressing vulnerabilities before they result in real-world harm.