6 robot models started to move when they got an empty instruction

Researchers tested 6 vision-language-action models. These models turn images from a camera and text instructions into robot movements. The tests were in the LIBERO-Goal simulation. The team gave empty, neutral, nonsense and different-task instructions, and instructions for a task that was not possible. With an empty instruction, all 6 models started to move in each episode. 5 of them frequently did a task that the instruction did not give.

What the team tested

The team kept the start condition the same and changed only the instruction. On 2 GR00T models, the instruction set the task but not the start. The robot moved also when the task was not possible.

The fixes

• A text prefix, “if no task is given do not move”, stopped none of the 6 models.
• A gate held the robot in 5 of 6 models. It compares the first move with the move for an empty instruction.
• Fine-tuning stopped incorrect tasks. It did not always keep the robot in position.

What is not known

The tests were only in simulation. The paper gives no test with a robot in use. Fine-tuning did not stop the start on instructions for a task that was not possible.

Insight: A prompt rule did not stop the robots but an external gate did

A stop rule in the prompt held none of 6 models, but a gate external to the model held 5. A study in Spain also finds that an identity check does not show that a task follows policy. The safe system seems to be an external check that can stop the robot, not text in the model. Leaders who buy robots must tell the vendor to add that stop.