The CLIOPATRA attack gets private medical data from Clio

Researchers show CLIOPATRA, an attack on platforms that find insights in the chats of AI assistant users. The attacker puts malicious chats into the platform. These chats cause the platform to leak private data from the chat of a target user. In a test on Anthropic Clio, the attack got medical history in a maximum of 65% of the target chats.

How it works

The attacker carefully designs malicious chats and puts them into the system. These chats break more than one layer of protection. The system then leaks private data from the chat of the target user.

The test

The researchers did the test on Clio. The medical chats were synthetic chats that a computer made. These are the results:

• The attack got the medical history in a maximum of 65% of the target chats.
• Precision was almost 100%.
• The attack can hide the private data in the insights that the system makes.

Weak protections

The platforms use protections in layers, with heuristic methods. Examples are redaction of personal data and a privacy audit by a large language model. The researchers say that the audit is not reliable and does not find large leaks. They say that prompt injection has not had sufficient study as a risk in these systems.