COMPASS finds if an image was in the training data of a model
On a medical test, COMPASS has 0.64 AUC and the strongest baseline stays at chance level.
Claimed, not confirmed
Researchers show COMPASS, an attack on vision language models. It finds if an image was in the training data of a model. It uses a known set of non-member data to set the level of noise and the direction of the score. On general-domain data, the AUC has a maximum of 0.88. On a medical test, the AUC is 0.64.
Sources
Posted
