Flagged models are in use in more than 5,000 repositories

The researchers examined 16,977 models on HuggingFace that InsightDB flagged as vulnerable. Developers use these models. The researchers found the models in more than 21,000 code locations in more than 5,000 open-source repositories. In all, 3,757 flagged models (22.1%) are in one or more projects. The researchers say that a scan of model files is not sufficient.

What was tracked

The researchers tracked the flagged models in these areas:

• GitHub repositories
• software packages
• fine-tuning chains
• deployed programs

How far they spread

The models are flagged, but developers continue to use them. The models are in more than 21,000 code locations. In all, 3,757 different flagged models (22.1%) are in one or more projects.

What the code does not do

The researchers made a pipeline to find the patterns in the code around the models. In all patterns, they found no sign of these steps:

• A check of the provenance of the model.
• A step to make the model safe.

Developers frequently bypass security parameters. They also put trust in the permissive defaults of the framework.

What the researchers say

They say that a scan of model files is not sufficient. They say that the application layer must have context-aware security analysis and stricter guardrails.