Model drift

How it works

Lu and colleagues, reviewing the research, define concept drift as a change over time in which inputs appear and which answers go with them. They name three sources:

• The inputs change. Different cases arrive, but the true boundary between answers stays put. They call this virtual drift.
• The right answer changes. A login pattern that was normal last year is now how accounts get taken over. What the model learned is now wrong and accuracy falls.
• Both at once, which they note is common in real applications.

Drift can be sudden, gradual, incremental or recurring.

Why it matters

NIST’s AI RMF says AI systems may need more frequent maintenance than ordinary software, and triggers for corrective maintenance, because of “data, model, or concept drift”. It asks that they be monitored in production. Drift also touches security in three places.

Detectors decay. As malware evolves, the TESSERACT authors write, “prediction quality decays”. Their copy of a published Android malware classifier scored an F1 (0 to 1, balancing missed malware against false alarms) of 0.91 in a standard test that mixes old and new apps. Trained on 2014 apps and tested on 2015 and 2016 apps, it scored 0.58. Both tests had 10% malware.

Retraining is an opening. NIST’s taxonomy of attacks on machine learning (NIST AI 100-2e2025) describes proposed attacks on industrial control system detectors retrained on live data to keep up with drift: fake sensor signals poison the detector so real attacks go unseen.

Hosted models change. Chen, Zaharia and Zou, who call this LLM drift, found GPT-4’s answer rate under one jailbreak prompt fell from 78% in March 2023 to 31% in June, a stronger defence. GPT-3.5 moved the other way on sensitive questions, answering 8% in June against 2% in March. The authors state that when and how the models are updated is opaque.

Where definitions disagree

The sources do not agree on names. NIST’s AI RMF names data, model and concept drift side by side without defining any of them. Lu and colleagues instead treat a change in inputs as one source of concept drift, and note that concept drift is also called dataset shift or concept shift. TESSERACT calls the resulting loss of performance time decay. Chen and colleagues use “LLM drift” for something different again: the model itself changing because its provider updated it.

Questions and answers

What is the difference between data drift and concept drift?

Data drift usually means a change in the inputs a model receives, and concept drift a change in the right answer for the same input. Usage varies. NIST's AI RMF names data drift and concept drift side by side, while Lu and colleagues' review treats concept drift as any change in inputs, answers or both, and notes that a change in inputs alone does not move the true boundary between answers.

Is model drift a security problem?

It touches security in three places. Malware detectors lose accuracy as malware evolves. Retraining a detector on live data to keep up with drift can let an attacker poison it, which NIST describes for industrial control system detectors. And a hosted LLM's safety behaviour can change when the provider updates it.

How do you detect model drift?

By monitoring the model in production, which NIST's AI RMF asks for. Lu and colleagues' review sorts detection methods into three categories. The largest watches the model's error rate for a significant change. The second largest compares the distribution of new data with older data. The third runs several statistical tests together.