LLMjacking

How it works

LLMjacking usually starts with stolen credentials. In the first case Sysdig reported, in May 2024, they came from a server running an unpatched web framework. Sysdig later named software packages in public repositories as another popular source of keys.

Next the attacker tests the keys without paying for real answers. On Amazon Bedrock, Sysdig watched attackers send a request with an invalid setting. The error that came back showed the key could reach the model. They also checked whether prompt logging was on. Their code referred to OAI Reverse Proxy, a tool that says it will not use AWS keys with logging enabled. By September 2024 attackers were enabling models themselves and switching logging off.

Some attackers use the access themselves; others sell it. Sysdig found working keys pooled behind reverse proxies: servers that pass buyers’ prompts through the keys without showing them. One proxy sold 30 days of access for $30. Sysdig also says credentials are tested for LLM access before being sold.

Why it matters

The account owner pays. Sysdig’s worst-case estimate for one Claude 2.x model, at its quota limit in four AWS regions, was over $46,000 a day in 2024. For newer models such as Claude 3 Opus, Sysdig put it at two to three times that, over $100,000 a day. Attackers who use up the quota can also lock the owner out.

The account also carries prompts its owner never sent. Among the prompts Sysdig could read, from attackers who had not checked for logging, about 95% were roleplay, mostly adult. One user was in Russia, where sanctions cut off legal access. Microsoft describes a similar scheme against its generative AI services, without using the word: the Storm-2139 network used “exposed customer credentials scraped from public sources”, “altered the capabilities of these services”, and resold access with instructions for producing “non-consensual intimate images of celebrities”.

Where definitions disagree

Sysdig, which coined the term, defines it by access: “obtaining access to an LLM illegally”, most often with stolen credentials. MITRE ATLAS follows Sysdig’s reporting in case study AML.CS0030.

Wiz scopes it to “an enterprise’s cloud-based LLMs” and says attackers “often sell LLM access” rather than use it themselves. Wiz’s defences include adversarial training and reinforcement learning from human feedback, which change how the model behaves. Sysdig’s defences are about the keys: keep them out of code, rotate them, and watch how they are used.

LLMjacking is not cost harvesting, though both leave a bill. ATLAS defines cost harvesting by the intent to increase the victim’s costs. Sysdig describes LLMjacking attackers as wanting the model, for their own use or to sell.

Questions and answers

What is LLMjacking?

LLMjacking is an attack in which someone gains unauthorized access to a victim's paid language model services, most often with stolen cloud or API credentials, and uses or resells the access while the account owner pays. Sysdig's Threat Research Team coined the term in May 2024.

How do attackers get the credentials?

In Sysdig's first case, from a server running a vulnerable version of Laravel, CVE-2021-3129. Sysdig later named software packages in public repositories as another popular source. Microsoft says the Storm-2139 network used "exposed customer credentials scraped from public sources".

How much can LLMjacking cost the victim?

Sysdig estimated over $46,000 a day for one Claude 2.x model at its quota limit in four AWS regions in May 2024, worked out from quotas and list prices. In September 2024 it put newer models such as Claude 3 Opus at two to three times that, over $100,000 a day. Neither figure is a measured loss.

How do you defend against LLMjacking?

Treat it as credential theft. Sysdig recommends keeping keys out of code and repositories, using short-lived credentials, rotating keys, scanning for exposed secrets, and monitoring for unexpected model use. Leaving models disabled is not a safeguard: Sysdig says the activation request is "often more of a speed bump for attackers rather than a blocker", and has seen attackers enable models themselves.