Steering vector

How a steering vector is made and used

The usual method has three steps. Turner and colleagues (2023) and Panickssery and colleagues (2023) use a version of it.

1. Collect contrast examples. Make pairs of prompts, one with the behaviour and one without. Turner uses a pair as small as “Love” and “Hate”.
2. Subtract the activations. Send the two sets through the language model. Get the difference of the internal activations at one layer. Contrastive Activation Addition (CAA) calculates the mean difference for many pairs.
3. Add the vector at inference time. Add the difference to the activations of a new prompt. A positive multiplier makes the behaviour stronger. A negative multiplier makes it weaker.

Why steering vectors are a security subject

Defenders and attackers can use steering vectors:

• Monitor. Chen and colleagues (2025) use persona vectors to “monitor fluctuations in the Assistant’s personality at deployment time”.
• Examine training data. The same vectors can “flag training data that will produce undesirable personality changes” before fine tuning.
• Remove safety. Arditi and colleagues (2024) removed one direction and the models did not refuse dangerous instructions. They tell of it as a white-box jailbreak. Also see abliterated vs uncensored models.

In practice

Paper	Model	Behaviour
Subramani et al., 2022	Pretrained decoders	Sentiment change
Turner et al., 2023	LLaMA-3, OPT	Sentiment, toxicity, topic
Panickssery et al., 2023	Llama 2 Chat	Behaviour question tests
Arditi et al., 2024	13 open chat models	Refusal
Chen et al., 2025	Qwen 2.5-7B-Instruct, Llama-3.1-8B-Instruct	Evil, sycophancy, hallucination

Research on AI agent memory also uses the method. Neural Procedural Memory (Zhao and colleagues, June 2026) keeps skills from earlier tasks as steering vectors, not as text instructions. The authors report results “comparably to baselines using explicit textual instructions”.

Trade-offs

Research shows three limits:

1. The effect is not stable. Tan and colleagues (2024) found that “steerability is highly variable across different inputs”. For some concepts, the vectors are “brittle to reasonable changes in the prompt”.
2. Steering can make the model worse. Anthropic reports that steering at inference time to remove a trait “came with a side effect of making the model less intelligent”. CAA reports only a small decrease, on Llama 2 Chat.
3. A test is not proof. A vector can stop a behaviour on a test set. Because of limit 1, it can be unsatisfactory on new inputs.

Questions and answers

What is the difference between a steering vector and fine-tuning?

Fine-tuning changes the weights of a model with more training. A steering vector keeps the weights and adds numbers to the activations while the model operates. Turner and colleagues report that their ActAdd method needs no machine optimization and works with one pair of prompts. Panickssery and colleagues report that steering also works on top of fine-tuning.

Can an attacker use a steering vector?

Yes, if the attacker can change the activations of the model. Arditi and colleagues found one refusal direction in 13 open chat models of a maximum of 72B parameters. When they removed it, the models did not refuse dangerous instructions. They tell of this as a white-box jailbreak.