Model merging

How model merging works

Model merging works on the numbers in the models, not on data. Most methods start from one base model and some fine-tuned copies of it.

1. Find the change. For each copy, subtract the base weights from the fine-tuned weights. Ilharco and colleagues name this difference a “task vector”.
2. Clean the change. Remove small values. TIES-Merging resets small changes. DARE can drop “90% or even 99%” of them.
3. Mix. Add or average the changes. TIES-Merging also keeps only the values that agree in sign.
4. Add back. Put the mixed change on the base model.

Four frequent methods

Method	Year	What it does
Model soups	2022	Averages the weights of models fine-tuned with different hyperparameters
Task arithmetic	2022	Adds or subtracts task vectors to add or remove a skill
TIES-Merging	2023	Sets small changes to zero and settles sign differences
DARE	2023	Drops most of each change at random and scales up the remaining values

Wortsman and colleagues tell why a simple average can work. Fine-tuned models “often appear to lie in a single low error basin”. Their best ViT-G soup got 90.94% top-1 accuracy on ImageNet (2022).

Why model merging is important

Model merging gives new models at a low cost. The MergeKit README says that a merge can operate “entirely on CPU”. The DARE authors merge “without retraining or GPUs”.

• Builders get one model with some skills and no more cost at use.
• The open model community uses it frequently. The MergeKit paper (March 2024) says “thousands of models have been merged”.
• Security teams get a new risk. A merged model takes weights from many parents, and the merge does not check them.

In practice

Three results are important for a team that uses or ships merged open-weight models:

1. Misalignment spreads. Hammoud and colleagues (2024) found that current merge methods often overlook safety alignment, “leading to highly misaligned models”.
2. Safe parts can make a dangerous merged model. Li and colleagues (2026) made source models that “remain individually benign but produce severely misaligned models when merged”. They tested 9 LLMs from 3 model families.
3. Tests on the parents are not sufficient. Test the merged model for model alignment and backdoors on its own.

Look at each parent model as part of the AI supply chain. Record the parents, their versions and the merge method. Then you can find the provenance of the result.

Software can also find merges automatically. The Evolutionary Model Merge method of Sakana AI (2024) used evolutionary algorithms. It made “a Japanese LLM with Math reasoning capabilities”.

Trade-offs

• One base is necessary. The MergeKit README says that methods “require matching tensor shapes”. Git Re-Basin (2022) aligned independently trained ResNet models on CIFAR-10. Merges of language models usually use one base.
• Interference. Two models can change one weight in opposite directions. TIES-Merging names “disagreement on the sign” as one cause.
• Weak parents. The soup increase came “only when all individual models achieve high accuracy”, Wortsman and colleagues report.
• Leaderboard scores. A DARE merge was first in its size (7B) on the Open LLM Leaderboard at the time of the paper. A high score does not show that the model is safe.

Questions and answers

What is the difference between model merging and an ensemble?

An ensemble keeps all the models and uses each one on each input. Thus the cost increases with each model. Model merging makes one set of weights before use. Wortsman and colleagues show that their model soups add no inference or memory cost.

Can a merged model be safe if each source model is safe?

Not always. Hammoud and colleagues (2024) found that merge methods that overlook safety alignment can give highly misaligned models. Li and colleagues (2026) made source models that are each benign but give a severely misaligned merged model. Test the merged model, not only its parents.

Can you merge two models with different architectures?

Not with the usual tools. The MergeKit README says that merge methods "require matching tensor shapes". Git Re-Basin aligns independently trained networks first. Merges of large language models usually start from one base model.