Definition
Model merging Building one set of weights from several trained models with the same architecture, by arithmetic in parameter space and without further gradient training. The main recipes are weight averaging ("model soups", Wortsman et al., ICML 2022), task arithmetic (Ilharco et al., "Editing Models with Task Arithmetic", ICLR 2023), TIES-merging (Yadav et al., NeurIPS 2023) and DARE (Yu et al., "Language Models are Super Mario", 2023).
The usual situation: a team has one base model and several fine-tunes of it, say one for maths, one for code and one for chat, each made with Supervised fine-tuning (SFT) or LoRA. Serving three models costs three times the memory. Training one model on all three datasets needs the data, the compute and a new tuning run. Merging takes the finished checkpoints and produces one model with no gradient steps; a CPU is enough.
It only works when the models share a starting point. Fine-tuning moves the weights a short way from , and models fine-tuned from the same base tend to stay in one low-loss basin, where points between them also have low loss. Two networks trained from different random initialisations do not: their hidden units are permuted relative to each other, and averaging them gives a broken model. That is why merging is a tool for fine-tunes of one base, not for unrelated models.
Task vectors
- weights of the shared base model
- weights after fine-tuning on task t
- task vector: what fine-tuning on task t changed
- scaling coefficient, tuned on held-out data; is the plain weight average
Ilharco et al. showed that these vectors behave like directions with meaning. Adding several gives a multi-task model. Subtracting one, , reduces the behaviour it taught, for example toxic generation. Analogies transfer a change from one task to a related one.
Plain addition has a flaw. A task vector is mostly small, redundant changes, and the few large ones carry the skill. When vectors are summed, one model's large update on a parameter can be cancelled by another model's update of the opposite sign, and the many small values pile up as noise. TIES and DARE both clean the vectors before adding them.
TIES: trim, elect sign, disjoint merge
- keep the k% of entries with the largest magnitude in each task vector, zero the rest (the paper's default keeps 20%)
- elected sign of parameter j: the sign of the summed trimmed values, so larger total mass wins
- the models whose trimmed value on j is non-zero and agrees with the elected sign
- mean over agreeing models only; the merge is
- drop rate: share of delta entries set to zero at random
- rescaling that keeps the expected value of each entry, like dropout at training time
DARE (Drop And REscale) sparsifies at random instead of by magnitude. Yu et al. report that for SFT deltas of LLMs, (and in some cases 0.99) leaves performance almost unchanged, so most of each delta is redundant. The sparse vectors then collide less when added. DARE is a preprocessing step; the result is merged with task arithmetic or TIES ("DARE-TIES").
Methods and results to know
- Model soups
- Wortsman et al. 2022 averaged CLIP and ViT models fine-tuned with different hyperparameters. The greedy soup adds a model only if held-out accuracy improves; a ViT-G/14 soup reached 90.94% ImageNet top-1
- SLERP
- spherical interpolation between two models along the arc instead of the chord; common for pairs, not defined for more than two at once
- Fisher / RegMean
- weighted averages that use per-parameter importance (Matena & Raffel 2022) or layer activation statistics (Jin et al. 2023); they need some data, unlike TIES
- TIES defaults
- keep the top 20% of each task vector; λ chosen on validation data, with λ = 1 a reasonable default
- Tooling
- the open-source mergekit library (Arcee) implements linear, SLERP, task arithmetic, TIES, DARE and layer-stacking merges from a YAML config
- Different inits
- merging models trained from different random initialisations needs neuron alignment first (permutation matching, e.g. Git Re-Basin, Ainsworth et al. 2023)
Try it
| Parameter | τ₁, τ₂, τ₃ | Average (λ = 1/3) | Task arithmetic (Σ) | TIES |
|---|---|---|---|---|
| w1 | 0.80, 0.60, −0.20 | 0.40 | 1.20 | 0.70 (−0.20 trimmed) |
| w2 | −0.10, 0.40, 0.30 | 0.20 | 0.60 | 0.35 |
| w3 | 0.50, −0.70, 0.60 | 0.13 | 0.40 | 0.55 (−0.70 outvoted) |
| w4 | 0.05, 0.02, −0.90 | −0.28 | −0.83 | −0.90 |
| w5 | −0.60, 0.10, 0.05 | −0.15 | −0.45 | −0.60 |
| w6 | 0.20, −0.05, 0.10 | 0.08 | 0.25 | 0 (all trimmed) |
Averaging shrinks every update by : w4 is a large update that only task 3 made, and the average keeps a third of it (−0.28). Summing keeps magnitudes but adds the redundant 0.05 and 0.02. TIES keeps each model's large values at full size and drops both conflicts and noise.
1import numpy as np23def ties_merge(base, finetuned, k=0.2, lam=1.0):4 """base: {name: array}; finetuned: list of state dicts with the same keys and shapes."""5 merged = {}6 for name, w0 in base.items():7 tau = np.stack([ft[name] - w0 for ft in finetuned]) # (T, ...) task vectors8 flat = np.abs(tau.reshape(len(tau), -1))9 n_keep = max(1, int(round(k * flat.shape[1])))10 thr = -np.sort(-flat, axis=1)[:, n_keep - 1] # per-model magnitude cutoff11 tau = np.where(np.abs(tau) >= thr.reshape(-1, *[1] * (tau.ndim - 1)), tau, 0.0) # 1. trim12 sign = np.sign(tau.sum(axis=0)) # 2. elect sign13 agree = (np.sign(tau) == sign) & (tau != 0)14 count = np.maximum(agree.sum(axis=0), 1)15 merged[name] = w0 + lam * (tau * agree).sum(axis=0) / count # 3. disjoint mean16 return merged1718base = {"w": np.zeros(6)}19fts = [{"w": np.array(v)} for v in ([0.8, -0.1, 0.5, 0.05, -0.6, 0.2],20 [0.6, 0.4, -0.7, 0.02, 0.1, -0.05],21 [-0.2, 0.3, 0.6, -0.9, 0.05, 0.1])]22print(ties_merge(base, fts, k=0.5)["w"]) # [ 0.7 0.35 0.55 -0.9 -0.6 0. ]A practical recipe
- 1Collect fine-tunes of one base checkpoint with the same tokenizer; record each parent's scores on its own evaluation set.
- 2Start with task arithmetic and sweep on held-out data; this is the baseline to beat.
- 3Try TIES (or DARE-TIES) with density and near 1; keep per-model weights if one task matters more.
- 4Pick the merge with the best worst-task score, not only the best mean, and re-run safety and refusal tests.
- 5If no merge is close enough to the parents, fall back to multi-task fine-tuning or serving separate adapters.
Use when
- +No training data and no gradient steps; a CPU is enough
- +One model to serve instead of several fine-tunes
- +Task vectors can be subtracted to remove a behaviour
- +Cheap to search: try many λ and k values against a held-out set
Avoid when
- −Only for models fine-tuned from the same base (or carefully aligned)
- −Usually below a model trained on all tasks together, and below each parent on its own task
- −Interference grows with the number of merged tasks and with the size of the deltas
- −Hyperparameters (λ, density) still need validation data
Check yourself
Two 7B models were trained from scratch with different random seeds on the same data. You average their weights 50/50. What do you expect?
Check yourself
Three trimmed task vectors have values +0.4, −0.9 and +0.3 on one parameter. What does TIES assign (λ = 1)?
Remember
- ✓Task vector ; task arithmetic merges , and is the plain average.
- ✓Merging works for fine-tunes of one shared base; models from different random inits need neuron alignment first.
- ✓TIES: keep the top 20% magnitudes per vector, elect each parameter's sign by summed mass, average only the agreeing values.
- ✓DARE drops a share p of delta entries at random (p = 0.9 often costs little) and rescales the rest by 1/(1 − p).
- ✓Tune λ and density on held-out data, and run every parent's evaluation on the merged model.