Model merging

Model merging combines models fine-tuned from one base by arithmetic on their weights: average them, add task vectors θₜ − θ_pre, or resolve conflicts first (TIES, DARE). One model, several skills, no extra training.

8 min read752 wordsadvanced0 viewsupdated 2026-10-21

Definition

Model merging Building one set of weights from several trained models with the same architecture, by arithmetic in parameter space and without further gradient training. The main recipes are weight averaging ("model soups", Wortsman et al., ICML 2022), task arithmetic (Ilharco et al., "Editing Models with Task Arithmetic", ICLR 2023), TIES-merging (Yadav et al., NeurIPS 2023) and DARE (Yu et al., "Language Models are Super Mario", 2023).

The usual situation: a team has one base model and several fine-tunes of it, say one for maths, one for code and one for chat, each made with Supervised fine-tuning (SFT) or LoRA. Serving three models costs three times the memory. Training one model on all three datasets needs the data, the compute and a new tuning run. Merging takes the finished checkpoints and produces one model with no gradient steps; a CPU is enough.

It only works when the models share a starting point. Fine-tuning moves the weights a short way from θpre\theta_{\text{pre}}, and models fine-tuned from the same base tend to stay in one low-loss basin, where points between them also have low loss. Two networks trained from different random initialisations do not: their hidden units are permuted relative to each other, and averaging them gives a broken model. That is why merging is a tool for fine-tunes of one base, not for unrelated models.

Task vectors

τt=θt−θpre,θmerged=θpre+λ∑t=1Tτt\tau_t=\theta_t-\theta_{\text{pre}},\qquad \theta_{\text{merged}}=\theta_{\text{pre}}+\lambda\sum_{t=1}^{T}\tau_t
θpre\theta_{\text{pre}}
weights of the shared base model
θt\theta_t
weights after fine-tuning on task t
τt\tau_t
task vector: what fine-tuning on task t changed
λ\lambda
scaling coefficient, tuned on held-out data; λ=1/T\lambda=1/T is the plain weight average

Ilharco et al. showed that these vectors behave like directions with meaning. Adding several gives a multi-task model. Subtracting one, θpre−λτt\theta_{\text{pre}}-\lambda\tau_t, reduces the behaviour it taught, for example toxic generation. Analogies τC+(τB−τA)\tau_C+(\tau_B-\tau_A) transfer a change from one task to a related one.

Plain addition has a flaw. A task vector is mostly small, redundant changes, and the few large ones carry the skill. When vectors are summed, one model's large update on a parameter can be cancelled by another model's update of the opposite sign, and the many small values pile up as noise. TIES and DARE both clean the vectors before adding them.

TIES: trim, elect sign, disjoint merge

τ^t=top⁡k(τt),γj=sgn⁡(∑tτ^t,j),τm,j=1∣Aj∣∑t∈Ajτ^t,j,Aj={t:sgn⁡(τ^t,j)=γj}\hat\tau_t=\operatorname{top}_k(\tau_t),\qquad \gamma_j=\operatorname{sgn}\Big(\sum_t\hat\tau_{t,j}\Big),\qquad \tau_{m,j}=\frac{1}{|A_j|}\sum_{t\in A_j}\hat\tau_{t,j},\quad A_j=\{t:\operatorname{sgn}(\hat\tau_{t,j})=\gamma_j\}
top⁡k\operatorname{top}_k
keep the k% of entries with the largest magnitude in each task vector, zero the rest (the paper's default keeps 20%)
γj\gamma_j
elected sign of parameter j: the sign of the summed trimmed values, so larger total mass wins
AjA_j
the models whose trimmed value on j is non-zero and agrees with the elected sign
τm,j\tau_{m,j}
mean over agreeing models only; the merge is θpre+λτm\theta_{\text{pre}}+\lambda\tau_m
model merging: combine fine-tuned weights that share one base, in parameter spaceθ_preshared baseθ₁ θ₂ θ₃fine-tuned ×3τₜ = θₜ − θ_pretask vectorsmerge(τ₁,τ₂,τ₃)avg / TA / TIESθ_pre + λ·τ_mno extra training1 · task vectors2 · trim top 50%3 · elect sign4 · disjoint mean, λ = 1τ₁τ₂τ₃τ₁τ₂τ₃ΣTIESavgΣ (TA)w1w2w3w4w5w60.80−0.100.500.05−0.600.200.600.40−0.700.020.10−0.05−0.200.300.60−0.900.050.100.8000.500−0.6000.600.40−0.7000000.300.60−0.90001.400.700.40−0.90−0.6000.700.350.55−0.90−0.6000.400.200.13−0.28−0.150.081.200.600.40−0.83−0.450.25+++−−·trim: keep each vector's 3 largest |τ| · elect: sign of the trimmed sum · mean over agreeing non-zerosw3: 0.50, −0.70, 0.60 → sign + → TIES (0.50 + 0.60)/2 = 0.55plain average (0.50 − 0.70 + 0.60)/3 = 0.13: the conflicting −0.70 cancels most of the updatew6: every entry was small (≤ 0.20) and trimmed, so TIES leaves the base value (τ = 0)DARE instead drops each entry with probability p and rescales the survivors by 1/(1 − p)
fig.Top: the merging pipeline. Bottom: TIES on three 6-parameter task vectors, keeping 3 of 6 entries per vector. Parameter w3 has updates 0.50, −0.70 and 0.60. The elected sign is + (trimmed sum 0.40), so TIES averages only 0.50 and 0.60 and gets 0.55. A plain average gives 0.13, because the conflicting −0.70 cancels most of the other two updates. w6 had only small entries (at most 0.20 in magnitude); all were trimmed and the parameter stays at the base value.
τ~t=(1−mt)⊙τt1−p,mt,j∼Bernoulli⁡(p)\tilde\tau_t=\frac{(1-m_t)\odot\tau_t}{1-p},\qquad m_{t,j}\sim\operatorname{Bernoulli}(p)
pp
drop rate: share of delta entries set to zero at random
1/(1−p)1/(1-p)
rescaling that keeps the expected value of each entry, like dropout at training time

DARE (Drop And REscale) sparsifies at random instead of by magnitude. Yu et al. report that for SFT deltas of LLMs, p=0.9p=0.9 (and in some cases 0.99) leaves performance almost unchanged, so most of each delta is redundant. The sparse vectors then collide less when added. DARE is a preprocessing step; the result is merged with task arithmetic or TIES ("DARE-TIES").

Methods and results to know

Model soups
Wortsman et al. 2022 averaged CLIP and ViT models fine-tuned with different hyperparameters. The greedy soup adds a model only if held-out accuracy improves; a ViT-G/14 soup reached 90.94% ImageNet top-1
SLERP
spherical interpolation between two models along the arc instead of the chord; common for pairs, not defined for more than two at once
Fisher / RegMean
weighted averages that use per-parameter importance (Matena & Raffel 2022) or layer activation statistics (Jin et al. 2023); they need some data, unlike TIES
TIES defaults
keep the top 20% of each task vector; λ chosen on validation data, with λ = 1 a reasonable default
Tooling
the open-source mergekit library (Arcee) implements linear, SLERP, task arithmetic, TIES, DARE and layer-stacking merges from a YAML config
Different inits
merging models trained from different random initialisations needs neuron alignment first (permutation matching, e.g. Git Re-Basin, Ainsworth et al. 2023)

Try it

TIES: trim to top 20%, elect sign, disjoint mean, λ = 1.00τ₁ trimτ₂ trimτ₃ trimmerged96 params, +/− · tick below = 1 of the task's 16 important · sign conflicts: 5gain kept: 1 − L_t(θ)/L_t(θ_pre), %-5005010083.7task 170.8task 269.6task 374.7meanmean gain kept vs λ (k = 20% for TIES)00.511.5task arithmetic · TIES · average 51.4%each fine-tuned model alone keeps 100% of its own task
method
fig.Three task vectors over 96 parameters; each task depends steeply on 16 of them and barely on the rest, where fine-tuning left small random drift. Bars show the share of each task's fine-tuning gain the merged model keeps. Default (TIES, keep 20%, λ = 1): 83.7%, 70.8% and 69.6%, mean 74.7%, with only 5 sign conflicts left after trimming (74 in the raw vectors). The plain average keeps 51.4% on average. Task arithmetic at λ = 1 keeps 57.7%; its best, λ = 0.7, keeps 70.8%, while TIES at λ = 0.8 keeps 76.7%. Push λ to 1.5 and task arithmetic falls to −21.8%: worse than the base model. Trimming too hard hurts too: keep 10% and TIES drops to 59.0%.
Parameterτ₁, τ₂, τ₃Average (λ = 1/3)Task arithmetic (Σ)TIES
w10.80, 0.60, −0.200.401.200.70 (−0.20 trimmed)
w2−0.10, 0.40, 0.300.200.600.35
w30.50, −0.70, 0.600.130.400.55 (−0.70 outvoted)
w40.05, 0.02, −0.90−0.28−0.83−0.90
w5−0.60, 0.10, 0.05−0.15−0.45−0.60
w60.20, −0.05, 0.100.080.250 (all trimmed)
Worked example: the diagram's three vectors merged four ways (λ = 1 for TA and TIES, keep 3 of 6). Computed in Python

Averaging shrinks every update by 1/T1/T: w4 is a large update that only task 3 made, and the average keeps a third of it (−0.28). Summing keeps magnitudes but adds the redundant 0.05 and 0.02. TIES keeps each model's large values at full size and drops both conflicts and noise.

python
1import numpy as np
2
3def ties_merge(base, finetuned, k=0.2, lam=1.0):
4 """base: {name: array}; finetuned: list of state dicts with the same keys and shapes."""
5 merged = {}
6 for name, w0 in base.items():
7 tau = np.stack([ft[name] - w0 for ft in finetuned]) # (T, ...) task vectors
8 flat = np.abs(tau.reshape(len(tau), -1))
9 n_keep = max(1, int(round(k * flat.shape[1])))
10 thr = -np.sort(-flat, axis=1)[:, n_keep - 1] # per-model magnitude cutoff
11 tau = np.where(np.abs(tau) >= thr.reshape(-1, *[1] * (tau.ndim - 1)), tau, 0.0) # 1. trim
12 sign = np.sign(tau.sum(axis=0)) # 2. elect sign
13 agree = (np.sign(tau) == sign) & (tau != 0)
14 count = np.maximum(agree.sum(axis=0), 1)
15 merged[name] = w0 + lam * (tau * agree).sum(axis=0) / count # 3. disjoint mean
16 return merged
17
18base = {"w": np.zeros(6)}
19fts = [{"w": np.array(v)} for v in ([0.8, -0.1, 0.5, 0.05, -0.6, 0.2],
20 [0.6, 0.4, -0.7, 0.02, 0.1, -0.05],
21 [-0.2, 0.3, 0.6, -0.9, 0.05, 0.1])]
22print(ties_merge(base, fts, k=0.5)["w"]) # [ 0.7 0.35 0.55 -0.9 -0.6 0. ]
TIES-merging over state dicts in numpy, checked on the worked example (run here). With PyTorch the same code applies to model.state_dict() tensors

A practical recipe

  1. 1Collect fine-tunes of one base checkpoint with the same tokenizer; record each parent's scores on its own evaluation set.
  2. 2Start with task arithmetic and sweep λ∈{0.3,0.5,0.7,1.0}\lambda\in\{0.3,0.5,0.7,1.0\} on held-out data; this is the baseline to beat.
  3. 3Try TIES (or DARE-TIES) with density k∈{0.1,0.2,0.5}k\in\{0.1,0.2,0.5\} and λ\lambda near 1; keep per-model weights if one task matters more.
  4. 4Pick the merge with the best worst-task score, not only the best mean, and re-run safety and refusal tests.
  5. 5If no merge is close enough to the parents, fall back to multi-task fine-tuning or serving separate adapters.

Use when

  • +No training data and no gradient steps; a CPU is enough
  • +One model to serve instead of several fine-tunes
  • +Task vectors can be subtracted to remove a behaviour
  • +Cheap to search: try many λ and k values against a held-out set

Avoid when

  • −Only for models fine-tuned from the same base (or carefully aligned)
  • −Usually below a model trained on all tasks together, and below each parent on its own task
  • −Interference grows with the number of merged tasks and with the size of the deltas
  • −Hyperparameters (λ, density) still need validation data

Check yourself

Two 7B models were trained from scratch with different random seeds on the same data. You average their weights 50/50. What do you expect?

Check yourself

Three trimmed task vectors have values +0.4, −0.9 and +0.3 on one parameter. What does TIES assign (λ = 1)?

Remember

  • ✓Task vector τt=θt−θpre\tau_t=\theta_t-\theta_{\text{pre}}; task arithmetic merges θpre+λ∑tτt\theta_{\text{pre}}+\lambda\sum_t\tau_t, and λ=1/T\lambda=1/T is the plain average.
  • ✓Merging works for fine-tunes of one shared base; models from different random inits need neuron alignment first.
  • ✓TIES: keep the top 20% magnitudes per vector, elect each parameter's sign by summed mass, average only the agreeing values.
  • ✓DARE drops a share p of delta entries at random (p = 0.9 often costs little) and rescales the rest by 1/(1 − p).
  • ✓Tune λ and density on held-out data, and run every parent's evaluation on the merged model.
← back to Generative AI roadmap