# Model merging

> Model merging combines models fine-tuned from one base by arithmetic on their weights: average them, add task vectors θₜ − θ_pre, or resolve conflicts first (TIES, DARE). One model, several skills, no extra training.

Source: https://www.mlfoundry.in/learn/model-merging  
Section: Generative AI › Customization  
Level: advanced · 8 min read · updated 2026-10-21

**Model merging:** Building one set of weights from several trained models with the same architecture, by arithmetic in parameter space and without further gradient training. The main recipes are weight averaging ("model soups", Wortsman et al., ICML 2022), **task arithmetic** (Ilharco et al., "Editing Models with Task Arithmetic", ICLR 2023), **TIES-merging** (Yadav et al., NeurIPS 2023) and **DARE** (Yu et al., "Language Models are Super Mario", 2023).

The usual situation: a team has one base model and several fine-tunes of it, say one for maths, one for code and one for chat, each made with **Supervised fine-tuning (SFT)** or **LoRA**. Serving three models costs three times the memory. Training one model on all three datasets needs the data, the compute and a new tuning run. Merging takes the finished checkpoints and produces one model with no gradient steps; a CPU is enough.

It only works when the models share a starting point. Fine-tuning moves the weights a short way from $\theta_{\text{pre}}$, and models fine-tuned from the same base tend to stay in one low-loss basin, where points between them also have low loss. Two networks trained from different random initialisations do not: their hidden units are permuted relative to each other, and averaging them gives a broken model. That is why merging is a tool for fine-tunes of one base, not for unrelated models.

## Task vectors

$$
\tau_t=\theta_t-\theta_{\text{pre}},\qquad \theta_{\text{merged}}=\theta_{\text{pre}}+\lambda\sum_{t=1}^{T}\tau_t
$$

- $\theta_{\text{pre}}$: weights of the shared base model
- $\theta_t$: weights after fine-tuning on task t
- $\tau_t$: task vector: what fine-tuning on task t changed
- $\lambda$: scaling coefficient, tuned on held-out data; $\lambda=1/T$ is the plain weight average

Ilharco et al. showed that these vectors behave like directions with meaning. Adding several gives a multi-task model. Subtracting one, $\theta_{\text{pre}}-\lambda\tau_t$, reduces the behaviour it taught, for example toxic generation. Analogies $\tau_C+(\tau_B-\tau_A)$ transfer a change from one task to a related one.

Plain addition has a flaw. A task vector is mostly small, redundant changes, and the few large ones carry the skill. When vectors are summed, one model's large update on a parameter can be cancelled by another model's update of the opposite sign, and the many small values pile up as noise. TIES and DARE both clean the vectors before adding them.

## TIES: trim, elect sign, disjoint merge

$$
\hat\tau_t=\operatorname{top}_k(\tau_t),\qquad \gamma_j=\operatorname{sgn}\Big(\sum_t\hat\tau_{t,j}\Big),\qquad \tau_{m,j}=\frac{1}{|A_j|}\sum_{t\in A_j}\hat\tau_{t,j},\quad A_j=\{t:\operatorname{sgn}(\hat\tau_{t,j})=\gamma_j\}
$$

- $\operatorname{top}_k$: keep the k% of entries with the largest magnitude in each task vector, zero the rest (the paper's default keeps 20%)
- $\gamma_j$: elected sign of parameter j: the sign of the summed trimmed values, so larger total mass wins
- $A_j$: the models whose trimmed value on j is non-zero and agrees with the elected sign
- $\tau_{m,j}$: mean over agreeing models only; the merge is $\theta_{\text{pre}}+\lambda\tau_m$

> Figure: Top: the merging pipeline. Bottom: TIES on three 6-parameter task vectors, keeping 3 of 6 entries per vector. Parameter w3 has updates 0.50, −0.70 and 0.60. The elected sign is + (trimmed sum 0.40), so TIES averages only 0.50 and 0.60 and gets 0.55. A plain average gives 0.13, because the conflicting −0.70 cancels most of the other two updates. w6 had only small entries (at most 0.20 in magnitude); all were trimmed and the parameter stays at the base value.

$$
\tilde\tau_t=\frac{(1-m_t)\odot\tau_t}{1-p},\qquad m_{t,j}\sim\operatorname{Bernoulli}(p)
$$

- $p$: drop rate: share of delta entries set to zero at random
- $1/(1-p)$: rescaling that keeps the expected value of each entry, like dropout at training time

DARE (Drop And REscale) sparsifies at random instead of by magnitude. Yu et al. report that for SFT deltas of LLMs, $p=0.9$ (and in some cases 0.99) leaves performance almost unchanged, so most of each delta is redundant. The sparse vectors then collide less when added. DARE is a preprocessing step; the result is merged with task arithmetic or TIES ("DARE-TIES").

**Methods and results to know**

- **Model soups:** Wortsman et al. 2022 averaged CLIP and ViT models fine-tuned with different hyperparameters. The greedy soup adds a model only if held-out accuracy improves; a ViT-G/14 soup reached 90.94% ImageNet top-1
- **SLERP:** spherical interpolation between two models along the arc instead of the chord; common for pairs, not defined for more than two at once
- **Fisher / RegMean:** weighted averages that use per-parameter importance (Matena & Raffel 2022) or layer activation statistics (Jin et al. 2023); they need some data, unlike TIES
- **TIES defaults:** keep the top 20% of each task vector; λ chosen on validation data, with λ = 1 a reasonable default
- **Tooling:** the open-source mergekit library (Arcee) implements linear, SLERP, task arithmetic, TIES, DARE and layer-stacking merges from a YAML config
- **Different inits:** merging models trained from different random initialisations needs neuron alignment first (permutation matching, e.g. Git Re-Basin, Ainsworth et al. 2023)

## Try it

> Figure: Three task vectors over 96 parameters; each task depends steeply on 16 of them and barely on the rest, where fine-tuning left small random drift. Bars show the share of each task's fine-tuning gain the merged model keeps. Default (TIES, keep 20%, λ = 1): 83.7%, 70.8% and 69.6%, mean 74.7%, with only 5 sign conflicts left after trimming (74 in the raw vectors). The plain average keeps 51.4% on average. Task arithmetic at λ = 1 keeps 57.7%; its best, λ = 0.7, keeps 70.8%, while TIES at λ = 0.8 keeps 76.7%. Push λ to 1.5 and task arithmetic falls to −21.8%: worse than the base model. Trimming too hard hurts too: keep 10% and TIES drops to 59.0%.

Worked example: the diagram's three vectors merged four ways (λ = 1 for TA and TIES, keep 3 of 6). Computed in Python

| Parameter | τ₁, τ₂, τ₃ | Average (λ = 1/3) | Task arithmetic (Σ) | TIES |
| --- | --- | --- | --- | --- |
| w1 | 0.80, 0.60, −0.20 | 0.40 | 1.20 | 0.70 (−0.20 trimmed) |
| w2 | −0.10, 0.40, 0.30 | 0.20 | 0.60 | 0.35 |
| w3 | 0.50, −0.70, 0.60 | 0.13 | 0.40 | 0.55 (−0.70 outvoted) |
| w4 | 0.05, 0.02, −0.90 | −0.28 | −0.83 | −0.90 |
| w5 | −0.60, 0.10, 0.05 | −0.15 | −0.45 | −0.60 |
| w6 | 0.20, −0.05, 0.10 | 0.08 | 0.25 | 0 (all trimmed) |

Averaging shrinks every update by $1/T$: w4 is a large update that only task 3 made, and the average keeps a third of it (−0.28). Summing keeps magnitudes but adds the redundant 0.05 and 0.02. TIES keeps each model's large values at full size and drops both conflicts and noise.

```python
import numpy as np

def ties_merge(base, finetuned, k=0.2, lam=1.0):
    """base: {name: array}; finetuned: list of state dicts with the same keys and shapes."""
    merged = {}
    for name, w0 in base.items():
        tau = np.stack([ft[name] - w0 for ft in finetuned])          # (T, ...) task vectors
        flat = np.abs(tau.reshape(len(tau), -1))
        n_keep = max(1, int(round(k * flat.shape[1])))
        thr = -np.sort(-flat, axis=1)[:, n_keep - 1]                  # per-model magnitude cutoff
        tau = np.where(np.abs(tau) >= thr.reshape(-1, *[1] * (tau.ndim - 1)), tau, 0.0)   # 1. trim
        sign = np.sign(tau.sum(axis=0))                               # 2. elect sign
        agree = (np.sign(tau) == sign) & (tau != 0)
        count = np.maximum(agree.sum(axis=0), 1)
        merged[name] = w0 + lam * (tau * agree).sum(axis=0) / count   # 3. disjoint mean
    return merged

base = {"w": np.zeros(6)}
fts = [{"w": np.array(v)} for v in ([0.8, -0.1, 0.5, 0.05, -0.6, 0.2],
                                    [0.6, 0.4, -0.7, 0.02, 0.1, -0.05],
                                    [-0.2, 0.3, 0.6, -0.9, 0.05, 0.1])]
print(ties_merge(base, fts, k=0.5)["w"])   # [ 0.7   0.35  0.55 -0.9  -0.6   0.  ]
```

TIES-merging over state dicts in numpy, checked on the worked example (run here). With PyTorch the same code applies to model.state_dict() tensors

## A practical recipe

1. Collect fine-tunes of **one** base checkpoint with the same tokenizer; record each parent's scores on its own evaluation set.
2. Start with task arithmetic and sweep $\lambda\in\{0.3,0.5,0.7,1.0\}$ on held-out data; this is the baseline to beat.
3. Try TIES (or DARE-TIES) with density $k\in\{0.1,0.2,0.5\}$ and $\lambda$ near 1; keep per-model weights if one task matters more.
4. Pick the merge with the best worst-task score, not only the best mean, and re-run safety and refusal tests.
5. If no merge is close enough to the parents, fall back to multi-task fine-tuning or serving separate adapters.

> **Check compatibility before merging:** All inputs must have identical architecture, parameter names and tokenizer, and be fine-tuned from the same base checkpoint. A chat model whose vocabulary gained new special tokens has a different embedding matrix shape; mergekit and similar tools need an explicit rule for those rows. Merge LoRA adapters into their base first, or merge the low-rank deltas $BA$, not $A$ and $B$ separately: averaging the factors does not average their product.

> **Evaluate the merge like a new model:** A merged model can pass a quick chat test and still lose a skill, or a safety behaviour from one parent. Run every parent's evaluation on the merge, compare with the parents and with the base, and sweep $\lambda$ and the density $k$ on held-out data. Leaderboard gains from merges can also come from benchmark contamination in one parent; see **LLM evaluation and benchmarking**.

**Strengths**

- No training data and no gradient steps; a CPU is enough
- One model to serve instead of several fine-tunes
- Task vectors can be subtracted to remove a behaviour
- Cheap to search: try many λ and k values against a held-out set

**Limitations**

- Only for models fine-tuned from the same base (or carefully aligned)
- Usually below a model trained on all tasks together, and below each parent on its own task
- Interference grows with the number of merged tasks and with the size of the deltas
- Hyperparameters (λ, density) still need validation data

**Question:** Two 7B models were trained from scratch with different random seeds on the same data. You average their weights 50/50. What do you expect?

- A. About the quality of either parent, slightly better
- B. A much worse model, because their hidden units are not aligned
- C. Exactly the quality of the better parent
- D. The same as TIES-merging them

**Answer:** B. A much worse model, because their hidden units are not aligned — Independent training runs end in different basins whose neurons are permuted relative to each other, so the midpoint has high loss. Merging works for fine-tunes of one base because they stay in that base's basin. Aligning the neurons first (permutation matching) can partly fix this.

**Question:** Three trimmed task vectors have values +0.4, −0.9 and +0.3 on one parameter. What does TIES assign (λ = 1)?

- A. −0.067 (the mean of all three)
- B. +0.35
- C. −0.9
- D. 0, because the signs conflict

**Answer:** C. −0.9 — The trimmed sum is −0.2, so the elected sign is −. Only the −0.9 entry agrees, and the disjoint mean over agreeing entries is −0.9. Sign election weighs by magnitude, not by number of votes.

**Key points**

- Task vector $\tau_t=\theta_t-\theta_{\text{pre}}$; task arithmetic merges $\theta_{\text{pre}}+\lambda\sum_t\tau_t$, and $\lambda=1/T$ is the plain average.
- Merging works for fine-tunes of one shared base; models from different random inits need neuron alignment first.
- TIES: keep the top 20% magnitudes per vector, elect each parameter's sign by summed mass, average only the agreeing values.
- DARE drops a share p of delta entries at random (p = 0.9 often costs little) and rescales the rest by 1/(1 − p).
- Tune λ and density on held-out data, and run every parent's evaluation on the merged model.

---

From MLFoundry (https://www.mlfoundry.in). The web page adds interactive figures and diagrams.
