# Open-weight vs closed models

> Open-weight models ship their parameters under a licence (Apache-2.0, MIT or a custom community licence) so you can self-host, fine-tune and keep data in-house; closed models are reached only through an API. Volume, privacy and licence terms decide.

Source: https://www.mlfoundry.in/learn/open-weight-vs-closed-models  
Section: Generative AI › Foundations of Generative AI  
Level: core · 8 min read · updated 2026-10-21

**Open-weight vs closed:** An **open-weight** model is one whose trained parameters can be downloaded and run by anyone who accepts its licence. A **closed** model is available only as a hosted service: you send requests to the provider's API and never receive the weights. Open weights are not the same as **open source AI**. The Open Source Initiative's Open Source AI Definition 1.0 (October 2024) also requires the training and inference code and enough information about the training data for a skilled person to build a substantially equivalent system.

The choice decides who runs the model, who sees the data, what you are allowed to change, and how the bill grows with traffic. It is usually made per workload, not per company. Many teams prototype on a closed API and move steady, high-volume or sensitive workloads to open weights once the requirements are clear.

> **Not legal advice:** This article summarises what common licences say in general terms. Licence texts change between model versions, and how they apply depends on your use and jurisdiction. Read the actual licence for the exact model version you use, and ask counsel for anything that matters.

## What you actually get

With open weights you get a file of tensors (commonly **safetensors**), a tokenizer and a config. An 8B model in bf16 is about 16 GB. You serve it yourself with an inference server such as vLLM, TGI or llama.cpp (see **Inference servers (vLLM, TGI)**). Nothing leaves your network, the version never changes unless you change it, and you can read logits, quantise (**Quantization**), fine-tune (**LoRA**, **QLoRA**) or merge models.

With a closed model you get an HTTPS endpoint, a price per token, rate limits and a contract. The provider handles GPUs, scaling, safety filtering and upgrades. Your prompts and data cross your boundary, and the contract and data-processing terms set how they are logged, retained or used. Models are retired on the provider's schedule, so outputs can change when a version is deprecated.

> Figure: Top: open weights. A 16 GB safetensors file for an 8B model runs on your own GPUs; data stays inside and you can fine-tune, quantise and pin versions, but you own operations, safety filters and licence compliance. Middle: openness is a spectrum, from a closed API through custom-licence and permissive (Apache-2.0, MIT) weights to open source AI with code and data information. Bottom: a closed API; no infrastructure, but data leaves under contract and versions are retired by the vendor.

## Licences

"Open weights" says nothing about what you may do with them; the licence does. Permissive software licences (Apache-2.0, MIT) allow commercial use, modification and redistribution with attribution. Custom model licences add conditions: caps on user numbers, acceptable-use policies, naming rules for derivatives, or limits on using outputs to train other models. Use-restriction licences (the RAIL family) attach behavioural restrictions that must be passed on to everyone downstream.

Examples of licence types, as of each model's release. Terms differ between versions of the same family; check the licence file shipped with the exact weights

| Licence | Example | Commercial use | Key conditions |
| --- | --- | --- | --- |
| Apache-2.0 | Mistral 7B, Mixtral 8×7B (2023) | yes | keep the licence and NOTICE file, mark changes; explicit patent grant that ends if you sue over patents |
| MIT | DeepSeek-R1 (2025) | yes | keep the copyright and licence notice |
| Llama 2 Community License | Llama 2 (2023) | yes, with limits | separate licence needed above 700M monthly active users; acceptable-use policy; no using outputs to improve other LLMs |
| CreativeML OpenRAIL-M | Stable Diffusion v1 (2022) | yes | use-based restrictions that every derivative and downstream user must also accept |
| Provider terms (closed) | hosted API models | per contract | no weights; usage policy, data retention and training-on-data terms set by the contract |

## The cost crossover

An API bill is linear in tokens. A self-hosted deployment is mostly fixed: GPUs are paid by the hour whether busy or idle, and someone must run them. One GPU serves a fixed number of tokens a month, so the self-hosted cost is a step function:

$$
C_\text{API}=\frac{V\,p}{10^6},\qquad C_\text{self}=\Big\lceil\frac{V}{\kappa}\Big\rceil\,h\cdot730+F,\qquad \kappa=\text{tps}\cdot u\cdot3600\cdot730
$$

- $V$: tokens per month; $p$ = API price per million tokens (blended input and output)
- $h$: price per GPU-hour; 730 hours per month, always on
- $\kappa$: tokens one GPU serves per month: batched throughput (tokens/s) × average utilisation $u$
- $F$: fixed monthly cost: engineering time, monitoring, redundancy

At high volume the per-token cost of self-hosting approaches the marginal rate $h\cdot730/\kappa$. If that rate is above the API price, self-hosting never pays on cost alone, however much traffic you have. Utilisation is the lever people miss. A GPU that is busy 20% of the time costs 2.5× more per token than one busy 50% of the time.

> Figure: Illustrative prices. Defaults (1B tokens/month, API $2/1M, GPU $2.50/h, 1,500 tokens/s per GPU, 50% utilisation, $3,000/month ops): one GPU serves 1.97B tokens/month; the API costs $2,000 and self-hosting $4,825. Break-even is 4.24B tokens/month. Self-hosting is first cheaper at 3.33B, but the third GPU, needed above 3.94B, tips it back. At 10B: API $20,000 vs self-hosting $13,950 on 6 GPUs ($1.40/1M). Set ops to $0 and break-even drops to 0.91B. Drop utilisation to 20%: the marginal cost is $2.31/1M, above the API price, and self-hosting never wins.

## Worked example

**Defaults of the figure**

1. Capacity per GPU: $1500\times0.5\times3600\times730=1.971\times10^9$ tokens/month.
2. One GPU-month: $2.50\times730=\text{\textdollar}1{,}825$. Marginal self-host cost: $1825/1971=\text{\textdollar}0.93$ per million tokens, below the API's $\text{\textdollar}2$.
3. At $V=1$B: API $=10^9\times2/10^6=\text{\textdollar}2{,}000$; self-host $=1\times1825+3000=\text{\textdollar}4{,}825$. The API wins.
4. At $V=10$B: $\lceil10/1.971\rceil=6$ GPUs, $6\times1825+3000=\text{\textdollar}13{,}950$ against API $\text{\textdollar}20{,}000$. Self-hosting wins by $\text{\textdollar}6{,}050$ a month.
5. Break-even: 2 GPUs ($\text{\textdollar}6{,}650$) undercut the API from $6650/2=3.325$B, but above $2\kappa=3.942$B a third GPU costs $\text{\textdollar}8{,}475$. The API stays cheaper until $8475/2=4.2375$B tokens/month.

```python
import math
HOURS = 730

def monthly_costs(tokens, api_price, gpu_hour, tps, util, ops):
    cap = tps * util * 3600 * HOURS                 # tokens one GPU serves per month
    gpus = max(1, math.ceil(tokens / cap))
    self_host = gpus * gpu_hour * HOURS + ops
    api = tokens * api_price / 1e6
    return api, self_host, gpus

for v in (1e9, 3.4e9, 4e9, 1e10):
    api, self_host, gpus = monthly_costs(v, api_price=2.0, gpu_hour=2.5, tps=1500, util=0.5, ops=3000)
    print(f"{v/1e9:4.1f}B tokens: API ${api:,.0f}  self-host ${self_host:,.0f} on {gpus} GPU(s)")
# 1.0B: API $2,000  self-host $4,825 on 1     3.4B: $6,800 vs $6,650 on 2
# 4.0B: API $8,000  self-host $8,475 on 3    10.0B: $20,000 vs $13,950 on 6
```

Monthly API vs self-hosted cost; replace the illustrative prices and the measured throughput with your own (runs as is)

## Beyond cost

**Strengths**

- Open weights: data stays on your infrastructure, which simplifies privacy, residency and air-gapped deployments.
- Open weights: full customisation (fine-tuning, quantisation, constrained decoding over logits), and a pinned version that never changes under you.
- Closed: no infrastructure or ML-ops staff, elastic capacity, and often the strongest available quality on hard tasks.

**Limitations**

- Open weights: you own uptime, scaling, security patches, safety filtering and licence compliance, and pay for idle GPUs.
- Closed: data leaves your boundary, per-token cost grows with traffic, rate limits apply, and retired versions can change behaviour.
- Both: anyone with weights can fine-tune safety training away; even fine-tuning APIs allowed this (Qi et al., ICLR 2024, broke GPT-3.5 Turbo's safety with 10 examples for under $0.20).

> **Weights files can run code:** PyTorch `.bin` and `.pt` checkpoints are Python pickles, and loading an untrusted pickle can execute arbitrary code. Prefer `safetensors`, which stores only tensors and metadata. Pin the exact repository revision you reviewed.

> **The licence follows the fine-tune:** A LoRA adapter or merged model built on licensed weights is usually a derivative. Naming rules, user caps and use restrictions in the base licence typically apply to it too. Check the base licence before you distribute or sell a fine-tune (see **Copyright and licensing**).

**Question:** Which does the OSI Open Source AI Definition 1.0 require that an "open-weight" release does not?

- A. A downloadable parameter file
- B. Permission for commercial use only
- C. The training and inference code and sufficiently detailed information about the training data
- D. A hosted API

**Answer:** C. The training and inference code and sufficiently detailed information about the training data — Open weights are only the parameters. OSAID 1.0 asks for the preferred form for modification: the parameters, the complete code used to train and run the model, and detailed data information, all under terms that allow use, study, modification and sharing.

**Question:** A GPU costs $\text{\textdollar}3$/hour, serves 2B tokens/month at your real utilisation, and an API costs $\text{\textdollar}1$ per million tokens. Ignoring fixed costs, when does self-hosting become cheaper?

- A. Above 2B tokens/month
- B. Above 1B tokens/month
- C. Never: the marginal self-host cost is about $1.10 per million tokens
- D. Always, because there is no per-token fee

**Answer:** C. Never: the marginal self-host cost is about $1.10 per million tokens — One GPU-month is $3\times730=\text{\textdollar}2{,}190$ for $2\times10^9$ tokens, i.e. $\text{\textdollar}1.095$ per million, above the API's $\text{\textdollar}1$. More traffic only adds GPUs at the same rate. To win you must raise utilisation, use cheaper or faster GPUs, or quantise.

**Key points**

- Open-weight: you download parameters and run them under a licence. Closed: API only. Open source AI (OSI OSAID 1.0) also requires code and training-data information.
- Licences range from permissive (Apache-2.0, MIT) to custom community licences with user caps, acceptable-use policies and naming rules, to RAIL use restrictions passed downstream.
- API cost is linear, $Vp/10^6$; self-hosting is a step function, $\lceil V/\kappa\rceil h\cdot730+F$. Self-hosting pays only if the marginal rate $h\cdot730/\kappa$ is below the API price and volume clears the fixed cost.
- Utilisation drives self-hosting cost: idle GPUs are paid for.
- Beyond cost: privacy, residency, version pinning and customisation favour open weights; zero ops and frontier quality favour closed APIs.

---

From MLFoundry (https://www.mlfoundry.in). The web page adds interactive figures and diagrams.
