Open-weight vs closed models

Open-weight models ship their parameters under a licence (Apache-2.0, MIT or a custom community licence) so you can self-host, fine-tune and keep data in-house; closed models are reached only through an API. Volume, privacy and licence terms decide.

8 min read742 wordscore0 viewsupdated 2026-10-21

Definition

Open-weight vs closed An open-weight model is one whose trained parameters can be downloaded and run by anyone who accepts its licence. A closed model is available only as a hosted service: you send requests to the provider's API and never receive the weights. Open weights are not the same as open source AI. The Open Source Initiative's Open Source AI Definition 1.0 (October 2024) also requires the training and inference code and enough information about the training data for a skilled person to build a substantially equivalent system.

The choice decides who runs the model, who sees the data, what you are allowed to change, and how the bill grows with traffic. It is usually made per workload, not per company. Many teams prototype on a closed API and move steady, high-volume or sensitive workloads to open weights once the requirements are clear.

What you actually get

With open weights you get a file of tensors (commonly safetensors), a tokenizer and a config. An 8B model in bf16 is about 16 GB. You serve it yourself with an inference server such as vLLM, TGI or llama.cpp (see Inference servers (vLLM, TGI)). Nothing leaves your network, the version never changes unless you change it, and you can read logits, quantise (Quantization), fine-tune (LoRA, QLoRA) or merge models.

With a closed model you get an HTTPS endpoint, a price per token, rate limits and a contract. The provider handles GPUs, scaling, safety filtering and upgrades. Your prompts and data cross your boundary, and the contract and data-processing terms set how they are logged, retained or used. Models are retired on the provider's schedule, so outputs can change when a version is deprecated.

open weights: you download the parameters and run themmodel.safetensors8B × 2 B (bf16) ≈ 16 GByour GPUs / VPCvLLM, TGI, llama.cppyour appdata never leavesyou can: fine-tune (LoRA), quantise, read logits, pin a version, run offlineyou own: GPUs, scaling, uptime, safety filters, licence complianceopenness is a spectrum, not a switchclosed APIuse via requestsAPI + hostedfine-tuningopen weights,custom licenceopen weights,Apache-2.0 / MITopen source AIweights+code+datamore you can inspect, change and redistribute →closed: you send requests; the vendor runs the modelyour appprompt + user dataHTTPS + JSON{model, messages}vendor serversweights never shipyou get: no infrastructure, pay per token, often the strongest modelsyou accept: data leaves your boundary (contract governs), rate limits,version deprecation, vendor usage policy, no access to weights“open weights” ≠ open source: a licence can still restrict users, uses or derivatives
fig.Top: open weights. A 16 GB safetensors file for an 8B model runs on your own GPUs; data stays inside and you can fine-tune, quantise and pin versions, but you own operations, safety filters and licence compliance. Middle: openness is a spectrum, from a closed API through custom-licence and permissive (Apache-2.0, MIT) weights to open source AI with code and data information. Bottom: a closed API; no infrastructure, but data leaves under contract and versions are retired by the vendor.

Licences

"Open weights" says nothing about what you may do with them; the licence does. Permissive software licences (Apache-2.0, MIT) allow commercial use, modification and redistribution with attribution. Custom model licences add conditions: caps on user numbers, acceptable-use policies, naming rules for derivatives, or limits on using outputs to train other models. Use-restriction licences (the RAIL family) attach behavioural restrictions that must be passed on to everyone downstream.

LicenceExampleCommercial useKey conditions
Apache-2.0Mistral 7B, Mixtral 8×7B (2023)yeskeep the licence and NOTICE file, mark changes; explicit patent grant that ends if you sue over patents
MITDeepSeek-R1 (2025)yeskeep the copyright and licence notice
Llama 2 Community LicenseLlama 2 (2023)yes, with limitsseparate licence needed above 700M monthly active users; acceptable-use policy; no using outputs to improve other LLMs
CreativeML OpenRAIL-MStable Diffusion v1 (2022)yesuse-based restrictions that every derivative and downstream user must also accept
Provider terms (closed)hosted API modelsper contractno weights; usage policy, data retention and training-on-data terms set by the contract
Examples of licence types, as of each model's release. Terms differ between versions of the same family; check the licence file shipped with the exact weights

The cost crossover

An API bill is linear in tokens. A self-hosted deployment is mostly fixed: GPUs are paid by the hour whether busy or idle, and someone must run them. One GPU serves a fixed number of tokens a month, so the self-hosted cost is a step function:

CAPI=V p106,Cself=⌈Vκ⌉ h⋅730+F,κ=tps⋅u⋅3600⋅730C_\text{API}=\frac{V\,p}{10^6},\qquad C_\text{self}=\Big\lceil\frac{V}{\kappa}\Big\rceil\,h\cdot730+F,\qquad \kappa=\text{tps}\cdot u\cdot3600\cdot730
VV
tokens per month; pp = API price per million tokens (blended input and output)
hh
price per GPU-hour; 730 hours per month, always on
κ\kappa
tokens one GPU serves per month: batched throughput (tokens/s) × average utilisation uu
FF
fixed monthly cost: engineering time, monitoring, redundancy

At high volume the per-token cost of self-hosting approaches the marginal rate h⋅730/κh\cdot730/\kappa. If that rate is above the API price, self-hosting never pays on cost alone, however much traffic you have. Utilisation is the lever people miss. A GPU that is busy 20% of the time costs 2.5× more per token than one busy 50% of the time.

monthly cost: API = V·price vs self-host = ⌈V/cap⌉·GPU·730 h + ops$10$100$1,000$10.0k$100.0k$1.00M10M100M1B10B100Btokens per month (log)break-evenat 1B tokens/monthAPI$2,000self-host$4,825GPUs needed1cheaperAPIself $/1M$4.83capacity / GPU1.97Bmarginal $/1M$0.93break-even4.24Bmarginal = GPU·730 h / capacityAPI cheaper below break-evenprices are illustrative
fig.Illustrative prices. Defaults (1B tokens/month, API $2/1M, GPU $2.50/h, 1,500 tokens/s per GPU, 50% utilisation, $3,000/month ops): one GPU serves 1.97B tokens/month; the API costs $2,000 and self-hosting $4,825. Break-even is 4.24B tokens/month. Self-hosting is first cheaper at 3.33B, but the third GPU, needed above 3.94B, tips it back. At 10B: API $20,000 vs self-hosting $13,950 on 6 GPUs ($1.40/1M). Set ops to $0 and break-even drops to 0.91B. Drop utilisation to 20%: the marginal cost is $2.31/1M, above the API price, and self-hosting never wins.

Worked example

Defaults of the figure

  1. 1Capacity per GPU: 1500×0.5×3600×730=1.971×1091500\times0.5\times3600\times730=1.971\times10^9 tokens/month.
  2. 2One GPU-month: 2.50×730=$1,8252.50\times730=\text{\textdollar}1{,}825. Marginal self-host cost: 1825/1971=$0.931825/1971=\text{\textdollar}0.93 per million tokens, below the API's $2\text{\textdollar}2.
  3. 3At V=1V=1B: API =109×2/106=$2,000=10^9\times2/10^6=\text{\textdollar}2{,}000; self-host =1×1825+3000=$4,825=1\times1825+3000=\text{\textdollar}4{,}825. The API wins.
  4. 4At V=10V=10B: ⌈10/1.971⌉=6\lceil10/1.971\rceil=6 GPUs, 6×1825+3000=$13,9506\times1825+3000=\text{\textdollar}13{,}950 against API $20,000\text{\textdollar}20{,}000. Self-hosting wins by $6,050\text{\textdollar}6{,}050 a month.
  5. 5Break-even: 2 GPUs ($6,650\text{\textdollar}6{,}650) undercut the API from 6650/2=3.3256650/2=3.325B, but above 2κ=3.9422\kappa=3.942B a third GPU costs $8,475\text{\textdollar}8{,}475. The API stays cheaper until 8475/2=4.23758475/2=4.2375B tokens/month.
python
1import math
2HOURS = 730
3
4def monthly_costs(tokens, api_price, gpu_hour, tps, util, ops):
5 cap = tps * util * 3600 * HOURS # tokens one GPU serves per month
6 gpus = max(1, math.ceil(tokens / cap))
7 self_host = gpus * gpu_hour * HOURS + ops
8 api = tokens * api_price / 1e6
9 return api, self_host, gpus
10
11for v in (1e9, 3.4e9, 4e9, 1e10):
12 api, self_host, gpus = monthly_costs(v, api_price=2.0, gpu_hour=2.5, tps=1500, util=0.5, ops=3000)
13 print(f"{v/1e9:4.1f}B tokens: API ${api:,.0f} self-host ${self_host:,.0f} on {gpus} GPU(s)")
14# 1.0B: API $2,000 self-host $4,825 on 1 3.4B: $6,800 vs $6,650 on 2
15# 4.0B: API $8,000 self-host $8,475 on 3 10.0B: $20,000 vs $13,950 on 6
Monthly API vs self-hosted cost; replace the illustrative prices and the measured throughput with your own (runs as is)

Beyond cost

Use when

  • +Open weights: data stays on your infrastructure, which simplifies privacy, residency and air-gapped deployments.
  • +Open weights: full customisation (fine-tuning, quantisation, constrained decoding over logits), and a pinned version that never changes under you.
  • +Closed: no infrastructure or ML-ops staff, elastic capacity, and often the strongest available quality on hard tasks.

Avoid when

  • −Open weights: you own uptime, scaling, security patches, safety filtering and licence compliance, and pay for idle GPUs.
  • −Closed: data leaves your boundary, per-token cost grows with traffic, rate limits apply, and retired versions can change behaviour.
  • −Both: anyone with weights can fine-tune safety training away; even fine-tuning APIs allowed this (Qi et al., ICLR 2024, broke GPT-3.5 Turbo's safety with 10 examples for under $0.20).

Check yourself

Which does the OSI Open Source AI Definition 1.0 require that an "open-weight" release does not?

Check yourself

A GPU costs $3\text{\textdollar}3/hour, serves 2B tokens/month at your real utilisation, and an API costs $1\text{\textdollar}1 per million tokens. Ignoring fixed costs, when does self-hosting become cheaper?

Remember

  • ✓Open-weight: you download parameters and run them under a licence. Closed: API only. Open source AI (OSI OSAID 1.0) also requires code and training-data information.
  • ✓Licences range from permissive (Apache-2.0, MIT) to custom community licences with user caps, acceptable-use policies and naming rules, to RAIL use restrictions passed downstream.
  • ✓API cost is linear, Vp/106Vp/10^6; self-hosting is a step function, ⌈V/κ⌉h⋅730+F\lceil V/\kappa\rceil h\cdot730+F. Self-hosting pays only if the marginal rate h⋅730/κh\cdot730/\kappa is below the API price and volume clears the fixed cost.
  • ✓Utilisation drives self-hosting cost: idle GPUs are paid for.
  • ✓Beyond cost: privacy, residency, version pinning and customisation favour open weights; zero ops and frontier quality favour closed APIs.
← back to Generative AI roadmap