Definition
Open-weight vs closed An open-weight model is one whose trained parameters can be downloaded and run by anyone who accepts its licence. A closed model is available only as a hosted service: you send requests to the provider's API and never receive the weights. Open weights are not the same as open source AI. The Open Source Initiative's Open Source AI Definition 1.0 (October 2024) also requires the training and inference code and enough information about the training data for a skilled person to build a substantially equivalent system.
The choice decides who runs the model, who sees the data, what you are allowed to change, and how the bill grows with traffic. It is usually made per workload, not per company. Many teams prototype on a closed API and move steady, high-volume or sensitive workloads to open weights once the requirements are clear.
What you actually get
With open weights you get a file of tensors (commonly safetensors), a tokenizer and a config. An 8B model in bf16 is about 16 GB. You serve it yourself with an inference server such as vLLM, TGI or llama.cpp (see Inference servers (vLLM, TGI)). Nothing leaves your network, the version never changes unless you change it, and you can read logits, quantise (Quantization), fine-tune (LoRA, QLoRA) or merge models.
With a closed model you get an HTTPS endpoint, a price per token, rate limits and a contract. The provider handles GPUs, scaling, safety filtering and upgrades. Your prompts and data cross your boundary, and the contract and data-processing terms set how they are logged, retained or used. Models are retired on the provider's schedule, so outputs can change when a version is deprecated.
Licences
"Open weights" says nothing about what you may do with them; the licence does. Permissive software licences (Apache-2.0, MIT) allow commercial use, modification and redistribution with attribution. Custom model licences add conditions: caps on user numbers, acceptable-use policies, naming rules for derivatives, or limits on using outputs to train other models. Use-restriction licences (the RAIL family) attach behavioural restrictions that must be passed on to everyone downstream.
| Licence | Example | Commercial use | Key conditions |
|---|---|---|---|
| Apache-2.0 | Mistral 7B, Mixtral 8×7B (2023) | yes | keep the licence and NOTICE file, mark changes; explicit patent grant that ends if you sue over patents |
| MIT | DeepSeek-R1 (2025) | yes | keep the copyright and licence notice |
| Llama 2 Community License | Llama 2 (2023) | yes, with limits | separate licence needed above 700M monthly active users; acceptable-use policy; no using outputs to improve other LLMs |
| CreativeML OpenRAIL-M | Stable Diffusion v1 (2022) | yes | use-based restrictions that every derivative and downstream user must also accept |
| Provider terms (closed) | hosted API models | per contract | no weights; usage policy, data retention and training-on-data terms set by the contract |
The cost crossover
An API bill is linear in tokens. A self-hosted deployment is mostly fixed: GPUs are paid by the hour whether busy or idle, and someone must run them. One GPU serves a fixed number of tokens a month, so the self-hosted cost is a step function:
- tokens per month; = API price per million tokens (blended input and output)
- price per GPU-hour; 730 hours per month, always on
- tokens one GPU serves per month: batched throughput (tokens/s) × average utilisation
- fixed monthly cost: engineering time, monitoring, redundancy
At high volume the per-token cost of self-hosting approaches the marginal rate . If that rate is above the API price, self-hosting never pays on cost alone, however much traffic you have. Utilisation is the lever people miss. A GPU that is busy 20% of the time costs 2.5× more per token than one busy 50% of the time.
Worked example
Defaults of the figure
- 1Capacity per GPU: tokens/month.
- 2One GPU-month: . Marginal self-host cost: per million tokens, below the API's .
- 3At B: API ; self-host . The API wins.
- 4At B: GPUs, against API . Self-hosting wins by a month.
- 5Break-even: 2 GPUs () undercut the API from B, but above B a third GPU costs . The API stays cheaper until B tokens/month.
1import math2HOURS = 73034def monthly_costs(tokens, api_price, gpu_hour, tps, util, ops):5 cap = tps * util * 3600 * HOURS # tokens one GPU serves per month6 gpus = max(1, math.ceil(tokens / cap))7 self_host = gpus * gpu_hour * HOURS + ops8 api = tokens * api_price / 1e69 return api, self_host, gpus1011for v in (1e9, 3.4e9, 4e9, 1e10):12 api, self_host, gpus = monthly_costs(v, api_price=2.0, gpu_hour=2.5, tps=1500, util=0.5, ops=3000)13 print(f"{v/1e9:4.1f}B tokens: API ${api:,.0f} self-host ${self_host:,.0f} on {gpus} GPU(s)")14# 1.0B: API $2,000 self-host $4,825 on 1 3.4B: $6,800 vs $6,650 on 215# 4.0B: API $8,000 self-host $8,475 on 3 10.0B: $20,000 vs $13,950 on 6Beyond cost
Use when
- +Open weights: data stays on your infrastructure, which simplifies privacy, residency and air-gapped deployments.
- +Open weights: full customisation (fine-tuning, quantisation, constrained decoding over logits), and a pinned version that never changes under you.
- +Closed: no infrastructure or ML-ops staff, elastic capacity, and often the strongest available quality on hard tasks.
Avoid when
- −Open weights: you own uptime, scaling, security patches, safety filtering and licence compliance, and pay for idle GPUs.
- −Closed: data leaves your boundary, per-token cost grows with traffic, rate limits apply, and retired versions can change behaviour.
- −Both: anyone with weights can fine-tune safety training away; even fine-tuning APIs allowed this (Qi et al., ICLR 2024, broke GPT-3.5 Turbo's safety with 10 examples for under $0.20).
Check yourself
Which does the OSI Open Source AI Definition 1.0 require that an "open-weight" release does not?
Check yourself
A GPU costs /hour, serves 2B tokens/month at your real utilisation, and an API costs per million tokens. Ignoring fixed costs, when does self-hosting become cheaper?
Remember
- ✓Open-weight: you download parameters and run them under a licence. Closed: API only. Open source AI (OSI OSAID 1.0) also requires code and training-data information.
- ✓Licences range from permissive (Apache-2.0, MIT) to custom community licences with user caps, acceptable-use policies and naming rules, to RAIL use restrictions passed downstream.
- ✓API cost is linear, ; self-hosting is a step function, . Self-hosting pays only if the marginal rate is below the API price and volume clears the fixed cost.
- ✓Utilisation drives self-hosting cost: idle GPUs are paid for.
- ✓Beyond cost: privacy, residency, version pinning and customisation favour open weights; zero ops and frontier quality favour closed APIs.