Originally published on AI Tech Connect.
TL;DR: when does self-hosting beat the API in 2026? Three quick rules, then the tables. Self-host wins when your inference workload is sustained, predictable and large enough to keep at least one B200 above ~50% utilisation 24/7. Cost-per-million-tokens collapses to around $0.02 at that point. API wins when traffic is bursty, when you swap models more than once a quarter, or when you cannot dedicate a platform engineer to the serving stack. Hybrid wins for everyone else. Run the steady-state slice on a reserved B200; spill the long tail to an API or to TPU spot capacity. Pro tip Run the utilisation calculation before the price calculation. A B200 reserved at $2.25/hr ($1,650/month, 24/7) only beats a per-token API once you can keep it meaningfully busy. Idle silicon is the most expensive…
