Critical utilisation: the one number that tells you whether to self-host a voice model

Where the critical-utilisation formula comes from

A GPU-hour costs the same whether the card serves calls or sits idle, while an API charges only for the minutes you use. So a GPU earns its place when the API minutes it absorbs in an hour would have cost more than the hour itself. Everything else in the self-hosting decision builds on that comparison.

Put numbers on it. A GPU that can carry N calls at once serves at most 60 × N call-minutes an hour, a call-minute being one minute of a live call. If it is busy a share U of the time, it saves 60 × N × U × a in API charges, where a is the API's price per call-minute. Setting that saving equal to the GPU's hourly price P gives the threshold, U* = P ÷ (60 × N × a), called critical utilisation.

Above U*, the GPU costs less than the API on hardware alone. Below it, the API is cheaper however well the deployment is run. The API price has to be per call-minute, so text to speech priced per character and LLMs priced per token need converting first; at 450 TTS characters a call-minute, $40 per million characters becomes $0.018. The full self-hosting cost model explains the character and token assumptions behind each conversion.

Critical utilisation for STT, TTS and LLMs in October 2026

With on-demand GPU prices and published stream counts, speech models clear the bar easily and LLMs mostly do not. The plain reading of the table is that a speech-to-text (STT) or text-to-speech (TTS) GPU pays for its hardware if it is busy about one hour in seventeen. An LLM GPU must be busy about half the time against a cached mini-class API, and most of the time against nano-class models.

That matters because utilisation is capped by your call curve, not chosen. A fleet has to be sized for the busiest hour plus headroom, so business-hours calling with a peak-to-average ratio of 3.5 and 20% headroom reaches at most 1 ÷ (3.5 × 1.2), about 24%. That clears every speech threshold in the table and none of the LLM ones.

The stream counts are 48 per L4 and 32 per L40S from NVIDIA's published tables. The LLM's 50 sessions per H100 is a planning figure inside Softcery's modelled range of 40 to 75, not a measurement, and it is the least certain row.

How prices, discounts and stream counts move the threshold

Because U* is a simple ratio, each input moves it in a predictable direction. A cheaper GPU lowers it in proportion. A cheaper API raises it, because each API minute the GPU replaces is worth less. Fewer streams per GPU raise it, because the card replaces fewer API minutes in each hour.

Work out your own critical utilisation from three inputs

You need three numbers: what your GPU costs an hour, how many concurrent streams it carries at your latency target, and what the API charges per call-minute. The stream count is the least certain. Vendor claims and NVIDIA's own tables differ by ten to a hundred times, so measure it with a load test on your own audio and prompts, as described in how many concurrent calls a 24 GB GPU carries.

Measure utilisation in the same units. It is the average number of concurrent calls a GPU carries, measured hourly, divided by the stream count you load-tested. The GPU busy percentage in monitoring tools measures compute, not calls, so it does not tell you how much API spend the card replaced. Paste the function below into Python and swap in your values; it prints the first, second and fourth rows of the table.

def critical_utilisation(gpu_per_hour, streams_per_gpu, api_per_minute):
    """Share of the time a GPU must be busy to cost less than the API it replaces."""
    return gpu_per_hour / (60 * streams_per_gpu * api_per_minute)

# STT: AWS L4 on-demand, 48 streams, Deepgram Nova-3 at $0.0048 a minute
print(f"STT {critical_utilisation(0.805, 48, 0.0048):.1%}")   # STT 5.8%
# TTS: AWS L40S on-demand, 32 streams, ElevenLabs Flash at $40 per 1M characters x 450 a minute
print(f"TTS {critical_utilisation(1.861, 32, 40 * 450 / 1_000_000):.1%}")   # TTS 5.4%
# LLM: Lambda H100 PCIe, 50 sessions, gpt-4.1-mini with 80% cached input at $0.0023 a minute
print(f"LLM {critical_utilisation(3.29, 50, 0.0023):.1%}")   # LLM 47.7%

What critical utilisation leaves out

U* is a hardware test, so passing it is necessary but not sufficient. It ignores the engineers who run inference, which Softcery estimates at $300,000 to $500,000 a year for a dedicated US team. Adding them turns the threshold into a break-even volume, worked through at 100,000, 1 million and 10 million minutes in the self-hosting cost model, along with an editable break-even spreadsheet.

It is also a test for one GPU at a time, and it applies to the last GPU you add, not the fleet's average. The first GPU in a fleet is busy almost every hour, while the last one may work only during the peak. That is why a fleet can pass on average and still contain GPUs that each cost more than the API they replace.

Finally, it assumes the self-hosted model is as good as the API. If a weaker model causes failed calls, repeated questions or longer calls, that cost belongs in the comparison too. Sonex Labs sells speech and LLM APIs (see Sonex pricing), so it has a stake in the answer; every input above is sourced so you can check it.