Self-host the base load, burst to an API: a hybrid design for voice agents in India

What "base load and burst" means for a voice agent

The base load is the traffic you can count on at almost every hour of the week. The burst is everything above it: the mid-morning peak, a campaign, a seasonal spike. A hybrid design owns GPUs for the first and rents capacity by the minute, from an API, for the second, so that no owned GPU sits idle through the night.

Whether a given GPU belongs in the owned fleet depends on its critical utilisation, the share of time it must be busy to cost less than the API on hardware alone. It equals the GPU's hourly price ÷ (60 × concurrent streams per GPU × the API's price per call-minute). A GPU that will be busier than that is worth owning; below it, that traffic is cheaper on the API. Critical utilisation for STT, TTS and LLMs works out the values: about 5 to 6% for speech to text (STT) and text to speech (TTS), and about 48% for an LLM against a cached mini-class API.

Sonex Labs sells hosted speech and LLM APIs, including the Vyas LLM API, and runs its own speech models on GPUs with a standby, so it sits on both sides of this design. The cost figures below come from public price pages and an hour-by-hour model, labelled as such.

Why STT and TTS should cover the peak on their own GPUs

For speech, even the GPU that only works during the busiest hour of the week pays for itself. An STT GPU on an AWS L4 needs to be busy about 5.8% of the time to beat Deepgram, and a TTS GPU on an L40S about 5.4% to beat ElevenLabs Flash. A business-hours call curve keeps even its last GPU busier than that.

So when the cheapest fleet was modelled hour by hour, on a synthetic weekday profile at 10 million call-minutes a month, STT and TTS came out covering about 100% of the peak, with under 1% of minutes spilled to the API. For these two components, burst capacity is insurance against outages and deploys, not a way to save money.

The exception is a model that carries far fewer streams. NVIDIA's L4 tables list a multilingual code-switching STT model at a maximum of 3 streams, against 49 to 83 for its English models. At 3 streams, critical utilisation becomes 0.805 ÷ (60 × 3 × 0.0048), about 93%, so a GPU would have to be busy almost all the time to beat the API. For code-switched STT, keeping the traffic on the API is usually the cheaper choice, which is one more reason to load-test on real Hinglish or Tamil-English calls before deciding.

How the LLM split works at 10 million minutes a month

The LLM is where bursting pays, because its threshold is high. Take a Lambda H100 PCIe at $3.29 an hour serving about 50 voice sessions. Against gpt-4.1-mini with cached prompts, at about $0.0023 a call-minute, it needs to be busy about 48% of the time. Against nano-class models at about $0.0013 it needs 84%, which almost no call curve reaches.

So only GPUs that stay busy more than half the week are worth owning. In the same hour-by-hour model, that was 3 H100s, with about 73% of LLM minutes sent to the API. The table compares that hybrid with the two pure designs, GPUs only and no people.

The hybrid comes out about 4% dearer than all-API and 41% cheaper than a GPU fleet sized for the peak. That is the honest result: bursting makes a self-hosted LLM nearly cost-neutral, not cheaper. The reasons to self-host the base load are lower latency, data that must not leave your servers, or a fine-tuned model. The hybrid is how you get them without paying for a peak-sized fleet.

Residency in India costs more on the LLM tier

Hosting in India moves the LLM threshold up, not down. E2E Networks charges ₹347.70 an hour for a committed H100 before 18% GST, about $4.32 an hour with GST at ₹95.04 per dollar. On that GPU the busy share needed against cached gpt-4.1-mini rises to 4.32 ÷ (60 × 50 × 0.0023), about 63%. Indian clouds buy residency and shorter network paths, not cheaper hardware.

A hybrid also sends some calls to the burst path, so residency has to hold there too. If your contracts or sector rules require call data to stay in India, the API you overflow to must meet the same terms as your own GPUs. How India's DPDP Act applies to voice AI covers the data-protection side.

Indian teams do win on people, because engineering salaries are about a fifth of US ones. That, not the hardware, is where the full self-hosting cost model finds the Indian advantage.

Failover is the other half of a hybrid

A cascaded voice agent chains several services, and their failures add up. Softcery's arithmetic is that four hosted services at 99.9% each multiply to about 99.6%, and at 99% each to about 96% (Softcery). At 99.6%, the agent is unavailable about 0.4% of 730 hours, close to 3 hours a month.

A hybrid can do better, because the API you burst to is also a second path for each component. If the two paths fail independently and switching is instant, a component's downtime shrinks to the time both are down at once. The catch is the word instant: a fallback only helps once you notice the primary has failed.

Streaming connections are bad at telling you. A LiveKit Agents issue opened on 17 October 2026 reports that several streaming STT plugins never notice a WebSocket that goes silent without closing, and that the fixed ones take about 45 seconds, "45s of dead air before failover" (livekit/agents #7900). Another reports an STT stream that does not reconnect after a mid-stream disconnect. Forty-five seconds of a caller talking to nobody is a lost call, whatever the uptime figure says.

Frameworks do provide the switching itself. LiveKit's fallback adapters move to the next provider on errors including timeouts and mid-stream disconnects, and probe the failed one in the background until it recovers (LiveKit fallback strategies). What they cannot supply is fast detection of a stream that stays open but stops answering, and that part is yours to build.

How Sonex fails over: a held standby and a one-second STT budget

Sonex's own failover, as its code describes it, follows two rules: keep paid standby capacity off until it is needed, and never make a caller repeat themselves.

What to settle before you build a hybrid

A hybrid is only as good as its least-checked path, so settle these before the first call goes through it. The deployment steps around them, from measuring your call curve to load-testing, are in how to self-deploy open-source voice models.

• Warm API contracts: keep the fallback account, rate limits and any minimum commitment active, because a fallback you have not called in months is not a fallback.
• Spill rate: track the share of minutes served by the API. For the LLM it is planned (about 73% in the model above); for STT and TTS any spill means an incident or a fleet that is too small.
• Detection time: alert on a stream that stops returning results while audio is still flowing, not only on a closed connection.
• Spot capacity: spot H100s are far cheaper, such as $1.29 an hour on Azure against $6.98 on demand, but can be taken back at short notice, so keep them for batch work unless a live session can move mid-call.
• Regions: a second active region needs its own spare GPU per component, doubling the minimum fleet at zero traffic. Many teams start with one region plus the API as the second path.
• People: count the engineers who run the GPUs; that cost, not the hardware, decides whether self-hosting pays at your volume.