Self-hosted Whisper vs a speech-to-text API for phone calls: cost per audio hour is the wrong number

Why cost per audio hour misleads for live calls

Cost per audio hour is the right number for batch transcription and the wrong one for calls. In batch work you queue recordings and the GPU runs flat out, so dividing its hourly price by the hours of audio it gets through gives a fair cost. A live call is different. It arrives when the caller dials, needs an answer within a few hundred milliseconds, and holds a transcription stream open for its whole length, silences included.

Streaming APIs price that open stream directly. AssemblyAI bills streaming on "the time the WebSocket connection is open", so a session open for 60 minutes with 30 minutes of audio is billed for 60, at $0.15 an hour for its Universal-Streaming model (AssemblyAI pricing). Deepgram's streaming Nova-3 is $0.0048 a minute pay-as-you-go, a rate it marks as a limited-time promotion against a list price of $0.0077. How different vendors meter silence is covered in why a silent speech-to-text stream still costs money.

A self-hosted GPU bills the same way, only more coarsely: it costs as much at 3 a.m. as at the noon peak. AssemblyAI, which sells an API and so has a stake, makes the point that a GPU idling at 15% still bills at 100%. The arithmetic holds whoever says it.

Sonex Labs also sells a streaming speech-to-text model, Valmiki, billed by the second, so it sits on the API side of this comparison. Every calculation below is shown so you can rerun it with your own prices.

The "Whisper API costs 10x more" argument, redone with utilisation

The claim comes from an OpenAI community thread of December 2023 (OpenAI forum). The poster's numbers were an RTX 4090 at about $0.79 an hour on RunPod, which could transcribe roughly 20 users' worth of audio each hour. At $0.006 a minute, the API would charge $7.20 for those 20 audio hours, against $0.79 for the GPU.

That holds only if the GPU is full every hour you pay for it. Kept on all month, the card costs $0.79 × 730 hours = $576.70. At the API's $0.36 per audio hour, that money buys 1,602 hours of transcription. The card could handle 20 × 730 = 14,600 audio hours, so it beats the API only if it is busy more than 1,602 ÷ 14,600, about 11% of the time.

A second thread a few weeks later reached the opposite answer (OpenAI forum). Its author compared a spot RTX 4090 at about $0.39 an hour with the API's $0.36 per audio hour and concluded the costs were roughly one to one. One poster counted 20 hours of audio in every paid GPU-hour and the other counted one. That gap is utilisation, the share of paid GPU time that has audio in it, and it decides the answer.

Live calls also change the denominator. Twenty audio hours per GPU-hour is batch throughput, from the GPU working through files faster than real time. A streaming call cannot be sped up: it delivers one second of audio per second, and the latency target limits how many calls can share each pass of the model. For calls, the number that matters is streams per GPU, not batch speed.

How many live calls one GPU carries with Whisper

Whisper has no native streaming mode. It loads every input at 16 kHz and works on windows of up to 30 seconds (openai/whisper audio.py), so live use needs a wrapper that cuts audio into chunks and transcribes them as they arrive. Deepgram, a competitor, notes that this chunking adds latency. NVIDIA's Riva serving stack lists Whisper only for offline use, at about 77 times real time in batch on an L4 and 370 on an H100.

For streaming, published numbers are thin and far apart. Dograh warns that a setup clean in single-stream tests "can still fall apart at 10 concurrent connections". io.net's guide advises one GPU per 20 to 40 concurrent streams. NVIDIA measures 48 low-latency streams on an L4 for its own English streaming models, which are not Whisper.

Mixed-language calls are the trap. NVIDIA's multilingual code-switching streaming model manages only 8 streams on an L4, at about 23.6 seconds of latency (NVIDIA Riva ASR performance). If your callers mix Hindi and English, plan from the low end, and see Hindi and Hinglish speech-to-text accuracy on 8 kHz calls for what accuracy looks like on that audio.

Bursty call curves set the cost per minute

Utilisation comes from your call curve, not the GPU. You must provision for the busiest hour plus headroom, so average utilisation is roughly 1 ÷ (peak-to-average ratio × 1.2) with 20% headroom. Business-hours inbound calling, at a ratio of about 3.5, tops out near 24%, as the full self-hosting cost model works through.

Outbound campaigns are burstier. A campaign that dials 4 hours a day on weekdays uses 20 of the week's 168 hours, a peak-to-average ratio of 168 ÷ 20 = 8.4, so utilisation falls to about 1 ÷ (8.4 × 1.2), roughly 10%. The table shows what that does to one AWS L4 at $0.805 an hour, using cost per call-minute = GPU price ÷ (60 × streams × utilisation).

The busy share a GPU needs to beat the API on hardware alone is its critical utilisation. Against Deepgram's $0.0048 at 48 streams it is 0.805 ÷ (60 × 48 × 0.0048), about 5.8%. At 10 streams, a realistic figure for some streaming Whisper setups, it is about 28%, so a campaign GPU 10% busy loses before anyone is paid to run it. Against AssemblyAI's $0.0025 the thresholds roughly double, to about 11% and 54%. The method is explained in critical utilisation for self-hosted models.

People then decide the volume. With half an engineer on STT and the GPU 60% busy, the cost model puts break-even against Deepgram at about 1.8 million call-minutes a month for a US team and about 0.3 million for a team in India. For scale, 1 million minutes on Deepgram costs $4,800 a month, about ₹4.56 lakh at ₹95.04 per dollar.

Accuracy problems that can erase the saving

A cheaper model that hears less can cost more than it saves. Distil-large-v3 runs about 6.3 times faster than Whisper large-v3, at a word error rate of about 9.7% against 8.4% on short-form audio (Hugging Face figures, via Dograh). The faster model carries more streams, but every extra misheard word can become a wrong answer, a repeated question or a failed call.

A rough illustration shows the scale. At 1 million minutes a month and 3 minutes a call, about 333,000 calls, a model that fails 0.5 percentage points more of them, at $2 of value lost per failure, costs about $3,333 a month. Self-hosting STT at that volume saves at most about $4,300 a month on hardware if the GPUs run 60% busy, and nearer $3,000 with the three L4s a real fleet with a spare needs. A small accuracy gap wipes out most of it. Both inputs are assumptions, so replace them with your own failure rate and call value.

Whisper also invents text when its own silence checks miss dead air or hold music, because its decoder always tries to output something. Dograh cites a 2024 study that found hallucinations in about 1.4% of transcriptions. A voice agent may treat that text as the caller speaking, so self-hosting Whisper means building your own silence gate, as described in why Whisper hallucinates on silent phone audio.

Older comparisons need their dates. GDELT estimated self-hosted Whisper at $163,680 a year against $38,880 for a hosted model, "not counting developer and administrator costs". That was March 2024, and GPU prices have fallen since (AWS cut on-demand prices for its H100 instances by about 44% in June 2025), so read it as a direction rather than a price.

Hosted open-weight Whisper as a middle option

There is a third route between running GPUs and paying a proprietary API: paying someone else to host the open model. Groq lists Whisper large-v3 at $0.111 per audio hour and large-v3-turbo at $0.04, and Together lists streaming Whisper Large v3 at $0.0035 a minute. You get the open model's behaviour and pay per use, without idle GPUs.

You still own the accuracy work, such as silence gating and testing on phone audio, because the model is the same. Optimised runtimes such as faster-whisper or WhisperLive can raise streams per GPU if you do self-host, and cost per minute falls in direct proportion. No benchmark of their streaming concurrency with a documented method was found, so treat any figure you read as a starting point for your own load test.