Pipecat vs LiveKit for Phone Agents: How Calls Connect

On this page

Why the choice for phone agents starts with how calls arrive
How much of the conversation each framework handles for you
Does the framework change latency on phone calls?
Production traps from running Pipecat on phone calls
How to compare Pipecat and LiveKit on your own calls
Sources

Why the choice for phone agents starts with how calls arrive

Both frameworks do the same core job. A phone voice agent is usually a cascade: speech-to-text (STT) turns the caller's audio into words, a large language model (LLM) writes the reply, and text-to-speech (TTS) turns the reply back into audio. The framework is the plumbing between those models and the phone line. It moves audio in and out, decides when the caller has finished speaking, and stops the agent when the caller talks over it.

Pipecat, released under the BSD 2-Clause licence, treats every step as a processor in a pipeline that you assemble yourself. LiveKit Agents, under Apache 2.0, runs on LiveKit's real-time media server, where every speaker is a participant in a room and the framework manages the session. Both can be self-hosted, and both companies sell managed hosting as well.

The phone network does not speak WebRTC, the browser protocol both frameworks also support. A call reaches software in one of two ways. A carrier can stream the call's audio to your server over a WebSocket, a long-lived connection that carries small chunks of audio in each direction. Or it can hand the call over SIP (Session Initiation Protocol), the signalling standard that business phone systems and SIP trunks use. Which of these your carrier offers is the first fact to settle, because each framework is shaped around one of them.

How Pipecat takes a phone call

Pipecat's FastAPI WebSocket transport is, in its documentation's words, "primarily designed for telephony integrations" (Pipecat docs). The carrier sends its own message format, and a serializer for that carrier converts the messages into Pipecat frames and back. Pipecat ships serializers for Twilio Media Streams, Plivo, Exotel, Telnyx and Vonage's Audio Connector (Pipecat serializers). Without the right serializer the transport drops incoming messages, so the carrier and the serializer have to match.

For SIP, Pipecat relies on Daily, the company behind it. A Daily room can connect its own SIP leg to your carrier, and Daily then carries the audio to the bot over WebRTC (Pipecat docs: Daily SIP). The docs give static egress addresses for carrier allow-lists and the ability to swap carriers without changing bot code as the reasons to take that route.

On-premises phone systems are the gap. A request for a native Asterisk AudioSocket transport, opened in September 2025, was closed in July 2026 by pointing users to a community package (pipecat issue 2702). An Asterisk transport pull request took the same path. A maintainer suggested a community integration as the faster route, so its author closed it in May 2026 and published it separately as pipecat-asterisk (pipecat pull request 3229).

How LiveKit takes a phone call

LiveKit puts a SIP service in front of its rooms. Your SIP provider's inbound trunk points at LiveKit, dispatch rules decide which room each call joins, and a SIP participant is "automatically created for each caller" (LiveKit telephony docs). Your agent then talks to that participant like any other. LiveKit says its SIP service is designed to work with any SIP provider and lists Twilio, Telnyx, Exotel, Plivo, Wavix, Sinch and didlogic as tested. LiveKit Cloud includes the SIP service; a self-hosted deployment has to run it separately. Indian numbers add a further condition, because carriers there require call audio to stay on servers in India. The guide to putting a Pipecat or LiveKit agent on a +91 number covers the SIP settings.

LiveKit has also started accepting WebSocket streams. On 1 September 2026 it announced Connectors, starting with Twilio Programmable Voice and WhatsApp Business Calling, which bring a call into a room without a SIP trunk (LiveKit blog). Its own Twilio guide still calls SIP trunking "the recommended approach" and suggests the connector mainly for teams with existing Twilio workflows (LiveKit Twilio connector).

The two frameworks are not exclusive. Pipecat has a LiveKit transport that lets a Pipecat bot join a LiveKit room as a participant (Pipecat docs: LiveKit transport), so a team can use LiveKit's SIP service with a Pipecat pipeline behind it.

How much of the conversation each framework handles for you

The second difference is how much logic you write yourself. LiveKit's own comparison page, written by an interested party, presents LiveKit Agents as event-driven, with state management and interruption handling built in. It describes Pipecat as a static pipeline where state and interruptions are left to the developer (LiveKit field guide). That is a fair description of the design, if not of the trade-off. More built-in behaviour means less code to write, while a lower-level pipeline means fewer defaults to discover and override when a phone call does something unusual.

Turn-taking is where the difference shows up on calls. Turn-taking covers two decisions: when the caller has finished, so the agent can reply, and when a caller's sound is a real interruption rather than a backchannel, a short "uh-huh" or "okay" that only shows the caller is listening. Phone audio makes both harder, because it is narrowband, noisy and often delayed.

Turn-taking defaults in Pipecat

Pipecat's default way to end a caller's turn is Smart Turn v3, a model that predicts from the audio whether the speaker has finished (Pipecat docs: user turn strategies). The voice activity detector, which marks when sound stops, waits 0.2 seconds by default. If you swap Smart Turn for a plain silence rule, that rule waits a further 0.6 seconds by default, so the shortest wait becomes 0.8 seconds. A 5-second backstop ends a turn that never closes.

Smart Turn reads 16 kHz audio, and phone calls arrive at 8 kHz. Until a March 2026 fix it silently misjudged turns on 8 kHz input, as the guide to 8 kHz phone audio breaking 16 kHz speech models explains. For interruptions, MinWordsUserTurnStartStrategy makes the caller say a set number of words before the agent stops talking. That number applies only while the agent is speaking; when it is silent, one word starts a turn.

Turn-taking defaults in LiveKit

LiveKit's adaptive interruption handling uses an acoustic model to tell barge-ins from backchannels (LiveKit docs). It is on by default only when the agent runs on LiveKit Cloud, or in development mode with Cloud credentials. A self-hosted agent can switch it on, and if LiveKit's hosted inference is unavailable the session falls back to detection based on voice activity alone.

Two open issues show how much the settings matter. In livekit/agents issue 3515, min_interruption_duration appeared to have no effect once min_interruption_words was set. A contributor explained that the duration check applies only to interruptions triggered by voice activity, not to those triggered by transcripts (issue 3515). In issue 6415, LiveKit's turn detector committed full sentences in 0.3 seconds but left many one-word answers such as "Yes." or "Okay." waiting 2.5 seconds (issue 6415). A maintainer replied that the model needs earlier conversation to judge short replies, so tests without context are not a fair measure.

Short answers are a large share of what callers say on a phone agent, so this kind of behaviour is worth testing on either framework before launch. The guide to backchannels and barge-in on Hindi voice bots shows how often a "haan" or "hmm" cuts an agent off.

Does the framework change latency on phone calls?

Probably not by much, though nobody has published a controlled comparison. A searcher looking for a Pipecat vs LiveKit latency benchmark on phone calls finds comparison articles and vendor pages, not measurements. Fora Soft, a software house that builds real-time products, argues that "the framework barely moves your latency", estimating framework overhead at tens of milliseconds while a speech-to-text, language model and text-to-speech cascade adds roughly 300 to 800 ms (Fora Soft, October 2026). That is a reasoned estimate, not a published test.

The reasoning holds up when you list where a phone agent's delay comes from. The agent waits to be sure the caller has finished, the language model takes time to produce its first words, the TTS model takes time to produce its first audio, and the phone network adds delay in both directions. The framework's share is mostly moving frames between those steps. The hop-by-hop guide to voice agent latency on Indian phone calls breaks the stages down, and the article on what drives an LLM's time to first token covers the largest one.

Turn-taking settings are the exception, because they sit inside the framework and can add whole seconds. The 2.5-second wait in LiveKit issue 6415 and Pipecat's 0.8-second minimum with a plain silence rule are both framework behaviour. A latency comparison that leaves each framework on its defaults is mostly comparing defaults.

Running models next to the agent

Teams that self-host models inside the agent process meet a different limit: concurrency, the number of calls served at the same time. In livekit/agents issue 7764, opened on 12 October 2026, LiveKit's Hugging Face plugin served sessions on one worker in turn because of a lock on each loaded model (issue 7764). With a 0.5-billion-parameter model on an Apple M4, time to first token was 99 ms for one session alone and 1,568 ms for the third of three simultaneous sessions.

The lesson applies to Pipecat as much as to LiveKit. Any model that serves callers one at a time turns extra calls into a queue, and a queue on a phone call is dead air. Serving the models from a separate inference server that batches requests avoids it, whichever framework sits in front.

Production traps from running Pipecat on phone calls

These come from Sonex's own Pipecat deployment, so they describe one framework only. Several have LiveKit equivalents, and the testing lessons apply to both.

A pre-warm path that failed on every call

To save time at the start of a call, we opened the STT connection before the pipeline existed. Pipecat's full connect routine also starts background tasks, and a processor cannot start tasks until the pipeline gives it a task manager. So the early connect raised an error every time. Our failover logic read that as a provider outage, retried three times and ran calls on the backup STT for two days while the primary provider was healthy.

The fix was to open only the socket early and let the normal start finish the rest. The deeper problem was the tests: every test of the warm path had mocked the very function that broke. A test of a connect path should call the real code and replace only the network socket.

Settings that behave differently from their names

Setting allow_interruptions to False in Pipecat's pipeline parameters made Pipecat drop the caller's audio while the agent spoke, which starved the STT on one carrier. We keep it on and use a mute strategy on the caller's turn instead, so the STT keeps hearing the caller while the agent finishes.

Audio pushed from a processor placed before the language model arrived after the reply. The model and TTS handle a turn as one long awaited chain, so any other frame on that link queues behind the whole reply. Pushing from the TTS processor itself, after the busy link, solved it.

Tools registered so that an interruption cannot cancel them were treated as asynchronous. Their answers reached the model later as a separate message, with Telugu letters escaped into \u codes, and the model misread a date from it. The article on why a voice agent ignores a tool result covers the message-format side of that bug.

One carrier left behind

We built one turn-taking configuration that every call path uses, but one carrier's pipeline lived in its own file, and a new setting reached every carrier except that one. A test now reads every call site and fails if one leaves the setting out. Whatever framework you pick, keep one builder for every transport and test that each one uses it.

How to compare Pipecat and LiveKit on your own calls

A fair comparison changes only the framework. Use the same carrier and phone numbers, the same STT, language model and TTS, the same server region, the same prompts and the same scripted callers. Set the turn-taking rules to match as closely as the two frameworks allow, and write down the settings, because defaults differ. Only then does a difference in results belong to the framework rather than to the setup around it.

Take timestamps at the same points in both: when caller audio reaches the server, when the turn closes, when the model's first token arrives, when TTS returns its first audio and when the agent's audio leaves. Neither framework can see the phone network's delay, so record a sample of calls at the far end as well and measure from the caller's last word to the agent's first. Report the median and the 95th percentile, since one slow turn in twenty is what callers remember.

Then test what phone calls do to an agent. Run short answers, long pauses, spelled-out numbers and callers who talk over the agent. Add load until the 95th-percentile turn time starts to rise, and cut a provider's connection mid-call to see how each framework recovers.

Whichever you choose, keep what matters outside it. Prompts, tool definitions, provider choices and recorded test calls carry over between frameworks; transport wiring and turn-taking settings do not. If the behaviour you want, such as how many words count as an interruption, lives in tests you can run on either framework, switching later is a rewrite of plumbing rather than of the agent. For developers building on Sonex, the voice agent and speech APIs for developers describe what the platform handles.

Decision chart for Pipecat vs LiveKit phone agents based on SIP or WebSocket telephony and self-hosting — Three questions for choosing between Pipecat and LiveKit for phone agents: how calls arrive, who owns orchestration, and where it runs.

Sources

Change log: 24 Oct 2026, Rewritten for publication. Removed Sonex's own latency measurements and the promise of a head-to-head benchmark. Added LiveKit Connectors (WebSocket calls, announced 1 September 2026), Pipecat's Daily SIP route and LiveKit transport, and corrected Pipecat's turn-ending default, which is Smart Turn v3, not a 0.6-second pause

Pipecat vs LiveKit for phone agents: how calls connect, what each handles, and how to test them