Stop your voice bot from being interrupted by "haan", "achha" and "hmm"

Why "haan" and "hmm" cut your voice bot off

A backchannel is the small sound a listener makes to show they are following: "haan", "hmm", "achha", "theek hai", "ji". People make them while the other person is still talking, and nobody expects the speaker to stop. Barge-in is the opposite case: the caller speaks over the agent because they want the turn, and the agent should stop and listen.

People handle the difference without thinking. A study of conversation in 10 languages found in every one "a general avoidance of overlapping talk" and short gaps between turns (Stivers and colleagues, PNAS, 2009). Backchannels are the overlap listeners expect, which is why a speaker carries on through them.

A voice pipeline does not know that. Most frameworks start the caller's turn on one of two signals: voice activity detection (VAD), a small model that decides whether a stretch of audio contains speech, or a transcript from speech-to-text (STT). "Hmm" passes both tests. The VAD hears 300 ms of voice, the STT returns "hmm" or "okay", the framework cancels the agent's audio, and the caller hears the bot stop mid-sentence and wait for them.

Hindi makes this harder because the same word does several jobs. "हाँ" can mean yes, "go on", or "wait, I have something to say", and a 2009 Interspeech paper by Kalika Bali studied whether pitch alone can tell those uses apart (Bali, Interspeech 2009). "Achha" can be a nod or a surprised "really?". A rule that looks only at the word cannot always tell a nod from an answer, which is why the fixes below combine the word, its length and the moment it arrives.

Hindi and Hinglish backchannels, and what the agent should do with each

Sort what callers say over the agent into three groups: sounds that mean "carry on", words that might need a pause, and words that must stop the agent. The table is our working list for Hindi and Hinglish calls. STT engines may return these words in Devanagari or in Roman letters, so your list needs both spellings.

The middle group is where most bad experiences come from. "Haan, but" or "achha, toh" starts as a backchannel and turns into a real turn, so the agent should not decide on the first word alone. Wait for the next word or two: if they are also backchannels, keep talking; if not, stop.

Agent replies shape this as much as the settings do. A long readout invites callers to say "haan" halfway through, so break dates, amounts and instructions into short sentences with natural gaps. The two samples below carry the same booking; the second leaves the caller places to agree without talking over the agent. A slightly informal delivery helps too, for reasons our piece on why a too-perfect Hindi voice hurts conversion explains.

What a word rule costs: evidence from Indian phone calls

We counted interruptions on 561 phone calls over 30 days, from 24 September to 24 October 2026, using our own production logs. On agents where any detected speech interrupts, 140 of 232 agent replies were cut off, which is 60%. On agents that need two or three words while the agent is speaking, 726 of 4,198 replies were cut off, about 17% (18% at two words, 16% at three). With any-speech barge-in, 80 of the 131 cut-offs that had a follow-up were caused by two words or fewer.

The word rule did catch backchannels. It discarded 575 short utterances spoken over the agent, and 128 of them (22%) were pure backchannels: "yeah", "okay", Telugu "ఆ" and "అవునండి", Kannada "ಹೌದು" and "ಓಕೆ". But 224 (39%) were real short answers such as "Wednesday", "tomorrow", "ఈరోజు" (today) or a number, and 149 (26%) were "hello" said over the agent's greeting. A count of words cannot tell "ಹೌದು" (yes) from "Tuesday".

Be clear about what these numbers are. They compare different agents with different prompts, not one agent before and after a change, and no agent changed its setting long enough in the window to measure that. Our logs do not record whether an interruption was wanted, so this is a cut-off rate, not a false barge-in rate.

Much of the traffic came from our own test and pilot agents, and testers talk over agents more than customers do. The calls were mainly in Telugu, Kannada and English; there was too little Hindi to list Hindi backchannels from data, so the Hindi table above is our design guidance.

The settings by name in Pipecat and LiveKit

Both open-source frameworks give you the same three levers: how long a sound must last, how many words it must contain, and whether a model judges it from the audio. Which one you reach for decides what your callers lose. If you are still connecting either framework to an Indian number, our guide to putting a Pipecat or LiveKit agent on a +91 number covers the SIP side first.

import re

from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.audio.vad.vad_analyzer import VADParams
from pipecat.processors.aggregators.llm_response_universal import (
    LLMContextAggregatorPair,
    LLMUserAggregatorParams,
)
from pipecat.turns.types import ProcessFrameResult
from pipecat.turns.user_start import MinWordsUserTurnStartStrategy
from pipecat.turns.user_stop import SpeechTimeoutUserTurnStopStrategy
from pipecat.turns.user_turn_strategies import UserTurnStrategies

# Words that mean "go on". Both scripts, because STT may return either.
BACKCHANNELS = {
    "hmm", "hm", "mm", "haan", "han", "haa", "ji", "achha", "acha", "accha",
    "theek", "thik", "hai", "ok", "okay", "sahi", "yeah", "right",
    "हम्म", "हाँ", "हां", "जी", "अच्छा", "ठीक", "है", "ओके", "सही",
}
# Strip punctuation only. A pattern like [^\w\s] would also delete Devanagari vowel signs.
PUNCT = re.compile(r"[.,!?।…\"'’:;()\-]+")


def spoken_words(text: str) -> list[str]:
    return PUNCT.sub(" ", text.lower()).split()


class BackchannelAwareStart(MinWordsUserTurnStartStrategy):
    """While the bot speaks, ignore utterances made only of backchannels.
    Any other word interrupts, even a single "nahi" or "Tuesday"."""

    def __init__(self, **kwargs):
        super().__init__(min_words=1, **kwargs)

    async def _handle_transcription(self, frame):
        words = spoken_words(frame.text)
        if not words or (self._bot_speaking and all(w in BACKCHANNELS for w in words)):
            await self.trigger_reset_aggregation()
            return ProcessFrameResult.CONTINUE
        return await super()._handle_transcription(frame)


def build_aggregators(context):
    params = LLMUserAggregatorParams(
        vad_analyzer=SileroVADAnalyzer(
            params=VADParams(confidence=0.7, start_secs=0.2, stop_secs=0.2)
        ),
        user_turn_strategies=UserTurnStrategies(
            start=[BackchannelAwareStart()],
            stop=[SpeechTimeoutUserTurnStopStrategy(user_speech_timeout=0.2)],
        ),
    )
    return LLMContextAggregatorPair(context, user_params=params)

How we set up turn-taking on Sonex phone calls

Every agent on our platform has one turn-taking configuration, and all five of our call paths build the pipeline from it: browser calls and four phone carriers. It has three parts. Interruptions are off, on with any detected speech, or on after N words (two by default). Turn detection is a Silero VAD with confidence, start, stop and minimum-volume settings, plus an end-of-turn pause that defaults to 0.2 s. Silence handling covers check-ins when the caller goes quiet and when to end the call.

Each phone carrier has floors that an agent's settings cannot go below. On the two carriers that bring us 8 kHz audio over Indian mobile networks, VAD confidence cannot drop under 0.7 or start delay under 0.3 s, because lower values made agents interrupt themselves on background noise. Turning interruptions off mutes the caller's turn, not the caller's audio: the STT keeps listening, because dropping audio while the agent spoke starved the STT on one carrier.

Two safety nets matter for short replies. If a turn opens but no final transcript arrives within 3 s, the turn is closed and the agent asks the caller, in the call's language, to say it again. And every utterance the word rule throws away is logged with its text, which is how we counted the 224 lost answers above.

If you use a word rule, log what it drops; it is the only way to see the answers your callers gave and your agent never heard. The same settings are exposed per agent on Sonex AI voice agents, and the chart below is the order in which we would choose them.

What a word rule can't fix, and the competing approaches

A word list is a blunt tool, and it fails in both directions. It ignores a "haan" that really meant "stop, I've understood", and it still interrupts on a noise the STT heard as a word. It also needs maintaining for every language you support. Our Telugu data shows a backchannel list built only from Hindi and English would have missed the politeness particle that cut agents off most often.

Audio-based models are the main alternative. They listen to how a sound is said, not just what it was, which is closer to how people judge a backchannel. LiveKit's adaptive mode is one, and research on Hindi "हाँ" suggests pitch carries part of the signal. The trade-offs are that these models are trained mostly on English conversation, may run as a hosted service, and are hard to inspect when they get an Indian-language call wrong.

Some teams argue for the opposite approach: let callers interrupt freely and keep agent replies short, so there is little to interrupt. That works for quick question-and-answer calls and suits callers who find being talked over rude. It works less well when the agent must read out a booking, a policy or an amount. Agents can also make their own small sounds while they think, which our article on filler audio for voice agents covers, and the same timing problems apply in reverse.

Noise that the STT turns into words needs its own fix, and noise cancellation is the right tool there, not for backchannels, which are real speech. In livekit/agents issue 6030, SIP background noise kept an idle timer from firing, because an energy-based speech-start event fired on noise without ever producing a transcript; the proposed fix ties idle timing to real transcripts. In practice the strongest fixes are the unglamorous ones: short agent sentences, a clean audio path, and logs of what was ignored. Check them against real calls in each language, as our guide to testing a multilingual voice agent describes.