full-duplex vs cascaded voice AI realism
Full-duplex speech-to-speech models lose to cascaded systems on reliability, tooling, and cost AI EngineerTL;DW
- Full-duplex speech models (like Moshi) enable overlapping speech and back-channeling, making conversations robust to human interaction patterns; half-duplex models break when users naturally interrupt or provide feedback.
- Cascaded systems (speech-to-text → LLM → text-to-speech) require entire stack to complete in ~200ms for natural conversation, but TTS alone costs 200ms+, making human-like latency impossible without tool-call optimization.
- Full-duplex models won't replace cascaded systems until they offer equal reliability, intelligence, observability, and personalization—current speech-to-speech models lack agent capabilities, tool calling, and abuse detection needed for production.
- Tool latency (500ms–4s) is now the main bottleneck, not TTS; filler-based solutions where LLMs keep conversation flowing during tool calls help mask unpredictable delays.
- On-device TTS models (<100M parameters) on smartphone CPUs eliminate cloud API costs and privacy risks, enabling profitable consumer voice apps without paying per-inference fees at scale.
- Voice API costs are unsustainable for consumer apps; hyperscalers run voice models at a loss as marketing, but TTS dominates production costs—consumer builders burning entire fundraising rounds on API bills.
TL;DW
- Full-duplex speech models (like Moshi) enable overlapping speech and back-channeling, making conversations robust to human interaction patterns; half-duplex models break when users naturally interrupt or provide feedback.
- Cascaded systems (speech-to-text → LLM → text-to-speech) require entire stack to complete in ~200ms for natural conversation, but TTS alone costs 200ms+, making human-like latency impossible without tool-call optimization.
- Full-duplex models won't replace cascaded systems until they offer equal reliability, intelligence, observability, and personalization—current speech-to-speech models lack agent capabilities, tool calling, and abuse detection needed for production.
- Tool latency (500ms–4s) is now the main bottleneck, not TTS; filler-based solutions where LLMs keep conversation flowing during tool calls help mask unpredictable delays.
- On-device TTS models (<100M parameters) on smartphone CPUs eliminate cloud API costs and privacy risks, enabling profitable consumer voice apps without paying per-inference fees at scale.
- Voice API costs are unsustainable for consumer apps; hyperscalers run voice models at a loss as marketing, but TTS dominates production costs—consumer builders burning entire fundraising rounds on API bills.
Zeghidour draws a line between naturalness and production readiness: full-duplex models like Moshi handle interruptions and paralinguistic cues but lack tool use, observability, and scalable economics. Cascaded STT-LLM-TTS pipelines stay dominant; Gradium's on-device TTS (Phonon) targets the cost problem by eliminating API round-trips.