Substrate layer
A conversation is a latency budget with opinions.
Text agents get to think. A voice agent that pauses for two seconds has already lost the call. This is the loop that holds, and the specific ways it broke first.
Proves the substrate holds under real-time constraints.
This loop is not a standalone product. It runs behind SuryaSetu, which is where you can see it carrying real users.
The loop
Five stages, and every one of them is on the clock.
Numbered because the order is real — each stage cannot begin until the one before it produces something usable.
- CallerPhone
- VADSilero
- STTSarvam
- SubstrateDrevan
- LLMGroq
- TTSBulbul
- TransportLiveKit
- 01Silero VADDetects speech boundaries locally, so the network is never the thing deciding whether someone is talking.
- 02Sarvam STTStreaming transcription tuned for Indian languages and code-switching between Hindi and English mid-sentence.
- 03Groq LLMInference chosen for latency, not benchmark scores. A correct answer that arrives late is a failed turn.
- 04Sarvam Bulbul TTSStreaming synthesis. Audio starts before the sentence is finished generating.
- 05LiveKit WebRTCReal transport with real jitter buffers, because a phone call is not a websocket.
The artifact
The only evidence that counts is a recording.
Latency numbers and architecture diagrams are what an engineer writes down afterwards. What actually settles whether a voice agent works is listening to it handle a stranger who did not expect the call.
A real call from the pilot will sit here: the agent qualifying a homeowner in Hindi and English and booking a site visit, with the transcript following the audio. It is not published until the recording is cleared and identifying details are removed.
Scars
What real calls broke that testing never would.
Every one of these was found by listening to recordings of real conversations with real people. Anyone who has shipped voice AI will recognise them immediately.
Barge-in that cancelled itself
Interrupt handling wedged the pipeline when the agent was cancelled mid-utterance. Found by listening to real recorded calls, not by testing.
False-interruption resume
A cough or background noise read as an interruption and killed the sentence. The agent now resumes rather than abandoning the turn.
Ringback transcribed as speech
The dial tone itself entered the transcriber and became input before the human ever spoke.
Turn latency, decomposed
Per-turn latency is broken down by stage in the call report, so a slow turn names the stage that caused it instead of being "the agent felt slow".
Verticals
A new industry is a config file, not a fork.
Solar and real estate run the same loop with different knowledge, personas, qualification logic, and compliance rules. Adding a vertical means writing a pack, not branching the agent.
Why this is the point
Built with