Multi-participant cloud telephony in India is inherently messy. Single-channel audio, crosstalk, overlapping speakers, dialect shifts, and constant code-switching make “who said what?” the hardest part—not transcription itself.
Generic ASR breaks in multi-speaker Indian calls. Multilingual, accented, noisy conversations demand diarization and transcription models trained on real Indian telephony conditions.
Cloud telephony infrastructure is another bottleneck. 8 kHz audio, lack of native real-time streaming, and platform restrictions make real-time AI unreliable, so pre-recorded pipelines remain the only stable option today.
The path forward for India’s multi-participant cloud telephony is ecosystem-level: richer multilingual datasets, dialect-specific benchmarks, stronger models, and upgraded cloud telephony standards—16 kHz audio and open streaming APIs—to achieve reliable real-time voice AI at scale.
I used to think “just get the transcript” was the hard part. Then I shipped my first real telephony pipeline for a multilingual market. One week in, our dashboards were confused, alerts were noisy, and the same question came up again and again: who actually said that?
If you work in customer service, healthcare, banking, or sales, you already know that voice is where truth shows up first. Calls carry intent, emotion, and context you can’t fake. The real challenge is turning those calls into something teams can trust.
And in India, where people switch between languages mid-sentence and dialects shift every few kilometres, it gets complicated fast.
Below is a real look at what breaks, what we learned, and what’s still hard.
Most B2C cloud telephony calls arrive as a single mixed stream. Great for storage, painful for diarization.
Models trained on clean audio or separate speaker channels get confused. You end up with:
Things get worse when there’s speaker overlap or crosstalk, which is common in conference calls or support triads. Two people talking at once can completely throw off the model.
In compliance-heavy areas like banking or crisis lines, this isn’t a small issue - it affects real decisions.
Indian calls don’t stick to one language. A sentence can start in English, switch to Hindi, and end in Marathi.
Off-the-shelf ASR systems aren’t ready for this - they miss words, butcher accents, and lose intent.
We had to find models that understood this mix naturally instead of forcing language detection as an afterthought.
Even within one region, pronunciation, vocabulary, and rhythm vary. A model that works great in Delhi may stumble in Nagpur or Coimbatore.
Training data diversity matters as much as model size here. Clean studio datasets don’t survive real-world accents, background noise, or regional speech quirks.
Even strong models lose to weak pipes.
Many cloud telephony providers still deliver 8 kHz audio, which humans handle fine but machines don’t.
We also tried real-time streaming, but most Indian providers don’t support it natively.
We even experimented with merging calls - having the person dial into another telephony number that did support streaming - but that meant putting the client on hold for a few seconds. Not ideal in live support.
Building a mobile app wasn’t an option either: Apple and Google both restrict background call recording, so you can’t legally or technically record voices from another app.
We switched to processing pre-recorded calls instead of chasing real-time.
The telephony provider recorded the calls, and we pulled them into our cloud pipeline for transcription and analysis.
For the heavy lifting - diarization, transcription, and even translation - we used Sarvam, a provider focused on Indian languages and trained on local, multilingual data.
That single change improved our Word Error Rate (WER), boosted diarization accuracy, and reduced chaos in multilingual conversations with speaker overlaps.
From those transcripts, we generated summaries and analytics that helped clients see patterns - sentiment, intent, and compliance - without needing perfect real-time performance.
Even with pre-recorded pipelines, accuracy and privacy are critical:
If we want voice AI that truly works for India, three areas need serious attention:
We need open, large-scale datasets that reflect real Indian speech - multilingual, code-switched, with background noise and overlapping talk.
Efforts like OpenSLR’s Indian speech corpora and AI4Bharat’s IndicSpeech are great starts, but we need broader participation from telecom providers and enterprises who already have diverse, anonymized call data.
Domain fine-tunable models with transparent metrics by language, dialect, and audio quality - not just overall accuracy.
Benchmarks should report how a model performs in messy, overlapping, real-world calls, not just on clean samples.
Better telephony standards: 16 kHz audio, open APIs, and reliable recording.
Real-time streaming must evolve beyond patchwork hacks and merged-call workarounds.
Building telephony for India isn’t about just getting the transcript. It’s about understanding the entire ecosystem - language, infrastructure, and context.
Our solution works today because we embraced what was possible: pre-recorded calls, Indian-trained models like Sarvam, and thoughtful analytics.
But real-time cloud telephony for multilingual India is still an unsolved problem. The path forward lies in better data, open standards, and collaboration across AI, telecom, and policy.
Voice will continue to lead how customers and clients interact. To make it reliable, accuracy, inclusivity, and safety have to be built in - from the first byte of audio to the last line of transcript.
At KeyValue, we build systems for problems that don’t have easy answers. If this talk sparked something for you, we’d love to talk.
Diarization is the process of identifying “who spoke when” in an audio recording. It separates a call into speaker-specific segments so each voice is correctly labeled. In Indian telephony—where audio is single-channel, low-quality, and often includes overlap—diarization becomes especially challenging and prone to errors.
Most Indian B2C calls arrive as a single mixed audio stream, not separate channels for each speaker. With crosstalk, overlaps, background noise, and low-bitrate (8 kHz) telephony audio, diarization models struggle to accurately determine “who said what.”
Indian conversations often switch between English, Hindi, and regional languages within the same sentence. Global ASR models—trained on clean, single-language datasets—aren’t built for this complexity, causing high error rates and misinterpretation of intent.
India’s dialect diversity means pronunciation, rhythm, and vocabulary change every few kilometres. Models trained on limited or studio-quality data fail to generalize to real-world accents, noisy environments, and region-specific speech patterns.
Most cloud telephony providers still operate on 8 kHz audio and lack native real-time streaming APIs. On top of that, mobile OS policies restrict background call recording. These constraints make real-time transcription and analytics unstable or technically infeasible.
Pre-recorded audio avoids streaming failures, ensures complete audio capture, and allows diarization and transcription models—especially Indian-language models—to work at higher accuracy. This leads to cleaner transcripts, better intent detection, and more trustworthy analytics.
India needs larger multilingual datasets, dialect-aware ASR models, and updated telecom standards such as 16 kHz audio, open APIs, and reliable real-time recording. These improvements are essential for voice AI to handle India’s linguistic and infrastructural complexity at scale.
