Code-switching is the practice of alternating between two or more languages, or dialects of the same language, within a single conversation, often within a single sentence. A Delhi speaker saying "Meeting kal ho gayi thi, but the follow-up is still pending" is code-switching between Hindi and English. It is not broken speech and not a sign of weak language skills; decades of linguistics research show it is systematic, rule-governed behavior that bilinguals use for emphasis, precision, identity, and social nuance.
For anyone building voice technology, code-switching matters for a blunter reason: it is one of the most reliable ways to break an otherwise strong speech recognition system. Models trained on monolingual data (which is nearly all training data) mistranscribe mixed-language speech badly, and the failure hits exactly the markets where voice AI is growing fastest: India, Southeast Asia, the Middle East, and Africa.
This post covers what code-switching is, the linguistic types with real examples, why it defeats monolingual ASR, what benchmarks exist, and what actually works when you need a model to handle it.
Code-Switching in Language: Definition and Types
Linguists conventionally distinguish three types of code-switching, following Shana Poplack's foundational work on Spanish-English bilinguals in New York:
| Type | What happens | Example |
|---|---|---|
| Inter-sentential | Language changes at a sentence or clause boundary | "I'll send the report tonight. Bukas na natin pag-usapan." (Taglish: "Let's discuss it tomorrow.") |
| Intra-sentential | Languages mix inside a single sentence | "Yaar, the deadline bilkul impossible hai." (Hinglish) |
| Tag-switching | A tag, filler, or discourse marker from one language is attached to an utterance in another | "It's fine, inshallah." / "We're done here, ¿verdad?" |
A related distinction: code-mixing is sometimes used specifically for intra-sentential mixing, while code-switching covers the broader phenomenon. In speech-technology papers the terms are used almost interchangeably.
Real-world patterns worth knowing:
- Hinglish (Hindi-English). Ubiquitous in Indian cities and media. English nouns and verbs are routinely embedded in Hindi grammar ("Usne file forward kar di"), and speakers may switch scripts mid-message when typing.
- Spanglish (Spanish-English). Common across the US and border regions. Poplack's classic example, "Sometimes I'll start a sentence in Spanish y termino en español," is itself intra-sentential switching.
- Taglish (Tagalog-English). The default register of urban Philippine speech; news anchors, call-center agents, and politicians all mix within sentences.
- Arabic multilayered switching. Speakers move between their dialect (say, Moroccan Darija), Modern Standard Arabic, and French or English. That's three codes in one conversation, sometimes with French embedded in Darija morphology ("ghadi n-confirmé-h", "I'll confirm it").
- Swahili-English (East Africa), Yoruba-English (Nigeria), Malay-English (Malaysia/Singapore) follow similar urban patterns.
Multilingualism is the global norm (most of the world's population speaks more than one language), and in multilingual societies code-switching is the unmarked, everyday register, not an exception. Any voice product deployed in Mumbai, Manila, Casablanca, Nairobi, or Lagos will receive code-switched input on day one.
Why Code-Switching Breaks Speech Recognition
A conventional ASR system is monolingual by construction, and code-switched audio violates its assumptions on four levels at once:
1. Vocabulary. The model's token inventory and lexicon come from one language. English words spoken mid-Hindi-sentence simply do not exist in a Hindi-only vocabulary, so the decoder substitutes the closest-sounding Hindi words, confidently and wrongly.
2. Language model. Whether it is an explicit n-gram LM or the implicit language model inside an end-to-end decoder, the text distribution was learned from monolingual text. A sequence like "deadline bilkul impossible hai" has near-zero probability under both a Hindi LM and an English LM, so the decoder steers the acoustics toward fluent-but-wrong monolingual output.
3. Phoneme inventory and accent. Each language carves up acoustic space differently. Bilinguals also pronounce embedded words with transfer effects (English "development" in a Hindi frame doesn't sound like Ohio English), so even a bilingual system faces pronunciations that exist in neither monolingual corpus.
4. Script and orthography. What is the correct transcript of a Hinglish utterance? Devanagari throughout, Latin throughout, or mixed-script output that renders each word in its native orthography? Mixed reference transcripts are what users expect, but they wreak havoc on naive WER computation and on models trained toward a single output script. Serious code-switching evaluation uses transliteration-aware metrics for exactly this reason.
These failures compound in production features downstream of the transcript. Keyword spotting misses the English product name rendered as a phonetically similar Hindi word; intent classifiers trained on clean monolingual text receive garbled mixtures; and voice-agent responses go off the rails because the NLU layer never saw what the user actually said. A few percent of absolute WER degradation concentrated on switched content words can halve task-completion rates even when the overall transcript "looks fine."
Language-identification front ends make things worse, not better: routing an utterance to "the Hindi model" or "the English model" guarantees half the words go to the wrong recognizer, and intra-sentential switches happen faster than LID systems can react.
The result is measurable. Models with respectable single-digit WER on monolingual test sets routinely degrade sharply on code-switched conversational speech. Mixed error rates several times higher are common in published evaluations, with errors concentrated precisely on the switched words, which are often the content-bearing ones (product names, technical terms, numbers).
Benchmarks and Corpora for Code-Switching ASR
The research community has a handful of standard resources. They are useful, but small and narrow relative to the phenomenon:
- SEAME, the canonical benchmark: ~100 hours of spontaneous Mandarin-English conversational and interview speech from Singaporean and Malaysian speakers, with dense intra-sentential switching.
- ASCEND: Mandarin-English spontaneous conversation from Hong Kong (~10 hours), a common evaluation set for multilingual models.
- MUCS 2021: Hindi-English and Bengali-English code-switched ASR challenge data from Indian lecture domains.
- Bangor Miami: Spanish-English conversational corpus widely used in linguistics and increasingly in ASR.
- ArzEn: Egyptian Arabic-English spontaneous speech.
- Fisher Spanish / Miami-style Spanglish subsets and South African soap-opera corpora (English mixed with isiZulu, isiXhosa, Sesotho, Setswana) round out the commonly cited list.
Notice the pattern: nearly every serious code-switching corpus is spontaneous conversation, because that is where switching lives. Read-speech corpora, the bulk of open ASR data, contain essentially none of it. Nobody writes code-switched prompts, and reading aloud suppresses the behavior anyway. We unpack this data-distribution gap in conversational vs. scripted speech data.
What Actually Works: Strategies for Code-Switching ASR
Massively multilingual models help, partially. Whisper, MMS, and similar models trained on many languages have seen both languages of any given pair, and they degrade more gracefully than monolingual systems. But their training mixtures are still overwhelmingly monolingual per utterance, and they often force one language token per segment, so on dense intra-sentential switching they hallucinate, translate instead of transcribe, or collapse to one language.
Model-side techniques. Published approaches include joint language-ID auxiliary losses, dual-script or transliteration-normalized output vocabularies (so "मीटिंग" and "meeting" don't fight), merged phone sets, and synthetic code-switched text for LM adaptation via translation-and-splice. Each yields incremental gains. Decide your output convention early and encode it in both training transcripts and evaluation references: mixed-script output (each word in its native orthography) is what users of Hinglish or Taglish keyboards expect, while single-script romanized output simplifies downstream NLP. There is no universally right answer, but an inconsistent one guarantees noisy training signal and unmeasurable WER.
Training on real code-switched speech is the dominant factor. Every study on the subject converges on the same conclusion: fine-tuning on in-domain, code-switched conversational audio beats architectural cleverness. The mechanism is simple. The model finally observes the actual distribution: bilingual pronunciation, switch-point acoustics, and mixed-language word sequences. Even tens of hours of genuine code-switched conversation moves mixed error rates substantially; hundreds of hours transform them.
Sourcing it is the hard part. You cannot script code-switching convincingly; elicited "please read this Hinglish sentence" audio sounds wrong and switches in unnatural places. The reliable source is recording real bilingual speakers in free conversation, then transcribing with a mixed-script convention. That is precisely how our corpora are built: spontaneous two-party conversations that contain code-switching at natural rates, with time-aligned transcripts that preserve it. For pairs where switching is heaviest, see our Hindi conversational dataset (Hinglish throughout), Tagalog (Taglish), and Moroccan Arabic / Darija (Darija-French-MSA), or the broader guide to speech data for low-resource languages.
Need training data?
SpeechData.ai offers conversational speech datasets in 60 languages, including heavily code-switched Hindi, Tagalog, Swahili, and five Arabic variants, with dual-channel audio, time-aligned mixed-script transcripts, and full commercial licensing. Explore the datasets or contact us to discuss your language pair.