Language identification
Also: LID
Updated
Definition
Language identification is the step where speech software works out which language is being spoken before, or while, it transcribes. It is what lets a multilingual tool switch languages without the user changing a setting.
A recognizer that assumes the wrong language produces confident nonsense: French run through an English model comes out as English-shaped gibberish. Traditional dictation solved this with a setting you had to change by hand, which is fine for someone who works in one language and miserable for the many people who switch mid-conversation.
Modern tools run a small detector on the first second or two of audio, pick the most likely language, and route the audio to the right model. The good ones can re-detect on each new dictation, so a Slack reply in German and the next one in English need no intervention.
How detection works
Some ASR models detect the language themselves as a side effect of transcribing; Whisper does this in its first pass. Others rely on a separate, tiny classifier that listens to the opening of an utterance and returns a language with a confidence score. Either way the decision has to be made early, because the wrong model has to be abandoned before it produces much, and it has to be made from very little audio, which is why the first word or two of a dictation is the moment where multilingual tools are most fragile.
Detection is not translation
Identifying the language and transcribing it in that language is the default and the safe behavior. Translation, where you speak one language and the text lands in another, is a separate and optional step, and it should be off unless you turn it on. A tool that translates by surprise has made a decision about your words that was not its to make.
How to dictate in more than one language on a Mac
Apple’s Dictation works with the languages you add to it, and you switch between them yourself while you dictate. Setting it up takes a minute.
- Choose Apple menu → System Settings, then click Keyboard in the sidebar.
- Go to Dictation and click the Edit button next to Languages.
- Select each language and region you want to dictate in.
- Start Dictation in any text field.
- To change language, click the language shown next to the cursor. It appears as an abbreviation or as characters that represent the language.
- Or press the 🌐 key and choose the language you want to use.
How do Whisper and Parakeet detect the language?
Whisper was trained to identify the spoken language as one of its tasks, alongside transcription and translation. When you do not name a language, its transcription tool detects one using up to the first 30 seconds of the audio, then transcribes the whole file as that language. The English-only models skip detection and assume English.
Parakeet TDT 0.6B v3 takes a narrower approach. NVIDIA says it automatically detects the language of the audio and transcribes it without additional prompting, across the 25 European languages it supports. A language outside that set is not supported.
Why does it detect the wrong language, and how to fix it
Detection is a guess made from a small amount of audio, and a few situations reliably trip it. Most of them have a simple fix, which is to tell the model the language yourself.
- The recording changes language partway through. Whisper’s tool detects once, from the opening of the file, and applies that language to the rest.
- The language has little training data. The Whisper model card reports lower accuracy on such languages.
- The language is not one the model supports at all, so it can only answer with a language it does support.
- The fix for Whisper: add --language and the language name to the command, for example --language German.
- The fix for Apple’s Dictation: pick the language yourself by clicking the language next to the cursor.
Questions people ask.
Can dictation software detect which language I am speaking?
Modern tools can, from the first second or two of audio, and route the speech to the right model. Older tools need you to pick the language in a setting before you speak.
What happens if it guesses the wrong language?
You get confident nonsense in the wrong alphabet or spelling, because the model transcribes sounds as the words it knows. Tools that re-detect on every dictation recover on the next sentence; tools with a fixed setting need the setting changed.
How do I change the dictation language on a Mac?
Add the language under System Settings → Keyboard → Dictation by clicking Edit next to Languages. While dictating, click the language shown next to the cursor, or press the 🌐 key and choose the language.
How do I stop Whisper from picking the wrong language?
Name the language in the command, for example whisper audio.mp3 --language German. Whisper then skips automatic detection, which otherwise uses up to the first 30 seconds of the audio.
Can a speech model detect the language automatically?
Yes. Whisper detects the language when you do not specify one, and NVIDIA’s Parakeet TDT 0.6B v3 detects it automatically across its 25 supported European languages.
