Hallucination
In speech recognition, a hallucination is text the model produces that nobody said: a plausible phrase invented from silence, noise, or a fragment too short to transcribe. It is the insertion error taken to its extreme.
Every speech model is trained to produce text, so when given audio that contains no speech it still tries. Batch models like Whisper are the well-known case: fed a few seconds of room tone, they will sometimes emit "Thank you for watching" or a full sentence in another language, because those strings were common in training data at the ends of recordings. The output is fluent and confidently wrong.
The same word is used for language models that make up facts, and the mechanism is related: a system optimized to produce likely output will produce it whether or not the input justifies it. In a dictation tool, a hallucination means text you never spoke lands in your document.
When it happens
Silence at the start or end of a recording, which is why key-up capture and voice activity detection matter. Very short utterances, a single word or a cough, where the model has too little to anchor on. Background speech from a video or a colleague, which is real speech but not yours. Long recordings, where some models lose their place and repeat a passage. Each has a known mitigation, and a tool that has not applied them will surprise you eventually.
How good tools prevent it
Run a voice activity detector first and never send non-speech to the recognizer. Trim silence at both ends. Discard results from clips below a minimum length. Watch for the model emitting text with no corresponding audio energy and drop it. On the formatting side, refuse any rewrite that adds a content word the transcript did not contain, so a formatting model cannot hallucinate either. A tool that does all of this will still occasionally mishear you; it should never invent a sentence.
In Flit
Flit checks the model’s rewrite against the transcript: if a content word you said is missing from it, the rewrite is discarded and the plain transcript lands instead. Silence and non-speech never reach the recognizer in the first place.
The details are not left to a model →Questions
Fair questions.
Why did my dictation app write something I never said?
Most likely the recognizer was given audio with no speech in it, such as silence at the end of a recording, or a very short clip, and produced a plausible phrase anyway. This is a known behavior of batch models and it is preventable with voice activity detection and silence trimming.
Can the formatting step hallucinate too?
Yes, if it is a language model rewriting your transcript freely. The safeguard is a rule that the output may remove fillers and fix punctuation but may never add a content word you did not say, and that a rewrite which breaks that rule is thrown away.
Related terms
