Discover: how offline speech recognition actually works
Acoustic model, language model, and why small local models mishear the words they mishear.
Two separate problems, one pipeline
An automatic speech recognition (ASR) engine solves two different problems in sequence, and mixing them up is the single most common source of confusion about what these systems can and cannot do.
The first is acoustic, the second is linguistic. They are trained on different data, and they fail in different ways.
The acoustic model
Answers: what phonemes does this waveform most resemble? It has never read a sentence in its life.
It was trained on pairs of (audio, phoneme sequence) and learned a purely acoustic mapping — the raw sound of a “t” versus a “d”, independent of any meaning.
The language model
Answers a completely different question: given these candidate phoneme or word sequences, which one is an actual sentence of the language?
It was trained on text, not audio. It knows that “two points” is far more likely to follow “add” than “de points” is, in a context about punctuation — even if the two sound almost identical.
Vosk is built on Kaldi, a toolkit that couples exactly these two components: a neural acoustic model and a statistical language model.
They are joined through a weighted decoding graph that searches for the overall most likely word sequence given both the sound and the grammar.
Why the small model mishears “deux points”
The acoustic part of a small model is not meaningfully worse at recognising phonemes — the bottleneck is the language model bundled with it.
vosk-cli-dictation ships with Vosk’s small models (about 50 MB per language) rather than the large ones (over a gigabyte). That bundled language model is a much smaller, much less contextual statistical model of the language:
- A large language model has seen enough real sentences to strongly prefer “deux points” over “de points” in a sentence about formatting, because it has effectively seen that pattern before.
- A small one has seen far less, so when two candidate word sequences sound almost identical — a true homophone, or a near-homophone under casual pronunciation — it has much less signal to break the tie correctly.
This is exactly why vosk-cli-dictation’s config.yaml has a recognition_aliases table: it is not a workaround for a bug, it is the practical answer to a real, structural property of small acoustic-only disambiguation.
You correct the specific homophones your own voice and vocabulary actually trigger, once, and the tool stops making that particular mistake for you.
What a bigger model buys you, concretely
Better contextual disambiguation is paid for in compute, on every single utterance, for as long as the process runs. There is no free improvement here.
Small model — the default
About 50 MB per language. Keeps the acoustic principle, and runs comfortably in the background on ordinary hardware.
Large model
Swapping the small French model for vosk-model-fr-0.22 is roughly 1.5 GB instead of ~50 MB. Far more context for the language model — at the direct cost of more CPU and RAM at runtime, since a larger graph is being searched at every partial result.
The trade-off this project makes, explicitly
vosk-cli-dictation defaults to the small models specifically so it runs comfortably in the background on ordinary hardware.
It treats the inevitable homophone errors as a configuration problem — solved once per recurring mistake, in a YAML file you can read and edit — rather than a model problem to be thrown more compute at.
Whether that trade-off suits you depends entirely on your machine and your vocabulary.