Real-time factor
Also: RTF, RTFx
Published
Definition
Real-time factor (RTF) is the time a speech recognition system takes to process audio, divided by the length of that audio. An RTF of 0.5 means one minute of speech is transcribed in 30 seconds; below 1 is faster than real time.
It is the standard way to say how fast a speech model is, independent of how long the recording was. The definition goes back to evaluations run by NIST, the US standards body, which called it the speed factor: total processing time divided by the duration of the source signal.
Its inverse, RTFx, has become the more common figure, because bigger is better and the numbers are easier to read. Both describe the same thing.
The formula
RTF is processing time divided by audio duration. If 10 seconds of audio take 2 seconds to transcribe, the RTF is 0.2.
RTFx turns it over: audio duration divided by processing time. The same example gives an RTFx of 5. The Open ASR Leaderboard, which publishes RTFx for the models it ranks, explains that an RTFx of 1 means a system processes speech as fast as it is spoken, and an RTFx of 2 means it takes half the time.
How to read a figure
Check which of the two you are looking at, because they point in opposite directions.
- RTF: lower is faster. Below 1 keeps up with live speech; above 1 falls behind.
- RTFx: higher is faster. Above 1 keeps up with live speech.
- An RTF of 0.01 and an RTFx of 100 are the same speed: an hour of audio in 36 seconds.
Why published figures are so high
Model cards often quote an RTFx in the hundreds or thousands. Those are measured on a data-centre graphics card with many recordings processed at once. One model’s card, quoting a figure above 3,000, adds that performance may vary depending on audio duration and batch size.
A laptop transcribing one voice gets nothing like that, and does not need to. The figure is useful for comparing models with each other, not for predicting what you will see.
What it does not tell you
RTF is an average over a whole recording. It says nothing about when the first word appears, or how long you wait after you stop speaking, and for dictation those are what you feel.
A model with an excellent RTF that waits for a whole chunk of audio before writing can feel slower than a streaming model with a worse one. For live use, latency is the measure to ask about, and RTF is the check that the model can keep up at all.
What changes it
The same model gives different figures in different conditions.
- The hardware, and whether the model runs on the CPU, the graphics chip or the Neural Engine.
- The size of the model, and whether it has been quantized.
- The length of the audio. Very short clips carry a fixed start-up cost that makes the ratio look worse.
- Batching. Processing many recordings at once raises RTFx sharply on a server.
Questions people ask.
What is a good real-time factor?
For live dictation, anything comfortably below 1, so the model keeps ahead of your speech. For transcribing files, lower simply means less waiting.
What is the difference between RTF and RTFx?
They are inverses. RTF is processing time divided by audio length, so lower is faster. RTFx is audio length divided by processing time, so higher is faster.
How do you calculate real-time factor?
Divide the time taken to transcribe by the duration of the audio. Ten seconds of audio transcribed in two seconds is an RTF of 0.2, or an RTFx of 5.
Does a low real-time factor mean low latency?
Not necessarily. RTF is an average over the recording. A model can have a low RTF and still make you wait, if it reads a whole chunk before writing anything.
What does an RTF of 1 mean?
The system takes exactly as long to transcribe the audio as the audio lasts. It only just keeps up with live speech.
Why is my speed lower than the figure on the model card?
Published figures are usually measured on a server graphics card with many recordings in a batch. One recording on a laptop is a different test.
