Where the delay goes between the key press and the pasted text

Dictation latency is the gap between letting go of the hotkey and seeing your words. Speech to text latency in that sense is made of about six things, and most people assume the model is all of it. The model is most of it, but not for the reason you would guess.

A second of speech costs almost as much as half a minute

Here is whisper.cpp with large-v3-turbo on an M4 Max, transcribing clips cut from the same recording:

Audio length	Encoder	Total
1 second	233 ms	711 ms
3 seconds	218 ms	642 ms
10 seconds	218 ms	733 ms
26 seconds	218 ms	866 ms

Twenty-six times more audio, twenty-two percent more time. The encoder line is flat because Whisper pads everything to 30 seconds. Its front end is defined that way: N_FRAMES in whisper/audio.py is 3000 mel frames at 10 ms each, which is exactly half a minute. Record two words and the model encodes your two words plus twenty-eight seconds of silence.

For dictation this is the whole story. Phrases are short, and short is where the fixed cost dominates: for a one-second phrase, the model spends more time on padding than on you.

Real time factor is measured where the fixed cost hides

Every engine advertises a real time factor — audio duration over processing time. Our own catalogue numbers, all five models on one 28-second clip through WhisperKit:

Model	RTFx
SenseVoiceSmall (int8)	247.5×
Parakeet TDT-CTC 110M	165.4×
Parakeet TDT v3 (int8)	122.3×
Whisper large-v3-turbo	20.1×
Whisper small	19.3×

Two things in that table matter more than the ranking.

Both Whisper models land in the same place despite one being three times the size. Weights are not what costs the time; the fixed 30-second window is.

The ratio moves with the material. On a 94-second clip the Whisper models drop to about 7.6×, while Parakeet TDT v3 climbs to 176×. A single RTFx number is a measurement of one clip length, and dictation phrases are nowhere near that length. Take it as a rough ordering, not as a promise about your two-second sentence.

Those figures come from WhisperKit inside the app, while the millisecond timings above come from whisper.cpp. Different implementations, not comparable side-by-side — what carries across is the shape: a fixed window cost that short audio cannot amortise.

The first run is a different number entirely

There is a cost you pay exactly once per model, and it is large enough to be mistaken for a hang.

On this machine, the first ever preparation of Whisper large-v3-turbo took 2 minutes 35 seconds. The second took 1.5 seconds. That is CoreML compiling the model for the Neural Engine, with the result cached by the system. SenseVoiceSmall’s first compile took around ten minutes.

That step is documented rather than mysterious: Apple’s guide to downloading and compiling a model on the user’s device notes that Core ML “saves models it compiles to a temporary location” and advises saving the compiled model somewhere permanent “if your app can reuse the model later”. What it does not tell you is how long the compile takes for a model of this size, which turns out to be minutes.

Nothing about the steady-state latency predicts that, and no benchmark reports it, because benchmarks warm up first. If you are judging an app by how long the first dictation took, you are measuring the compiler.

The last mile is cheaper than it looks

After recognition there are three more steps, and together they are noise next to the model.

Post-processing. Punctuation clean-up and the custom vocabulary run over the finished text. The vocabulary holds a pre-compiled snapshot rather than rebuilding its pattern per phrase — compiling is expensive, applying is not.

Getting the text into the app. Kekoso writes the transcript to the pasteboard as a promise rather than as data, posts a synthetic Command-V, and waits for the receiving app to ask for the content. That request is the receipt: proof the paste actually happened. The wait is polled every 20 ms, so up to 20 ms of the delay you feel is this.

Restoring your clipboard. There is a 200 ms pause before the previous clipboard contents go back, and it is the one number here that does not count as latency: your text is already on screen. The pause exists because applications often read the pasteboard more than once, and restoring instantly would lose the race against a second read.

What this means when choosing

For dictation, model choice is the whole game. Between Parakeet at 122× and Whisper at 20× there is roughly a sixfold difference in steady-state speed, and on short phrases Whisper’s fixed window widens the gap further. If dictation latency is what you care about, that is the decision.

Transcription speed on long files is not the same problem as dictation latency. For a one-hour recording the fixed window is amortised to nothing and Whisper’s accuracy on long-tail vocabulary may be worth waiting for. The right engine depends on the length of what you feed it, which is why an app that ships one model has already made the choice for you.

Do not judge on the first run. Two and a half minutes of CoreML compilation tells you about your Neural Engine cache, not about the app.

---

The accuracy side of the same choice is in Whisper vs Parakeet vs SenseVoice, and Whisper’s other short-audio failure — inventing text when the window is mostly padding — is measured in why Whisper says “Thank you” to an empty room. If you want to run timings like these yourself, the method is in how to measure word error rate on your own audio.

Questions people ask

How fast is Whisper on a Mac?

Faster than real time, but not in the way the ratio suggests for short phrases. On this M4 Max, whisper.cpp with large-v3-turbo took 711 ms for one second of speech and 866 ms for twenty-six seconds — because the encoder always processes a 30-second window, whatever you actually recorded.

Why does a short dictation take as long as a long one?

Whisper pads every input to 30 seconds. Its front end produces 3000 mel frames at 10 ms each by design, so a two-second phrase is encoded as 28 seconds of silence plus your voice. My measured encoder time was 218 ms whether the clip was 1 second or 26.

What is real time factor and why is it misleading?

RTFx is audio duration divided by processing time — 20x means a minute of audio takes three seconds. It is measured on long clips, so it hides the fixed costs that dominate short ones. The same Whisper models measured 20x on a 28-second clip and 7.6x on a 94-second one, while Parakeet went the other way, up to 176x.

Which model gives the lowest dictation latency?

In our catalogue measurements on one 28-second clip: SenseVoiceSmall 247x, Parakeet TDT-CTC 110M 165x, Parakeet TDT v3 122x, and both Whisper models around 20x. For dictation, where phrases are short, that gap is the single biggest decision you make about latency.

Why is the first transcription after installing a model so slow?

CoreML compiles the model for the Neural Engine on first use, and the system caches the result. On this machine the first ever prepare() of Whisper large-v3-turbo took 2 minutes 35 seconds; the second took 1.5 seconds. SenseVoiceSmall's first compile took about ten minutes.

Does pasting the text add noticeable delay?

Barely. The text goes onto the pasteboard as a promise, a synthetic Command-V is posted, and the app polls for the receipt every 20 ms. There is a 200 ms pause afterwards, but it happens after your text has already appeared — it is there so the original clipboard contents are not restored before the receiving app has finished reading.