Two engines, one recording, and 130 ms of disagreement about when a word starts

Two speech engines both advertise word-level timestamps. Give them the same 26-second recording and they will not agree on when the words happened.

I measured it: 130 ms of disagreement at the median, 460 ms at the worst, on 41 words that both engines transcribed identically. Neither is broken. Speech recognition timestamps simply arrive by different routes, and word level timestamps accuracy is a property of the route, not of the audio.

The measurement

Transcription timestamps come out of both engines in the same shape — a start and an end per word — so they look directly comparable. One file: a 26-second synthesised standup item, 16 kHz mono. Whisper large-v3-turbo through whisper.cpp with word-level output, Parakeet TDT v3 through the app that ships it. The transcripts were aligned by sequence matching and only the words both engines produced identically were compared, which left 41 of them.

	Whisper large-v3-turbo	Parakeet TDT v3
Timestamp quantisation	10 ms	20 ms
Words emitted	74	47
Words shorter than 60 ms	10	0
Shortest word	0 ms	80 ms

And the disagreement between them, on the 41 matched words:

Measure	Value
Median difference in word start	130 ms
Mean difference	176 ms
Worst case	460 ms (“watching”)
More than 100 ms apart	28 of 41
More than 200 ms apart	14 of 41

Part of it is a shift, part of it is scatter

The disagreement is not random. Whisper places words earlier than Parakeet in 28 of 41 cases, and the size of that lead is remarkably stable: 110 ms at the mean and 110 ms at the median, the two agreeing to the millisecond.

So you might think you could calibrate it away. Subtract the 110 ms and the residual disagreement is still 140 ms at the median and 350 ms at the worst. A systematic offset you can correct sits on top of genuine scatter you cannot.

Why the numbers come out different

Whisper timestamps accuracy and Parakeet’s are limited by different things, and the difference starts at the front end.

Whisper infers timing after the fact. Its audio front end is fixed: SAMPLE_RATE = 16000 and HOP_LENGTH = 160 in whisper/audio.py, which the file itself annotates as FRAMES_PER_SECOND ... # 10ms per audio frame. That is exactly the quantisation I measured — every Whisper timestamp in my run is a multiple of 10 ms, because the grid it can land on has 10 ms spacing. Word boundaries are then recovered from the model’s attention over that grid, which is an inference about where a word probably began, not a decision the model made while decoding.

Parakeet emits timing as part of decoding. TDT stands for a transducer that jointly predicts tokens and durations — the model card cites the paper “Efficient Sequence Transduction by Jointly Predicting Tokens and Durations”. The duration of each token is an output of the model, not a post-hoc alignment. My measurement showed its values quantised to 20 ms; the model card does not state the encoder frame rate, so I will not claim to know why that number rather than another.

The practical consequence of that architectural difference shows in the degenerate cases. Whisper produced ten words shorter than 60 ms, including “That” with a duration of exactly zero and “a” and “fix” at 10 ms each. No human says a word in ten milliseconds. Those are artefacts of fitting boundaries to a grid after the fact. Parakeet, which predicts duration directly, produced nothing below 80 ms in the same file, with a median word of 320 ms.

When 130 ms matters and when it does not

For finding a passage: irrelevant. If you are clicking a transcript to jump to a moment in a recording, a tenth of a second is invisible. Both engines are far better than you need.

For subtitles: it matters. Subtitle timing accuracy is the case where a tenth of a second is visible. Reading guidelines put the minimum time a cue stays on screen at about 0.8 seconds. A 130 ms error is a sixth of the shortest cue you will ever write; the 460 ms worst case is more than half of it. Subtitles that lead or trail the speech by that much read as out of sync even to people who cannot say why. If you are exporting SRT or VTT for anything public, check the timing against the video rather than trusting the numbers.

For cutting audio automatically: it matters most. Slice a recording on word boundaries taken from the transcript and a 460 ms error clips the front of a word or leaves half of the previous one. Whisper’s zero-duration “That” would produce an empty clip.

For research and legal use: know which engine produced them. A transcript with timings that another engine would place 130 ms elsewhere is fine as a reference, and is not a measurement. If timing is evidence, it needs hand verification, and that has always been true of machine transcripts.

---

The engines differ in more than timing, and the rest of the comparison is in Whisper vs Parakeet vs SenseVoice. If you are heading for subtitles, the cue-splitting rules that turn word times into readable lines are in the subtitle generator. And Whisper’s other timing artefact — inventing words where there is no audio at all — is measured in why Whisper says “Thank you” to an empty room.

Questions people ask

How accurate are word-level timestamps from speech recognition?

Accurate enough to find a passage, not accurate enough to trust to the frame. Two engines transcribing the same 26-second recording disagreed about when a word starts by 130 ms at the median, and by 460 ms in the worst case. Both call the feature word-level timestamps.

Why do Whisper and Parakeet give different timestamps for the same audio?

They derive them differently. Whisper's mel spectrogram advances 10 ms per frame, and word times are inferred from the model's attention afterwards. Parakeet TDT predicts token durations as part of decoding — the architecture is a transducer that jointly predicts tokens and their durations. One infers timing after the fact, the other emits it directly.

What is the time resolution of Whisper timestamps?

10 ms, and that is visible in the output: every value is a multiple of it. It comes from the audio front end, where HOP_LENGTH is 160 samples at a 16 kHz sample rate — exactly 100 frames per second. Parakeet's output in my measurement quantised to 20 ms.

Is one engine's timing more correct than the other?

Neither is ground truth, and I did not have hand-labelled boundaries to compare against. What I can say is that Whisper placed words earlier than Parakeet by about 110 ms on average, and that Whisper produced ten words shorter than 60 ms — including one of zero duration — while Parakeet produced none.

Does this matter for subtitles?

It does. Reading guidelines put minimum cue duration at around 0.8 seconds, so a 130 ms error is a sixth of the shortest cue you will ever write, and a 460 ms error is more than half of it. For subtitles, check the timing rather than trusting it.

Can I just apply a constant offset to fix the difference?

Only partly. The offset between the two engines was consistent enough to measure — 110 ms at both the mean and the median — but subtracting it leaves a median residual of 140 ms and a worst case of 350 ms. Part of the disagreement is a shift you can calibrate, and part of it is genuine scatter you cannot.