I gave an agent a folder of recordings and let it fix its own transcripts

Everyone demonstrating agent workflows shows the easy half: point the agent at a folder, watch transcripts appear. That part works and takes one tool call per file. The interesting half is what you do about the transcripts being wrong.

I recorded three standup items, ran them through a local transcription tool from a coding agent over MCP, and then had the agent repair its own output. Batch transcription is the trivial part of that; the repair is where the work is. Here is what the first pass produced.

Twenty-five errors in two hundred words

The recordings are synthetic speech, so the reference text is known exactly and the word error rate is arithmetic rather than judgement. The terms are ours — product names, a couple of colleagues, the sort of vocabulary that fills a standup.

What was said	What came back
Kekoso	Kakuzo
Siobhan	Yophan, and Syophan in a second file
Grafana board	Gryphon abort
rollback	robux
kubectl	Q-book
namespace	Nemo space
Parakeet	Perikid
SenseVoice	sans voice
Kubernetes	QBOR needs

Across the three files: 25 errors in 200 words, a 12.5% error rate. Almost all of it is terminology. The ordinary English is nearly perfect.

Two of these deserve a second look. rollback became robux — a currency in a children’s game, which is a plausible thing to hear and a bizarre thing to read in a deployment note. And Grafana board became Gryphon abort, which is not a mangled word but a grammatical phrase: “Siobhan is watching the Gryphon abort” parses fine, means something else entirely, and gives a reader no signal that anything went wrong.

The rule points backwards, and that changes who should write it

Custom vocabulary speech recognition works differently from how people assume, and the difference decides who should be doing the work.

Here is the part that surprised me, and the reason this is an agent task rather than a settings screen.

A vocabulary rule maps the mistake to the correct word — Kakuzo becomes Kekoso — because the replacement happens in the text after recognition. Which means you cannot write the rule in advance. You do not know that the model hears Kakuzo until it has heard Kakuzo. Setting up a custom vocabulary before you transcribe anything is not merely optional, it is impossible.

So the real loop is: transcribe, read the output, notice a word that is wrong, know what it should have been, write the rule. Nobody does that by hand for eleven terms. An agent does it in one pass, because it is holding both halves at once — it has just read the transcript, and it knows what your product is called.

The eleven rules it wrote went in as a single batch call, and the tool reported each outcome separately.

What that bought: 12.5% to 4.0%

Re-running the same three recordings with the vocabulary in place:

Recording	Errors before	Errors after
Deploy (73 words)	17 — 23.3%	7 — 9.6%
Models (70 words)	5 — 7.1%	0 — 0%
Support (57 words)	3 — 5.3%	1 — 1.8%
Total (200 words)	25 — 12.5%	8 — 4.0%

Two thirds of the errors gone, and one file came out perfect. What is left is not terminology at all: standup heard as “stand up”, ran clean as “run clean”, behind a canary as “behind the canary”. Those are the ordinary slips of speech recognition, and a vocabulary is not supposed to fix them.

The deploy file stayed worst because it was densest — six specialist terms in seventy-three words is more jargon than a real standup usually carries.

One name, two mistakes, and why the folder matters

Siobhan came out Yophan in the first recording and Syophan in the second. Nguyen was wrong in the first file (Nuyen) and completely correct in the third.

That is the argument for doing this over a folder instead of a file — and the reason to transcribe multiple files in one go before writing a single rule. To fix names in transcripts reliably, one rule per name is not enough — you need a rule per way the model gets the name wrong, and you only see the variants by looking at several recordings at once. Fix one file and the same person will be misspelled differently in the next one; worse, searching your archive for either spelling finds one recording and misses the other, so the archive quietly loses track of who was in which meeting.

Three things worth knowing before you build on this

Multi-word rules do not reach the timed segments. sans voice to SenseVoice fixed the flat transcript and left the timed words as “sans” and “voice”, and the same happened to Nemo space and QBOR needs. Single-word rules fixed both. If you are generating subtitles from segments, multi-word replacements will not be in them yet — we have filed that.

Case follows the mistake. The model wrote “Q-book” with a capital, so the replacement came out Kubectl rather than kubectl. For a shell command that is wrong in a way a reader will notice.

Re-transcribing the same file returns the old text. The queue caches a finished result by source, so calling the tool on the same path after fixing the vocabulary gave me the previous transcript, unchanged. It is deliberate — dropping a file in twice should not cost you the work twice — but it means an agent has to either transcribe once with the rules already in place, or apply the replacements to the text it already holds.

The one thing the agent cannot do

Adding and updating rules go through. Removing one does not: the tool returned a pending request, said in plain words that nothing had happened yet, and waited for a human to confirm it in the app. Everything above happened without me touching anything, and deleting a single test rule is where that stops.

That asymmetry is the right shape for an agent transcription workflow. An agent that can enrich your vocabulary without asking is useful. An agent that can quietly empty it is not.

---

The tools used here are described on the MCP transcription server page; the design decisions behind them, including why deletions work that way, are in what I learned building it. If you want the underlying accuracy picture rather than the vocabulary one, compression against word error rate measures how much the recording itself matters.

Questions people ask

How do I batch transcribe audio files?

Point something at the folder rather than dragging files one at a time. In this run a coding agent called a local transcription tool once per file over MCP — three files, three calls, no upload and no per-minute meter. Any script would do the same; the reason to use an agent is what happens after the transcripts come back.

Does a custom vocabulary actually improve accuracy?

Substantially, for the words it covers. Across three standup recordings the word error rate went from 12.5% to 4.0% — twenty-five errors down to eight — after eleven vocabulary rules. One of the three files went from 7.1% to zero errors. The remaining mistakes were ordinary speech slips, not terminology.

Why can't I set up a custom vocabulary in advance?

Because the rule maps the mistake to the correct word, not the other way round. You cannot write a rule for Kekoso until you know the model hears Kakuzo. That means transcribing first, reading the output, and writing rules from what actually went wrong — a loop nobody does by hand, and the exact loop an agent is good at.

Will the same word always be misheard the same way?

No, and that is the trap. The name Siobhan came out as Yophan in one recording and Syophan in another. Nguyen was wrong in one file and correct in another. One rule per name is not enough; you need one rule per way the model gets it wrong, which is an argument for looking at a whole folder rather than one file.

Do vocabulary fixes reach subtitle exports?

Single-word rules do. Multi-word rules currently fix the flat transcript but not the timed segments, so a rule like sans voice to SenseVoice will show up in the plain text and not in an SRT or VTT export. Worth knowing before you build a subtitle workflow on top of multi-word replacements.

Can an agent delete my vocabulary rules?

Not on its own. Removal is filed as a request that you confirm in the app; the tool returns a pending state and says explicitly that nothing has happened yet. Adding and updating rules go through immediately, deleting them does not.