Transcribe speech as it happens and label each speaker

You point it at a microphone or a file and it emits text chunk by chunk, so the transcript grows in step with the conversation. Each chunk is attributed to a speaker, so a two-person interview comes out as alternating turns rather than one run-on block. It handles Chinese, English, French, German, Italian, Japanese, Korean, Portuguese, Russian, and Spanish.

The useful extra is hotwords. Hand it a short list of names, product terms, or acronyms before you start and it leans toward those spellings, which is where transcripts of real meetings usually fall apart. Microsoft's earlier VibeVoice transcriber could already tell who said what, but only on a finished recording; this one does it live. Unlike the compressed laptop version we covered a few weeks ago, it is built for a GPU. There are two sizes, 7B and a smaller 1.5B, both under the MIT license.

Open Microsoft's official playground, pick a language, and record a minute of two people talking over a phone speaker. Watch the speaker label switch as they trade turns, then add each person's name as a hotword and check whether the transcript starts spelling them right. That one test shows both things this model is for. To run it yourself, the GitHub repo ships a browser demo that keeps a live connection open while you speak, plus a script for files; you need an NVIDIA GPU, and the 1.5B version is the one for a smaller card.

Speaker labels are where a model like this slips: people talking over each other, or a third voice joining late, can get merged or split. Clean single-speaker audio is not the reason to switch, since plenty of transcribers handle that well. And if your meetings are in a language outside the ten listed, this one is not for you yet.

• Keep live meeting notes with each remark tied to the person who made it.
• Transcribe interviews in Japanese, Korean, or Portuguese with speaker turns intact.
• Caption a panel or podcast where the value is in who is speaking.
• Log support calls with agent and customer separated.
• Get names and jargon spelled correctly by feeding them in as hotwords.

Try the demo

View model page

Read this issue on neuralexpedition.com →