Understand any audio and answer in text or speech

You hand Audex a piece of audio and a question, and it answers like a chat model that happens to have ears. Ask for a transcript and you get one. Ask what a foreign-language speaker said and it translates into English. Ask what is happening in a field recording and it describes the scene: footsteps, rain, a door closing. Flip the direction and it generates audio instead: type a script and it reads it aloud, or produces a sound effect from a description.

The difference in practice is that these were separate tools until now. A transcriber could not answer follow-up questions about the meeting it just transcribed; a text-to-speech model could not hear. Because Audex is one model, you can stay in a single conversation: transcribe an interview, ask which speaker sounded uncertain, then have it read a summary out loud.

It comes from NVIDIA's Nemotron lab as a compact 2B-parameter model, the small sibling of a much larger 30B version, and it keeps the ordinary chat and reasoning skills of its text backbone, including a step-by-step thinking mode.

The official demo Space on Hugging Face is the fast path: upload a short clip with layered sounds, a street scene or a meeting snippet, and ask it to describe what it hears, then ask a follow-up about one detail. That two-step is exactly what single-purpose audio tools cannot do. Running it locally is a heavier lift: it needs a GPU and a vLLM setup with a custom plugin, so treat the demo as the real entry point.

The license is NVIDIA's one-way noncommercial license: fine for testing and research, not for shipping inside a product. The model is English-focused, and this 2B edition skipped the reinforcement-learning polish its 30B sibling received, so expect rougher answers on hard questions. Test the speech output early; small unified models often sound flatter than dedicated text-to-speech systems.

• Transcribe an interview, then ask follow-up questions about what was said in the same chat.
• Translate a foreign-language voice note into English text.
• Get a written description of a field recording or sound effect.
• Draft a voiceover by typing a script and getting spoken audio back.
• Replace a three-tool audio chain with one model that hears and speaks.

Try the demo

View model page

Read this issue on neuralexpedition.com →