Mistral AI Announces Voxtral 3 Audio Models and Voice Agent SDK

Mistral AI Announces Voxtral 3 Audio Models and Voice Agent SDK

At a Glance

Item Value
Publisher Mistral AI
Published 2026-10-20
Source type Primary source (the publisher itself)

Values determined by this site’s code when the information was collected. Dates are JST.

Overview

Mistral AI has announced “Voxtral 3", a model family comprising Speech-to-Text (STT) and Text-to-Speech (TTS) models, along with a voice agent SDK. It combines the ability to perform high-accuracy transcription with audio input and generate emotionally rich, natural-sounding voice from text.

Designed for real-time multi-step conversational processing, this model family supports over 50 languages. In addition to supporting self-deployment to proprietary infrastructure, it is also available via Mistral AI’s EU-hosted API, enabling the construction of low-latency dialogue systems while ensuring data sovereignty.

Specifications

The main specifications stated in the published materials are as follows:

  • Parameter count: Voxtral Mini Transcribe 3 has 4B (4 billion) parameters.
  • Input/Output specifications:
  • Speech recognition: Outputs text from audio input. Supports over 50 languages (including 24 official EU languages and low-resource Asian and Indian languages). – Speech synthesis: Generates speech from text input. Supports over 35 languages and 50 regional dialects.
  • Latency: The real-time speech recognition model (Voxtral Realtime Transcribe 3) performs transcription with a low latency of under 500 milliseconds (sub-500ms).
  • Key speech recognition features: Features speaker diarization, context biasing to improve domain-specific vocabulary accuracy, and end-of-turn detection that identifies the end of an utterance from acoustic and semantic information.
  • Key speech synthesis features: Supports automatic adaptation of emotion, pace, and emphasis according to conversational context, professional voice adaptation using 30+ minutes of audio material (Pro Voice), custom instruction following (emotion, pacing, emphasis), and a vocabulary library to control pronunciation. Voice presets for 18 languages are also provided.
  • Distribution and usage format: Available via Mistral AI’s EU-hosted API or self-deployment to proprietary infrastructure. Published API pricing includes $0.0035 per minute for Voxtral Mini Transcribe 3 (batch), $0.008 per minute for Voxtral Mini Transcribe Realtime 3, $0.0040 per minute for Voxtral Small Transcribe 3, and $13 per million characters for Voxtral Speech 3.

Performance

Mistral AI, the publishing organization, has released evaluation results using various benchmarks. In the field of speech recognition, multilingual evaluations were conducted using Word Error Rate (WER: lower is better) and Character Error Rate (CER: lower is better) for transcriptions.

The speech recognition model Voxtral Small Transcribe 3 is reported to have achieved higher multilingual transcription accuracy than Whisper, Scribe v2, and Gemini 3.5 Transcribe on the multilingual speech recognition benchmark “FLEURS 45" (language-specified and language-auto-detection). Evaluated by CER for Japanese, Korean, Thai, and Chinese, and by WER for other languages, it showed excellent results in major languages including English, French, and Spanish.

It also showed high accuracy in public benchmarks (AMI-IHM, AMI-SDM, Switchboard, Switchboard CallHome, AISHELL-4) using the concatenated minimum-permutation word error rate (cpWER: lower is better), an evaluation metric for speaker diarization. Furthermore, in context biasing evaluations (Earnings 22) that provide specialized terminology or speaker names as input, it was shown that providing context significantly improves keyphrase recall for specialized terms.

For the speech synthesis model Voxtral Mini TTS 3.0 (Voxtral Speech 3), evaluations of cost-performance and quality compared to OpenAI TTS-1, Deepgram, ElevenLabs Flash, and Gemini Flash were presented. It is explained that high quality and cost-effectiveness are achieved across multifaceted items, such as customer support evaluation of numbers and proper nouns using “Voice Code Bench" by BeSimple AI, pronunciation control rate with intended phonemes, multilingual evaluation in 11 languages, and expressive English evaluation.

Strengths and Use Cases

Voxtral 3 is optimized for building advanced conversational applications such as contact center operations, internal service desks, virtual assistants, and hands-free copilots. Rather than just standalone speech recognition or text-to-speech processing, it is primarily intended for automating real-time, multi-step tasks.

Specifically, by combining it with the new voice agent SDK based on LiveKit, processes for “listening" (speech recognition), “thinking" (inference and tool calling), and “speaking" (speech synthesis) can be linked as a single streaming pipeline. Each turn is processed with a low latency of under one second, and a mechanism is provided for the agent to immediately stop speaking when the user interrupts.

Specific use cases during calls or dialogues include the following operational integrations:

  • Caller identity verification and inquiry tasks
  • Confirmation and guidance of billing information
  • Automated arrangement of interviews and reservations utilizing function calling, information retrieval from internal systems, and web searches

In addition to supporting calls via telephone lines (SIP) or browser/mobile applications (WebRTC), it can also connect directly with existing telephony infrastructure, IVR (interactive voice response systems), and contact center software.

On the speech recognition front, by providing domain-specific vocabulary such as specialized terminology or speaker names as context bias, it is suited for operations that transcribe industry-specific terms with high accuracy without requiring fine-tuning.

On the speech synthesis front, it excels at speaking by automatically determining emotion and style from conversational context. In addition to explicitly specifying emotions, speech pace, and words to emphasize through prompts, it also supports non-standard pronunciation control using IPA (International Phonetic Alphabet). Furthermore, it features a professional voice adaptation function (Pro Voice) that faithfully reproduces speaker-specific characters, emotional expressions, and speaking styles across multiple languages by preparing 30 minutes or more of audio material. Detailed voice guidance tailored to over 50 regional dialects and local customs (such as reading dates, times, addresses, and currencies) is possible.

How to Get It

Voxtral 3 models and voice agent features are provided in multiple formats depending on use case and infrastructure requirements.

For cloud-based usage, they can be accessed through Mistral AI’s API from European, US, and global endpoints. Additionally, browser-based “Studio playgrounds" are available to directly test the operation of voice agent dialogues, speech transcription, text-to-speech, and voice cloning.

For organizations that cannot export sensitive voice data externally, self-deployment to proprietary infrastructure is supported. By deploying the full stack in an on-premises or private cloud environment, organizations can completely manage the processing location and retention policy of voice data themselves, operating while maintaining data sovereignty. Those wishing to deploy the full stack on-premises or in a private cloud are guided to contact the Mistral AI team.

In addition, by using the voice agent SDK built on LiveKit, speech recognition with Voxtral Mini Transcribe Realtime 3, inference and tool calling with general-purpose language models like Mistral Small, and speech synthesis with Voxtral Speech 3 can be combined into a single pipeline. Because each stage of the pipeline is separated as an independent model, it is possible to flexibly swap out the backend LLM, generated voice, and supported languages for operation without rewriting the entire agent implementation.

Related Articles

What to Read Next

Sources