Speaker Diarization

From Transcription to Live Music: Gemini's Audio Stack — Thor Schaeff, Google DeepMind

Jun 9, 2026 · 19:34

Thor Schaeff from Google DeepMind presents the Gemini audio stack—Gemini 3 Flash Preview for deep audio understanding, Gemini 3.1 Flash Live for real-time sound-to-sound multimodal interaction, and Lyria 3 for music generation. He shows how a single API call extracts speaker labels, timestamps, emotions, language detection, and translation, and how speech generation uses a 'director's note' to modify a base voice's accent and tone. The talk culminates in a live demo where the Gemini Live model uses Lyria via tool calls to generate a German techno schlager about the UK startup scene.

Beyond Transcription: Building Voice AI That Understands Conversations — Hervé Bredin, pyannoteAI

Jun 5, 2026 · 25:20

Hervé Bredin, chief science officer at pyannoteAI, argues that speaker diarization benchmarks are misleading because they use headset audio while users rely on table microphones—Nvidia Parakeet reports 11.4% word error rate on AMI headset data but gets 26% on the same dataset's table mic. The episode covers how speaker diarization (who speaks when) is harder than it looks, especially when combining with transcription: overlapping speech, timestamp disagreements, and words falling between speaker boundaries create errors. Bredin demonstrates pyannoteAI's Precision 2 model achieving 3% diarization error rate (DER) against a 5% baseline on a two-speaker phone call. State of the art today: 2% DER on clean telephone calls but 41% in a noisy restaurant, showing the problem is far from solved. The reconciliation between diarization and STT is handled by a proprietary orchestration that works with any STT model.

The End of Awkward AI Transcriptions - Travis Bartley and Myungjong Kim

Jun 3, 2025 · 16:24

NVIDIA speech AI researchers Travis Bartley, Myungjong Kim, and Jae-hun detail how their Fast Conformer backbone powers both streaming (Parakeet) and multitask (Canary) models, achieving top-5 Hugging Face ASR leaderboard rankings by prioritizing customization over one-size-fits-all. They explain CTC/TDT for low-latency streaming and attention encoder-decoder for high-accuracy multitask models, with Subformer enabling unified speaker diarization and target-speaker ASR. Training uses open-source and proprietary data with pseudo-labeling via the Nemo toolkit. Deployment on NVIDIA Riva NIM with TensorRT optimization supports low-latency streaming and offline processing. Customization includes fine-tuning acoustic models, language models, punctuation, and inverse text normalization for domain-specific terms like medical and food ordering.