Are Speech-to-Speech Models Really Single Models?
From My Weird Prompts · 22:04
Show by Daniel Rosehill
Speech-to-speech AI promises to replace the clunky cascade of speech-to-text, LLM reasoning, and text-to-speech with a single native model. Examining open-weights systems like Kyutai's Moshi, Sesame's CSM, and Qwen3-Omni reveals what is actually under the hood. Rather than eliminating the traditional components, these systems compress encoders, adaptors, and decoders into a co-trained pipeline inside one set of weights. The investigation breaks down duplex token streams, internal text monologues, and where speaker diarization really happens. Episode 6738 — myweirdprompts.com/n/6738 (permanent short link: myweirdprompts.com/130763) Listen on Android: MWP FM Player — myweirdprompts.com/apk
More episodes of My Weird Prompts
More episodes
- Speech-to-Speech Voice Agent Pricing and Architecture
- Can You Just Buy AI Guardrails as a Service?
- Turn Detection and Barge-in in Voice AI
- When Chatbots Call the Police
- Smuggling Soviet Secrets as Sheet Music
- Open Source Voice Assistants and MCP
- Why Broadcast Headsets Still Use Weird Connectors
- Prompt Privacy in AI Inference Marketplaces
- How Canarytokens Work as Digital Tripwires
- Why Computers Can't Disable Headset Audio Processing
PodSearch is a free, independent podcast search engine. Read about PodSearch, the privacy policy or contact us.