What Breaks When You Train AI on Synthetic Taste
From My Weird Prompts · 27:18
Show by Daniel Rosehill
Training a model on 200 to 500 synthetic movie plots sounds like plenty of data, given that the average Netflix Prize user rated 208 films. Yet sample size is not what breaks a personal taste model. Replacing real films with synthetic plots traps the model in a narrow pocket covering under 10 percent of real preference diversity, while personalized reward models often learn surface correlates rather than actual preferences. The discussion also examines why fixed checkbox reasons and annotation fatigue distort the signal. Key sources: • Wang et al., Using Context Is Not Enough: Test-Time Training for Personalized Reward Modeling (P-TTT), submitted 2026-09-28. — https://arxiv.org/abs/2609.35109v1 • The Persona Fidelity Gap: Behaviorally Grounded LLM Personas Still Compress Real-User Preference Diversity, ICML 2026. — https://icml.cc/virtual/2026/75683 • Ryan et al., SynthesizeMe! Inducing Persona-Guided Prompts for Personalized Reward Models in LLMs, ACL 2025, submitted 2025-06-05. — https://arxiv.org/abs/2506.05598v1 Episode 6604 — myweirdprompts.com/n/6604 (permanent short link: myweirdprompts.com/119663) Listen on Android: MWP FM Player — myweirdprompts.com/apk
More episodes of My Weird Prompts
More episodes
- How Multi-Speaker TTS Models Keep Voices From Blending
- Merging LoRA Voice Adapters vs Keeping Them Separate
- What Goes Into a Character-Voice LoRA Adapter
- Image Annotation When Experts Are the Bottleneck
- How the Interface Shapes Data Annotation Tools
- Where RLHF Preference Annotation Tools Stand
- How Humans Actually Grade Generated Video
- Quantization Quietly Deletes Fine-Tuned Skills
- How Water Dowsing Works Without Finding Water
- Why Optimizing Voice Quality Erases Identity
PodSearch is a free, independent podcast search engine. Read about PodSearch, the privacy policy or contact us.