Robust Distortion-free Watermarks for Language Models
From Google TechTalks · 1:00:55
Show by Google TechTalks
A Google TechTalk, presented by John Thickstun, 2024-07-02A Google Algorithms Seminar. ABSTRACT: We describe a protocol for planting a watermark into text generated by an autoregressive language model (LM) that is robust to edits and does not change the distribution of generated text. We generate watermarked text by controlling the source of randomness--using a secret watermark key--that the LM decoder uses to convert probabilities of text into samples. To detect the watermark, any party who knows the key can test for statistical correlations between a snippet of text and the watermark key; meanwhile, our watermark is provably undetectable by anyone who does not know the watermark key. We instantiate our watermarking protocol with two alternative decoders: inverse transform sampling and Gumbel argmax sampling. We apply these watermarks to the OPT-1.3B, LLaMA 7B, and instruction-tuned Alpaca 7B LMs to experimentally validate their statistical power and robustness to various paraphrasing attacks. ArXiv: https://arxiv.org/abs/2307.15593Uploaded 09-07-24ABOUT THE SPEAKER: John Thickstun is a postdoctoral researcher at Stanford University, working with Percy Liang. Previously, he completed a PhD at the University of Washington, advised by Sham M. Kakade and Zaid Harchaoui. His current research focus is to improve the capabilities and controllability of generative models. His work has been featured in media outlets including TechCrunch and the Times of London, recognized by outstanding-paper awards at NeurIPS and ACL, and supported by an NSF Graduate Fellowship and a Qualcomm Innovation Fellowship.
More episodes of Google TechTalks
More episodes
- Curating the Dark Data in the Long Tail of Science
- Monads and Gonads
- SkyLib: A Global, User-based Library for Everything
- SV-WEB-JUG: Spring 2.5 and Spring Application Platform
- Hyperproductive Distributed Scrum Teams
- 360 Video
- Multi-Party Computation: From Theory to Practice
- The Extended Mind: Recent Experimental Evidence
- DSD: A Hybrid Analysis Tool for Bug Finding
- Chasing the Constants and its Implications in Differential Privacy
PodSearch is a free, independent podcast search engine. Read about PodSearch, the privacy policy or contact us.