Streaming Attention Approximation via Discrepancy Theory
From Google TechTalks · 40:01
Show by Google TechTalks
A Google TechTalk, presented by Ekaterina Kochetkova, 2025-10-23ABSTRACT: The memory requirements of LLM inference grow rapidly with the context length due to the demands of attention computation. We present BalanceKV - an algorithm that leverages the geometric properties of the key-value cache to compress it without significantly affecting the quality of attention computation. BalanceKV has strong theoretical guarantees grounded in discrepancy theory and demonstrates empirically validated performance improvements over existing methods. The full paper is available at arXiv:2502.07861.Uploaded 27-10-25About the Speaker: Ekaterina Kochetkova is a third year CS PhD student at EPFL working with Michael Kapralov. She is broadly interested in applying theoretical insights to develop efficient algorithms for large-scale machine learning. Her recent work focuses on optimizing the memory/runtime of LLM inference and on sublinear graph clustering methods that utilize learned vertex features. More information is available at https://ekaterina-kochetkova.github.io/e_kochetkova.github.io/.
More episodes of Google TechTalks
More episodes
- Making Great Decisions
- Streaming Private Continual Counting via Binning
- Distributed Testing with SmartFrog
- Differentially Private Synthetic Data without Training
- A Dynamic Network Layer for Advanced Cloud Computing
- Data Representation/Laplace Operator
- Persistent Pre-Training Poisoning of LLMs
- Threat Models for Memorization: Privacy, Copyright, and Everything In-Between
- Technology Assisted Reading Acquisition: Children Acquiring Literacy Naturally
- sex on the internet, the realities of porn, sexual privacy,
PodSearch is a free, independent podcast search engine. Read about PodSearch, the privacy policy or contact us.