Paper Accepted: Swift-SVD

Our paper “Swift-SVD: Theoretical Optimality Meets Practical Efficiency in Low-Rank LLM Compression” has been accepted to ICML 2026 and is available on arXiv. Swift-SVD studies low-rank compression of LLM weights, and proposes a method that is both theoretically optimal and practically efficient to compute, closing the gap between the two that prior low-rank compression methods struggled with.

This is a side project with my friend Yirui Liu, whom I met when I was a master’s student and he was a PhD student. The project started from a phone chat, where he brought up the problem, and it gradually grew into this paper.

Along the way, this project also led to an earlier, preliminary exploration: “KV-CoRE: Benchmarking Data-Dependent Low-Rank Compressibility of KV-Caches in LLMs”. KV-CoRE benchmarks how compressible KV-caches are under low-rank approximation, and shows that this compressibility is highly data-dependent.

The work has also been covered by a few domestic media outlets, including TechWalker and CSDN.