TurboQuant: Google’s New Trick for Squeezing AI Models Without Breaking Them

Google Research just dropped three new compression algorithms — TurboQuant, QJL, and PolarQuant — and they’re worth paying attention to if you’ve ever wrestled with memory bottlenecks in large language models or vector search.

The big idea here is vector quantization. Vectors are how AI models represent everything from image features to word meanings. High-dimensional vectors are powerful but memory-hungry. They clog up the key-value cache, which is essentially the model’s scratchpad for storing frequently accessed information. Traditional quantization tries to shrink these vectors, but it usually introduces its own memory overhead — you end up storing quantization constants in full precision, which can add 1-2 bits per number, partly defeating the purpose.

TurboQuant (presented at ICLR 2026) claims to solve that. It’s built on two other algorithms: QJL and PolarQuant (both at AISTATS 2026). Together, they aim to compress vectors aggressively without sacrificing accuracy.

How TurboQuant works

TurboQuant is a two-stage process. First, it randomly rotates the data vectors — a clever geometric trick that makes the data easier to quantize. Then it applies PolarQuant, which does the heavy lifting of compression using most of the available bits to capture the vector’s essential structure.

But here’s the neat part: TurboQuant doesn’t stop there. It takes the tiny residual error from the first stage and applies QJL — a 1-bit algorithm — to clean it up. This eliminates bias in the attention score calculation. The result is high compression with zero accuracy loss, at least according to their tests.

QJL: The 1-bit trick with no overhead

QJL stands for Quantized Johnson-Lindenstrauss. It uses the Johnson-Lindenstrauss Transform to project high-dimensional data down to a single sign bit per number — just +1 or -1. No memory overhead for storing quantization constants. It works because the algorithm uses a special estimator that balances a high-precision query against the ultra-low-precision data. The math checks out, and it seems to preserve the essential distance relationships between vectors.

PolarQuant: A different angle

PolarQuant takes a completely different approach to the memory overhead problem. Instead of representing vectors in standard Cartesian coordinates (x, y, z), it converts them into polar or spherical coordinates. This representation naturally separates magnitude from direction, which makes quantization more efficient. The result is less memory overhead and better compression ratios.

Why this matters

If you’re running LLMs at scale, the key-value cache is a constant pain point. Every token you generate requires storing key-value pairs for attention, and those pairs eat memory fast. Google’s benchmarks suggest these algorithms can significantly reduce that footprint without hurting model quality. For vector search engines — think semantic search, recommendation systems, or RAG pipelines — faster similarity lookups with less memory is a direct win.

That said, I’m a bit skeptical about “zero accuracy loss” claims in general. Every quantization method I’ve seen eventually hits a trade-off, especially at extreme compression ratios. Google’s testing might be thorough, but real-world deployments often expose edge cases the lab doesn’t catch. Still, the theoretical grounding here is solid — these aren’t heuristic hacks but mathematically principled approaches.

One thing I appreciate: they’re not pretending this is magic. The papers clearly explain the trade-offs and the math behind each step. That’s refreshing in an era where too many AI announcements are just marketing fluff.

If you’re working on model deployment or vector search infrastructure, these are worth a close look. The code and papers should be available — go poke at them yourself.