Vector Quantization

Vector Quantization (VQ) is a classical signal processing technique that achieves lossy data compression by mapping a large set of high-dimensional vectors to a finite, smaller set of representative prototype vectors (a codebook).

VQ is a powerful clustering method, developed in the early 1980s by Robert M. Gray, designed to model probability density functions for data compression. The process works by partitioning a vector space into regions (Voronoi regions) and assigning a centroid (codeword) to each; input vectors are then represented by the index of their closest centroid. This transformation drastically reduces the memory footprint, which is critical for handling massive datasets, like 1536-dimensional embeddings from models like OpenAI. Modern applications leverage VQ variants, such as Product Quantization (PQ), to accelerate Approximate Nearest Neighbor (ANN) search in vector databases and enable efficient, scalable solutions for image retrieval and large language model (LLM) embedding stores.

https://en.wikipedia.org/wiki/Vector_quantization