Does vector quantization require a neural network?

No. It predates neural codecs and can operate directly on vectors.

Is every codeword a real training example?

No. A learned centroid can be an average that does not match any individual example.

Can the original vector always be recovered?

Generally no. The index identifies the representative, not the exact original input.

AI Glossary / Generative AI and language / Advanced

Vector quantization

Definition

Vector quantization represents an input vector by one of a finite set of representative vectors called a codebook. The encoder selects a codeword, commonly the nearest under a chosen distance measure, and records its index. A decoder uses that index to recover the representative vector. Because multiple inputs can share one representative, ordinary vector quantization generally loses information.

What it means

A vector is a collection of values considered together. Instead of storing every component of each input at its original precision, a system stores a reference to a representative pattern.
This differs from independently rounding each component. Jointly representing components can exploit patterns in how they vary together. However, whether it improves a particular system depends on the codebook, the data and the distortion measure.
The index is meaningful only with the corresponding codebook. An index of three does not inherently describe a particular sound, color or semantic concept; it identifies an entry in an agreed representation.

How it works

For input x and codebook vectors c1 through cK, a nearest-neighbor quantizer chooses an index minimizing the specified distance between x and ci. A consistent rule resolves ties. Decoding retrieves the selected vector, which can then feed another stage.
One way to learn a codebook alternates assigning training vectors to their nearest representatives and updating representatives to the means of their assigned vectors. This is the familiar k-means-style procedure for squared Euclidean distortion. Other objectives and learning arrangements are possible.
For K codewords, an individual fixed-length index needs ceiling(log2 K) bits. Storage accounting must also consider the codebook and other metadata. Entropy coding can change average index costs when codeword frequencies are uneven; it does not restore information discarded by quantization.

Three practical examples

An image exercise represents small color vectors using a shared palette. The evaluation compares reconstructed colors with the originals and checks whether important visual distinctions survive.
A speech system converts learned audio representations into discrete code indices. A later decoder reconstructs audio, while a generative model can operate on the discrete sequence.
A simulation compresses paired sensor readings with a joint codebook. The team tests whether relationships between the two readings justify this approach compared with separate scalar quantizers.

Think of it this way

Imagine describing many paint shades by choosing the closest swatch in a sample book. Sending the swatch number is compact, provided the recipient has the same book.
The reconstructed shade is the selected swatch, not necessarily the original paint. Enlarging the book can offer finer choices, but the book itself becomes bigger and selection may require more work. This captures the basic tradeoff between representation cost and approximation quality.

Worked example

Consider an original two-dimensional codebook with four entries: c0 = [0, 0], c1 = [0, 4], c2 = [4, 0] and c3 = [4, 4]. Use squared Euclidean distance.
For x = [1, 3], the distances are 10 to c0, 2 to c1, 18 to c2 and 10 to c3. The encoder therefore outputs index 1, and decoding returns [0, 4]. The squared error summed across coordinates is 2; the mean squared error per coordinate is 1.
A nearby input [0.5, 3.5] also selects c1, with summed squared error 0.5. Receiving index 1 alone cannot distinguish these two original inputs. This is the information loss, even if the index itself is transmitted perfectly.
Four codewords need two bits per fixed-length index. If each original vector was stored as two 32-bit values, its raw payload was 64 bits. The index payload is therefore 32 times smaller, before overhead. Storing this four-entry codebook at the same precision costs 4 × 2 × 32 = 256 bits. For 100 vectors, indices plus that codebook cost 200 + 256 = 456 bits, compared with 6,400 raw bits. This simplified accounting excludes headers and any further compression.

Illustrative case study

Fictional Alder Audio tests a small codebook for classroom sound clips. Average reconstruction error looks acceptable, but listeners report that some brief consonants become difficult to distinguish.
The developers inspect those clips separately. They find that an average over many ordinary frames hides errors in short, important events. They compare codebook choices and evaluate reconstructed speech using criteria relevant to intelligibility, rather than relying only on a single numerical average.
A deployment test also catches a version mismatch: the encoder and decoder interpret the same indices using different codebooks. The team ties each encoded stream to its correct codebook version. Compact indices work only when the representation and its interpretation remain aligned.

Common misconceptions and mistakes

Quantization is not the same as lossless compression. A lossless coder can compress the resulting indices, but it cannot infer which of several inputs originally mapped to one codeword.
A larger codebook does not automatically deliver a better complete system. Training quality, search cost, storage overhead and the data distribution still matter.
Codeword numbers are identifiers, not numerical measures of similarity. Adjacent indices need not refer to nearby vectors.
Changing the scale of one component changes Euclidean distances and can change assignments. Units and preprocessing therefore form part of the quantizer's specification.

Business use

Define what must be preserved before optimizing storage. A small average distortion may be unacceptable if it removes details needed by users or downstream systems.
Measure total storage or transmission cost, including shared codebooks, metadata and any required decoder. Compare that cost at the intended data volume, since overhead can dominate small collections.
Plan how codebook updates will be handled. Retaining old encoded data requires a reliable way to recover the representation used to create it. Also test data unlike the training collection, where the chosen representatives may fit poorly.

Check your understanding

Answer: Which codeword represents [1, 3] in the example? c1 = [0, 4].

Why is its per-coordinate mean squared error one?

Answer: The summed squared error is two across two coordinates.

Why include codebook storage?

Answer: The indices require that codebook to be interpreted.

Related terms

Codebook
Quantization error
k-means
Lossy compression
codeword
scalar quantization
audio token
rate–distortion tradeoff

Learn next

Plot the four codewords and several input vectors. Draw their nearest-codeword regions, then vary one coordinate's scale and observe changes in assignment. Recalculate total storage for a much smaller collection to expose the codebook overhead.

Codebook

Sources

Stanford EE274, Lossy Compression Basics and Quantization, course notes. https://stanforddatacompressionclass.github.io/notes/lossy/quant.html Jurafsky and Martin, Speech and Language Processing, August 2025 draft, Text-to-Speech chapter: vector quantization in audio tokenization. https://web.stanford.edu/~jurafsky/slp3/old_aug25/16.pdf The codebook calculation and fictional audio case are original teaching illustrations.

Previous: Vector optimization Next: Vector-quantized generative adversarial network