No; standard truncated SVD uses the matrix itself.

Are the components unique?

Signs are arbitrary, and repeated singular values permit rotations within their subspaces.

Is a larger rank always better?

It reduces exact reconstruction error, but may increase cost or retain noise that hurts a downstream task.

AI Glossary / Probabilistic machine learning / Intermediate

Truncated singular value decomposition

Definition

Truncated singular value decomposition is a low-rank representation of a matrix obtained by retaining only its largest singular values and associated left and right singular vectors. It summarizes the strongest directions in the matrix while discarding the remaining components. It is used for dimensionality reduction, compression and approximate numerical computation.

What it means

A data matrix may have thousands of columns even when much of its structure can be represented using fewer directions. Truncated SVD replaces the original matrix with a product of smaller factors. The retained directions are linear combinations of the original features, not necessarily a subset of those features.

The meaning of strongest depends on the numerical matrix supplied. Feature scaling, term weighting and centering therefore affect the result. In common sparse-data software, truncated SVD is applied without centering, which preserves sparse input handling. This should be distinguished from principal component analysis on centered data, although the underlying matrix decomposition is closely related.

How it works

Write a matrix X as U Sigma V-transpose, with singular values in descending order. For a selected rank k, retain the first k columns of U and V and the corresponding k singular values. Their product X-k = U-k Sigma-k V-k-transpose is the truncated reconstruction.

For an exact SVD, this reconstruction minimizes Frobenius-norm error among matrices of rank at most k. Its squared error is the sum of the squared omitted singular values. The corresponding spectral-norm error is the next singular value. Ties can make the best representation nonunique.

When rows represent observations, multiplying X by V-k gives their reduced coordinates. A new row must use the same feature ordering, preprocessing and fitted V-k. Large problems may use iterative or randomized algorithms that approximate the leading components without computing the entire decomposition.

Selecting k is an application decision. Reconstruction error is useful, but a downstream prediction or retrieval task may favor a different rank. The mathematical best approximation to the input matrix is not automatically the best representation of what the organization cares about.

Three practical examples

1. A search prototype compresses a weighted document-term matrix into latent semantic coordinates for similarity calculations.

2. An analytics team reduces a dense numerical matrix before an exploratory clustering exercise, checking whether important small groups disappear.

3. A numerical model uses a truncated decomposition to simplify a matrix operation or suppress unstable contributions from very small singular values.

Think of it this way

Think of describing a piece of music with only its strongest patterns. The summary can capture much of the overall signal while losing a quiet but meaningful detail. Truncated SVD chooses patterns according to matrix magnitude, so an analyst must decide whether the omitted detail matters for the actual use case.

Worked example

Consider the diagonal matrix with diagonal entries 6, 3 and 1. Because all entries are nonnegative and already ordered, its singular values are 6, 3 and 1, with identity left and right singular-vector matrices.

The rank-one reconstruction retains only the six, giving diagonal entries 6, 0 and 0. Its squared Frobenius error is 3 squared plus 1 squared = 10, and its Frobenius error is the square root of 10, approximately 3.1623.

The rank-two reconstruction has diagonal entries 6, 3 and 0. Its squared Frobenius error is one, and its spectral-norm error is also one. The original squared Frobenius norm is 36 + 9 + 1 = 46. Rank two retains 45/46, approximately 97.83 percent, of this squared matrix magnitude.

That percentage is an energy calculation for this specified uncentered matrix. It should not automatically be labeled a centered statistical variance-explained percentage.

If the rows are observations, projecting onto the first two right singular vectors gives rows (6, 0), (0, 3) and (0, 0). The third observation loses its distinguishing feature entirely. A small reconstruction error can therefore coexist with information loss that matters for a particular task.

Illustrative case study

Fictional case: Willow Archive builds a document-search prototype from a sparse weighted term matrix. Analysts compare several truncated-SVD ranks using held-out relevance judgments and memory measurements.

A smaller representation performs well on common topics but misses a specialized collection whose distinguishing terms contribute little overall matrix magnitude. The team retains a larger rank and checks representative rare queries. It reports both retrieval quality and storage costs, recognizing that a high retained-energy percentage alone does not establish that the search experience is adequate.

Common misconceptions and mistakes

Truncation does not mean keeping the first k original columns. It keeps k singular directions, usually mixing many columns.

SVD does not require a square matrix and is not the same as an eigenvalue decomposition of an arbitrary matrix. Singular values are nonnegative, while singular-vector signs can be flipped consistently without changing the reconstruction.

Uncentered truncated SVD is not automatically equivalent to centered PCA. Sparse input also does not imply sparse reduced coordinates: the output can be dense. Finally, optimal reconstruction in a matrix norm is not a guarantee of denoising, fairness, interpretability or predictive usefulness.

Business use

The business value is a controllable tradeoff between representation size and retained structure. It can support text analysis, similarity search, compressed features and numerical efficiency. Teams should measure end-to-end memory, transformation cost and task performance, including uncommon cases. Fit preprocessing and the decomposition on the appropriate training data, then preserve those fitted transformations for consistent evaluation and deployment.

Check your understanding

What is the rank-two squared error in the example?

Answer: One, because only the singular value one is omitted.

Why can the third row become zero?

Answer: Its entire signal lies in the discarded singular direction.

Related terms

Singular value decomposition
Principal component analysis
Latent semantic analysis
low-rank approximation
randomized SVD
matrix norm and dimensionality reduction

Learn next

Next, compare centered and uncentered decompositions of the same matrix. Then plot singular-value decay and evaluate several ranks against the actual prediction, retrieval or compression objective.

Singular value

Sources

Primary references: Stanford CS168 lecture notes on singular value decomposition and low-rank approximation: https://web.stanford.edu/class/cs168/l/l9.pdf

Official scikit-learn TruncatedSVD documentation, including uncentered sparse-input behavior and sign ambiguity: https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.TruncatedSVD.html

The diagonal-matrix calculation and fictional case are original teaching material.

Independent review consulted the low-rank approximation theorem and official TruncatedSVD notes on uncentered inputs and sign ambiguity.

Previous: Truncated normal distribution Next: Trust