How many singular values does a matrix have?

A common compact listing contains min(m,n) values for an m-by-n matrix, including zeros; expanded formulations can represent additional zero directions.

Can singular values be negative?

No. Signs and phases are carried by singular vectors, while singular values are nonnegative.

What does the sum of squared singular values represent?

It equals the sum of squared magnitudes of all matrix entries, the squared Frobenius norm.

AI Glossary / Mathematics and optimization / Intermediate

Singular value

Definition

A singular value is a nonnegative number describing how much a matrix stretches a particular orthogonal input direction. Singular values arise in singular value decomposition and help characterize a matrix's rank, scale, sensitivity and suitability for lower-rank approximation.

What it means

A linear transformation may stretch some directions strongly, barely change others and collapse some to zero. Singular values describe those effects after suitable orthonormal coordinate changes. Their associated input and output directions are singular vectors.

Unlike eigenvalues, singular values are always real and nonnegative, even when the matrix has complex entries. They are also defined for rectangular matrices, whose input and output spaces may have different dimensions.

A large singular value represents strong amplification in its associated direction relative to the chosen units. It does not automatically represent an important business feature. Singular vectors usually combine original coordinates, and scaling those coordinates can change the entire singular-value picture.

How it works

For a real matrix A, the squared singular values are the relevant eigenvalues of AᵀA; for complex matrices, use A*A. These matrices are positive semidefinite, giving nonnegative eigenvalues. Taking square roots produces the singular values.

If Av=σu for unit right and left singular vectors v and u, the unit input v becomes an output of length σ. Singular values are commonly listed in descending order. The largest equals the matrix's Euclidean operator norm: the greatest output length produced by any unit input.

In exact arithmetic, the number of positive singular values equals rank. Tiny values can indicate directions whose recovery requires large amplification, making inverse or least-squares calculations sensitive to perturbations. For an invertible square matrix, the ratio of the largest to smallest singular value is its two-norm condition number.

These identities explain the mathematics but do not prescribe forming AᵀA in software. Doing so can square a condition number; direct numerical SVD methods are often preferable. Numerical rank also uses a tolerance rather than testing computed values for exact zero.

Three practical examples

A calibration team examines small singular values to identify combinations of unknown quantities that its measurements distinguish poorly.
A data analyst inspects a singular-value spectrum before choosing how many components to retain in a matrix approximation.
A numerical software reviewer checks whether a fitted system is sensitive despite being mathematically full rank.

Think of it this way

Think of stretching a circular rubber sheet into an ellipse. The longest and shortest principal stretches describe how the transformation changes lengths in special perpendicular directions.

This picture works directly in two dimensions. Higher-dimensional and rectangular transformations need a more general interpretation, and zero singular values can collapse directions completely. The analogy concerns geometry, not physical elasticity.

Worked example

Stretching and collapsing directions

Let A have rows (1,1) and (0,0). Then AᵀA has rows (1,1) and (1,1). Its eigenvalues are two and zero, so A's singular values are √2 and zero.

The unit vector v₁=(1,1)/√2 is mapped to (√2,0), whose length is √2. The perpendicular unit vector v₂=(1,−1)/√2 is mapped to zero. Thus one input direction is retained and the other is lost, giving rank one.

A's eigenvalues are one and zero because A is triangular. They are not the same as its singular values, illustrating why the terms cannot be interchanged.

For a separate sensitivity example, take B=diag(1,0.001). Its singular values are one and 0.001, so it is full rank with condition number 1,000. A perturbation of 0.001 in the second output coordinate corresponds to a change of one in the recovered second input coordinate. Full rank guarantees algebraic invertibility here, not resistance to measurement error.

Illustrative case study

A fictional manufacturing team estimates two hidden process adjustments from sensor readings. The fitted measurement matrix is invertible, so an early report describes the estimates as reliable.

A numerical review finds that one singular value is much smaller than the other. The associated direction combines both adjustments, and small measurement changes produce large changes in the recovered combination.

The team considers improving measurement design or applying a justified regularization approach. It also reports uncertainty instead of interpreting invertibility as complete evidence. The singular values expose a mathematical weakness; selecting a remedy still depends on the physical process and decision needs.

Common misconceptions and mistakes

Singular values are eigenvalues of the original matrix. They are related to eigenvalues of A*A, and generally differ from A's eigenvalues.
A zero singular value means every output is zero. It means at least one represented direction is collapsed.
A small singular value always represents noise. It may encode a weak but meaningful signal.
Full rank means all directions are measured equally well. Positive singular values can differ greatly.
The spectrum is unaffected by feature units. Rescaling coordinates can change it.

Business use

Singular values help assess whether numerical answers are robust enough for interpretation. They are useful in regression, reconstruction, compression and calibration, where hidden dependencies may not be obvious from the original table.

Ask how the matrix was scaled, which numerical tolerance was used and what information lies in weak directions. Discarding a small component may improve stability but remove useful detail. Evaluate that trade-off against the task rather than selecting a threshold solely because it yields a visually tidy spectrum.

Check your understanding

A two-by-two matrix has singular values one and 0.001. Is it rank deficient in exact arithmetic?

Answer: No. Both values are positive, so it has rank two. It may nevertheless be poorly conditioned and sensitive to perturbations.

Related terms

Learn next

Read condition number and pseudoinverse. For the worked matrix A, verify that any multiple of (1,−1) is invisible at the output, then explain why no inverse can recover that lost component.

Sources

NumPy, numpy.linalg.svd: https://numpy.org/doc/stable/reference/generated/numpy.linalg.svd.html. Official singular-value decomposition reference.

Aalto Dictionary of Machine Learning, singular value decomposition: https://aaltodictionaryofml.github.io/ADictML_Main.pdf. Nonnegative singular values and orthonormal factors. Stretching and perturbation calculations here are original examples.

Previous: Singular they Next: Singular value decomposition