Is this the same as singular value decomposition?

No. SVD applies to rectangular matrices as well, while the decomposition described here concerns normal square matrices.

Can eigenvalues be negative?

Yes for a general real symmetric matrix, but not for an exact covariance matrix.

Does a changed eigenvector sign change the decomposition?

No. Consistently changing a vector's sign leaves its spectral outer-product term unchanged.

AI Glossary / Time series and signal processing / Advanced

Spectral decomposition

Definition

Spectral decomposition expresses a normal square matrix through an orthonormal basis of eigenvectors and their eigenvalues. In the real symmetric case, it writes A = QΛQᵀ, where Q contains perpendicular unit-length eigenvectors and Λ contains the corresponding eigenvalues. This entry follows the supplied glossary's matrix meaning, which should be distinguished from analyzing a signal's frequency spectrum.

What it means

A matrix can mix several coordinates when it acts on a vector. Spectral decomposition finds a coordinate system in which its action becomes especially simple: each eigenvector direction is scaled by its eigenvalue. For a covariance matrix, these directions describe uncorrelated combinations of features, and the eigenvalues describe their variances. The directions are mathematical summaries, not automatically meaningful causes or business categories.

How it works

For a real symmetric matrix, find unit vectors q satisfying Aq = λq and arrange an orthonormal collection as the columns of Q. Then QᵀQ is the identity matrix, and QᵀAQ is diagonal. Equivalently, A is the sum of terms λqqᵀ, one for each eigenvector.
For a complex normal matrix, the corresponding expression uses the conjugate transpose Q* rather than an ordinary transpose. Normal means AA* = A*A; it does not mean a Gaussian distribution. Hermitian matrices have real eigenvalues, while a general normal matrix can have complex ones. A covariance matrix is positive semidefinite, so its eigenvalues are nonnegative in exact arithmetic.

Three practical examples

A principal-component analysis decomposes a covariance matrix to identify directions with large variance. Choosing how many directions to retain remains a separate modeling decision.
An optimization analyst decomposes a symmetric curvature matrix near a candidate solution. Its eigenvalues help describe how curvature differs across directions, including negative directions that a covariance matrix would not have.
A signal-processing system uses covariance eigenvectors to summarize relationships between sensor channels. This operates on a matrix built from the signals; it does not make the resulting eigenvalues temporal frequencies. These examples illustrate mathematical uses rather than guaranteed improvements in an AI system.

Think of it this way

Imagine rotating a coordinate grid until a transformation acts independently along each grid axis. In that grid, the transformation becomes a list of scale factors. Rotating back reconstructs its original action. The useful part is not merely changing how the grid looks, but exposing directions that the transformation does not mix together.

Worked example

Take the symmetric matrix A with rows [3, 1] and [1, 3]. Define q1 = [1, 1]/√2 and q2 = [1, -1]/√2. Both have unit length, and their dot product is zero. Multiplication gives Aq1 = 4q1 and Aq2 = 2q2, so the eigenvalues are four and two.
The first spectral term is 4q1q1ᵀ, with rows [2, 2] and [2, 2]. The second is 2q2q2ᵀ, with rows [1, -1] and [-1, 1]. Adding them gives [3, 1] and [1, 3], recovering A exactly. The trace, six, equals the eigenvalue sum; the determinant, eight, equals their product.
Now apply A to x = [2, 0]. In the eigenvector basis, both coordinates q1ᵀx and q2ᵀx equal √2. Scaling those coordinates by four and two and converting back gives [6, 2], matching direct matrix multiplication. The decomposition changes the description of the calculation, not its result.
If A represents the covariance of two centered, comparably scaled measurements, the first direction accounts for 4/(4+2) = two-thirds of their total variance. Retaining only that direction produces a rank-one covariance approximation with all four entries equal to two. The discarded covariance term has trace two. For the individual vector x, projection onto q1 gives [1, 1], leaving residual [1, -1] and squared reconstruction error two. That one vector's error should not be confused with a general guarantee about every observation.

Illustrative case study

Fictional Harbor Metrics explores the constructed covariance matrix above for two measured service indicators. Its analyst describes q1 as their common movement and q2 as their contrast. A manager proposes discarding q2 because it carries less variance.
The analyst explains that one-third of total variance remains in that direction and that a contrast could still matter to the business task. The team plans to assess the reduced representation against its actual prediction or monitoring objective. It also records the feature scaling used to form the covariance matrix. This scenario demonstrates interpretation choices; it does not establish that dimension reduction improves accuracy or that either direction has a causal meaning.

Common misconceptions and mistakes

Not every square matrix has an orthonormal eigenvector basis. Using a symmetric eigensolver on unsuitable input does not establish symmetry or normality. Eigenvectors are also not uniquely signed: replacing q by -q leaves qqᵀ unchanged. Repeated eigenvalues allow different orthonormal bases within the same eigenspace. Finally, uncorrelated components are not generally independent, and retaining high variance does not necessarily retain the information needed to predict a target.

Business use

Use spectral decomposition when a matrix's directions and scale differences help explain a model or reduce computation. Document how the matrix was constructed, whether its required properties hold, and how approximation error is measured. In covariance applications, review units and scaling before ranking eigenvalues. Validate any downstream simplification against the intended task instead of treating a variance-retention percentage as a universal quality score.

Check your understanding

Why does retaining only q1 fail to reconstruct the example matrix exactly?

Answer: The second eigenvalue is two, not zero. Its term contributes the remaining covariance structure. Dropping that term deliberately changes the matrix and reduces its rank.

Related terms

Learn next

Replace A by the matrix with rows [5, 1] and [1, 5]. The same directions have eigenvalues six and four. Their sum is ten and product is 24, matching trace and determinant. The leading direction now retains 60% of total variance, showing that unchanged directions can accompany a different variance balance.

Sources

Aalto Dictionary of Machine Learning, spectral decomposition and related matrix entries: https://aaltodictionaryofml.github.io/ADictML_Main.pdf NumPy, numpy.linalg.eigh documentation, symmetric and Hermitian eigenproblems: https://numpy.org/doc/stable/reference/generated/numpy.linalg.eigh.html All matrices, vectors and the Harbor Metrics scenario are original teaching constructions. The supplied category is retained, while the matrix-specific meaning is stated explicitly.

Previous: Spectral clustering Next: Spectral mixture kernel