Is matrix factorization always exact?

No. Many machine-learning methods seek an approximation.

Are latent factors automatically interpretable?

No. The same product can have multiple valid factor representations.

Is it the same as singular value decomposition?

SVD is one particular matrix factorization. Learned recommendation factors can use different objectives, weighting, constraints, and missing-data assumptions.

AI Glossary / Mathematics and optimization / Intermediate

Matrix factorization

Definition

Matrix factorization expresses a matrix as a product of simpler matrices, either exactly or approximately. In machine learning, the term often refers to learning a low-rank product that captures useful patterns in a larger data matrix. In recommendation, two factor matrices commonly represent users and items through short numerical vectors.

What it means

A large table can contain repeated structure: several rows may share patterns, and several columns may respond similarly. A low-rank factorization tries to describe those patterns with fewer latent dimensions rather than assigning an unrelated parameter to every cell.
Matrix factorization is a broad mathematical idea. LU, QR, and singular value decomposition are different factorizations with different structures and purposes. A recommendation model that learns two embedding matrices is one application, not a definition covering every factorization method.
Exact factorization reconstructs the matrix precisely. Approximate factorization accepts some reconstruction error to gain simplicity or capture a chosen structure. With missing observations, the task can also involve estimating unobserved entries, which requires assumptions and evaluation beyond reconstructing the observed cells.

How it works

For a matrix A with m rows and n columns, a common low-rank model is A≈UVᵀ, where U is m-by-k and V is n-by-k. The predicted entry at row i and column j is the dot product of their k-dimensional factor vectors. The product has rank at most k.
Choose an objective that specifies which entries matter, their weights, and any regularization. Recommendation data require care because an unobserved interaction is not necessarily a negative preference. The learning method may use gradient-based updates or alternate between fitting one factor while holding the other fixed.
Even when each alternating subproblem is a least-squares problem, the objective is not generally jointly convex in both factors. Different initializations or constraints can lead to different solutions. Factor coordinates may also be nonunique even when their product is unchanged.

Three practical examples

A recommendation system can estimate user-item scores from learned user and item vectors.
A data-analysis method can approximate a numerical matrix with a lower-rank representation to inspect dominant structure.
A numerical solver can use a structured factorization such as QR to solve a linear-algebra problem efficiently and reliably.

Think of it this way

Think of describing many combinations using a small set of shared ingredients and a recipe for each row and column. The analogy captures reusable structure, but latent coordinates need not correspond to named real-world ingredients. Their interpretation depends on the model and may change without changing the reconstructed table.

Worked example

Consider an original factor matrix U with rows (1,0), (0,1), and (1,1). Let V have rows (2,0), (0,3), (1,1), and (2,−1). Then U is three-by-two, V is four-by-two, and UVᵀ is three-by-four.
The first reconstructed row is (2,0,1,2), because the first U row selects each item's first coordinate. The second row is (0,3,1,−1). The third row is their sum: (2,3,2,1). For example, its fourth entry is (1,1)·(2,−1)=1.
The resulting matrix has rank two: its third row depends on the first two, and the leading two-by-two block has nonzero determinant 2×3=6. The factors therefore describe exact shared structure. If the third row's fourth entry were unobserved, this fitted factor pair would predict 1 there, but that number would still require evaluation as an estimate rather than being treated as an observed fact.
Now double the second column of U and halve the second column of V. Every product contribution from that coordinate stays unchanged, so UVᵀ is identical. The factor values are not uniquely determined by the reconstructed matrix alone.
This tiny example does not save storage: the full matrix has 12 entries, while the factors contain 3×2+4×2=14. For a 1,000-by-500 matrix with k=10, the factors contain 15,000 entries instead of 500,000 dense entries. That comparison concerns parameter counts; actual memory also depends on sparse storage, metadata, and implementation.

Illustrative case study

In a fictional media-service prototype, a team trains factor vectors from recorded user-item interactions. A review finds that the objective treats every missing interaction as a definite dislike. Many users simply never saw those items, so the target interpretation is questionable.
The team revises its weighting assumptions and evaluates recommendations on a suitable held-out design. It also examines users and items with little history, where factor estimates have limited support. The hypothetical scenario illustrates modeling choices and coverage limits, not a claim of measured engagement gains.

Common misconceptions and mistakes

Do not assume factorization always compresses a matrix or that every factorization is low rank. Do not interpret latent coordinates as stable named preferences without evidence. Rotations, scaling, or other compatible changes can leave the product unchanged.
Avoid treating low reconstruction error on observed entries as proof of good recommendations. Missingness, popularity, and exposure patterns can affect what the data mean. Also, alternating optimization that reduces loss does not by itself prove convergence to a global optimum or establish that all factor values converge to a unique solution.

Business use

Matrix factorization can offer compact scoring models when interactions contain useful low-dimensional structure. Document the matrix's row and column meanings, the latent dimension, treatment of missing entries, regularization, and evaluation criteria. Keep factor versions aligned: user vectors from one coordinate system cannot safely be combined with item vectors from another.
Review cold-start behavior and how new observations update the factors. If the intended output is a ranked recommendation list, evaluate that list rather than relying only on squared reconstruction error. Constraints such as nonnegativity or additional side information can change the interpretation and capabilities of the model.

Check your understanding

U has shape 200-by-8 and V has shape 50-by-8. What is the shape and maximum rank of UVᵀ?

Answer: It is 200-by-50 and has rank at most 8.

Related terms

Singular value decomposition
User matrix
Item matrix
Embedding
Collaborative filtering
Regularization
Low-rank approximation
alternating least squares
matrix completion

Learn next

Next, multiply a small pair of factor matrices and verify one reconstructed entry as a dot product. Then change both factors in a way that preserves their product to see why latent-coordinate meaning is not automatically unique.

Dot product

Sources

Primary teaching reference: Google, Recommendation Systems, Matrix Factorization, original course material on user/item factors and fitting objectives: https://developers.google.com/machine-learning/recommendation/collaborative/matrix . The exact three-by-four construction and storage comparison are original calculations.

Previous: Matrix difference equation Next: Matrix norm