Can a full-rank matrix be rectangular?

Yes. Its rank only needs to equal the smaller dimension.

Does full rank guarantee stable calculations?

No. A matrix can be very ill-conditioned while remaining full-rank in exact arithmetic.

Does full column rank guarantee a perfect regression fit?

No. It guarantees unique unconstrained least-squares coefficients, but a target vector can lie outside the column space.

AI Glossary / Mathematics and optimization / Intermediate

Full-rank

Definition

A matrix is full-rank when its rank equals the largest value possible for its dimensions. For an m-by-n matrix, that maximum is min(m, n). Rank measures the number of linearly independent columns, equivalently independent rows. A tall full-rank matrix has independent columns; a wide full-rank matrix has independent rows. Only a square full-rank matrix has an ordinary two-sided inverse.

What it means

Full-rank means that the matrix contains as many independent directions as its shape allows. It does not mean that every column is independent in a wide matrix, because there are more columns than the surrounding row space can support. In machine learning, rank helps explain whether linear coefficients can be identified from a design matrix. It also matters in solving equations and understanding redundant features. However, exact independence is different from numerical stability: columns can be technically independent while nearly duplicating each other.

How it works

Rank can be determined by row reduction in exact arithmetic or estimated from singular values in numerical work. A zero singular value indicates a missing independent direction in an exact mathematical matrix. Software typically uses a tolerance to decide which small singular values count as zero, because measurements and floating-point calculations are imperfect. For a full-column-rank design matrix, an unconstrained ordinary least-squares problem has a unique coefficient solution. For a full-row-rank wide matrix, every right-hand side is attainable, but the solution vector generally has free directions and is not unique.

Three practical examples

A regression analyst can detect that two feature columns are exact duplicates and therefore do not provide independent information for estimating unrestricted coefficients.
An equation-solving routine can distinguish a square invertible matrix from a rectangular matrix that needs a different solution approach.
A numerical review can inspect singular values to discover that a formally full-rank matrix is so close to rank deficiency that coefficient estimates are sensitive to small perturbations.

Think of it this way

Think of a collection of directions on a map. Two arrows that point along the same line do not provide two independent ways to move across a plane. Full-rank means obtaining as many independent directions as the matrix dimensions permit. The analogy does not measure how well separated those directions are; nearly parallel arrows can still be independent while making numerical calculations fragile.

Worked example

Consider the three-by-two matrix X with rows (1, 0), (0, 1) and (1, 1). Its two columns are independent, so rank(X) = 2 = min(3, 2), and X is full-column-rank. For target vector y = (1, 2, 2), XᵀX has rows (2, 1) and (1, 2), while Xᵀy = (3, 4). Solving the normal equations gives coefficients (2/3, 5/3). Predictions are (2/3, 5/3, 7/3), and prediction-minus-target residuals are (−1/3, −1/3, 1/3). The residual is nonzero: full column rank guarantees uniqueness of the least-squares coefficients here, not an exact fit to every possible y. Its squared norm is 1/3. Now consider the two-by-three matrix A with rows (1, 0, 1) and (0, 1, 1). It is full-row-rank. The equation Az = (1, 2) has the family z = (1 − t, 2 − t, t) for any real t, so full rank does not guarantee a unique solution for a wide matrix. Finally, the square diagonal matrix with diagonal values 1 and 0.0000000001 is full-rank mathematically but has condition number 10 billion in the Euclidean norm.

Illustrative case study

In a fictional forecasting review, Rowan Analytics discovers that a dataset includes both total units and a second column copied from the same total under a different name. The team removes the accidental duplication and checks other feature relationships. It also examines conditioning rather than declaring the design safe as soon as the reported rank is maximal. The case illustrates diagnosing representation problems; it does not claim that removing a column necessarily improves predictive accuracy.

Common misconceptions and mistakes

Full-rank does not mean square, symmetric or orthogonal. A rectangular full-rank matrix does not have an ordinary two-sided inverse. Rank is not the same as the count of nonzero entries or nonzero columns, because columns can depend on one another. A nonzero determinant establishes full rank only for a square matrix. Software-reported rank depends on numerical tolerance and scaling. Full rank also does not establish that a model generalizes, that its features are meaningful or that its coefficients have a causal interpretation.

Business use

When linear models behave unexpectedly, inspect both rank and conditioning of the actual encoded design matrix, including intercepts and transformed features. Document numerical tolerances if rank decisions affect production behavior. Use stable numerical solvers rather than forming an inverse simply because theory permits it; the normal equations in the example are for transparent hand calculation. Consider whether near-dependencies arise from measurement precision or feature construction. A regularized problem can have a unique solution even when the original design is rank-deficient, but regularization changes the estimation problem and should be explained.

Check your understanding

Why does the worked wide matrix allow infinitely many solutions despite being full-rank?

Answer: Its rank is two but it has three unknown coordinates. One free direction remains, represented by the parameter t.

Related terms

Learn next

verify that Xᵀ times the worked residual equals zero, the least-squares orthogonality condition. Then duplicate a column of X and determine the new rank. Compare exact rank with numerical rank after adding a very small perturbation to the duplicate.

Sources

Sources: Jung and colleagues, The Aalto Dictionary of Machine Learning, full-rank, https://aaltodictionaryofml.github.io/ADictML_Main.pdf ; NumPy, numpy.linalg.matrix_rank, https://numpy.org/doc/stable/reference/generated/numpy.linalg.matrix_rank.html . The rectangular examples, least-squares calculation, conditioning example and fictional feature review are original.

Previous: Full joint probability distribution Next: Full-reference metric