Must a matrix be square?

No. Rectangular matrices are common, especially for datasets and transformations between different dimensions.

Is matrix multiplication entrywise multiplication?

No. Ordinary multiplication combines rows and columns through dot products; entrywise multiplication acts on matching positions.

Does every matrix have an inverse?

No. An ordinary two-sided inverse requires a square, nonsingular matrix.

AI Glossary / Mathematics and optimization / Beginner

Matrix

Definition

A matrix is a rectangular array of entries arranged in rows and columns. A matrix with m rows and n columns has shape m-by-n. In AI and applied mathematics, numerical matrices represent datasets, transformations, model weights, relationships, and systems of equations, with their meaning determined by how the entries are defined.

What it means

A matrix is both an organized collection of numbers and an object with specific algebraic operations. Its shape matters: a three-by-two matrix has six entries but represents a different arrangement from a two-by-three matrix.
The entry Aᵢⱼ lies in row i and column j. In a dataset convention, rows often represent examples and columns features, but that convention is not universal. In a transformation matrix, columns describe the images of input basis vectors under a chosen coordinate convention.
The same numerical array can support different interpretations. A row might represent a customer in one application and a destination state in another. Dimensions and labels therefore need to accompany the numbers; storage shape alone does not establish business meaning.

How it works

Matrix addition combines corresponding entries and requires matching shapes. Multiplication by a scalar scales every entry. Transposition exchanges rows and columns: the transpose of an m-by-n matrix is n-by-m.
Ordinary matrix multiplication is different from elementwise multiplication. If A is m-by-n and B is n-by-p, their product AB is m-by-p. Each output entry is a dot product of a row of A with a column of B.
Matrix multiplication can express several weighted sums at once or compose compatible linear transformations. Its order generally matters. A matrix need not be square, invertible, symmetric, or dense. Those are additional properties, and each has consequences for what operations and interpretations are appropriate.

Three practical examples

A training dataset can store one numerical feature vector per row, with labels kept in a correspondingly ordered target array.
A neural-network layer can use a weight matrix to transform a batch of input representations into output scores.
A graph-analysis task can store connection weights in an adjacency matrix whose row and column labels identify nodes.

Think of it this way

Think of a carefully labeled spreadsheet rectangle with rules for combining its numbers. The analogy helps with rows and columns, but matrix algebra adds structure beyond a generic table. A table containing mixed text, dates, and unrelated fields is not automatically suitable for meaningful matrix multiplication.

Worked example

Consider an original feature matrix X with three rows: (1,2), (3,0), and (−1,4). It has shape three-by-two. Let the coefficient vector be w=(2,−1)ᵀ, with shape two-by-one. The product Xw has shape three-by-one.
Calculate each row's dot product: the first output is 1×2+2×(−1)=0; the second is 3×2+0×(−1)=6; the third is (−1)×2+4×(−1)=−6. Thus Xw=(0,6,−6)ᵀ. Each row produces one score using the same coefficients.
The transpose Xᵀ has rows (1,3,−1) and (2,0,4), so its shape is two-by-three. Transposing changes the arrangement; it does not compute an inverse or undo the scoring operation.
To see why multiplication order matters, take A with rows (1,1) and (0,1), and B with rows (2,0) and (0,1). Their product AB has rows (2,1) and (0,1). Reversing the order gives BA with rows (2,2) and (0,1). The results differ even though both products are defined.
Finally, elementwise multiplication of A and B gives rows (2,0) and (0,1), which differs from both products. An implementation must therefore specify whether it intends algebraic matrix multiplication or entry-by-entry multiplication. Similar-looking operators in different software environments can represent different choices.

Illustrative case study

In a fictional forecasting project, one service exports a feature matrix with columns ordered as quantity, age, and price. Another service expects price, quantity, and age. Their arrays have compatible shapes, so multiplication runs without a dimension error but produces incorrect scores.
The team adds a documented feature-order contract and checks a few complete examples against hand calculations. It also records how missing values and scaling are handled. This hypothetical scenario illustrates why semantic alignment matters as much as valid dimensions; it does not claim a measured model improvement.

Common misconceptions and mistakes

Do not assume that two arrays with the same number of entries have interchangeable shapes. Do not reverse multiplication order without checking dimensions and meaning. Transpose and inverse are different operations, and many matrices have no ordinary inverse.
Do not treat category identifiers as meaningful numerical measurements merely because they fit into a matrix. Also, a zero entry can mean a genuine zero value, an absent graph edge, or a storage convention; its interpretation depends on the model. Missing data should not silently be replaced with zeros unless that choice is justified and documented.

Business use

Matrices give AI systems a compact way to process many numerical relationships. For review, record row and column meanings, feature order, units, scaling, and the intended operation. A shape check catches some mistakes but cannot detect correctly shaped data attached to the wrong meaning.
Storage choices also matter. A sparse matrix can represent many implicit zeros efficiently, while a dense representation stores every entry. The useful choice depends on the actual data and operations. Before estimating memory or computation, identify both the dimensions and whether the structure can be exploited.

Check your understanding

A has shape four-by-three and B has shape three-by-two. What is the shape of AB?

Answer: Four-by-two. The inner dimensions match, and the outer dimensions determine the result. BA is not defined with these shapes.

Related terms

Vector
Scalar
Transpose
Sparse matrix
Rank
Inverse matrix
Linear map
matrix multiplication
design matrix
adjacency matrix

Learn next

Next, calculate a small matrix-vector product by hand and label every row and column. Then study rank and matrix factorization to understand redundancy and ways to express a matrix through simpler components.

Sources

Primary terminology reference: Alexander Jung and contributors, The Aalto Dictionary of Machine Learning, matrix entry and examples of matrix representations: https://aaltodictionaryofml.github.io/ADictML_Main.pdf . The feature-product and noncommuting two-by-two examples are original calculations.

Previous: Materialized view Next: Matrix calculus