AI Glossary / Mathematics and optimization / Beginner
Rank (Tensor)
Definition
Tensor rank, in TensorFlow and similar array-programming terminology, is the number of axes a tensor has. A scalar has rank zero, a vector has rank one, and a matrix has rank two. This usage describes data organization rather than linear independence.
What it means
A tensor can organize values along several axes, such as examples, time steps and features. Its rank tells you how many axis positions are needed to address an individual element. A tensor with shape (12,8,5) has three axes and therefore rank three.
Rank alone does not tell you how long those axes are, what they mean or how much memory the tensor occupies. Shapes (12,8,5) and (2,2,2) both have rank three, despite very different element counts.
The terminology is overloaded. In linear algebra, matrix rank counts independent directions. In tensor-decomposition mathematics, tensor rank can also refer to a minimum number of simple components. This entry follows the source list's programming sense: number of axes, also called tensor order in some mathematical discussions.
How it works
Inspect the tensor's shape and count its axis lengths. Shape (7,) has one axis, shape (7,1) has two, and shape () has none. The trailing comma in the first notation helps distinguish a one-axis shape from an ordinary number in Python-style notation.
Adding an axis of length one increases rank without adding elements. Removing such an axis reduces rank. Reshaping can combine or split axes when the total number of elements remains compatible, while transposing rearranges existing axes and preserves rank.
Operations may also reduce axes. Averaging a matrix over its feature axis can return one value per example, yielding a vector. If the operation keeps the reduced dimension as an axis of length one, the result remains rank two.
A program may know the rank before it knows every axis length. For example, a batch dimension can vary while the tensor still has a known two-axis structure. An unknown rank in a symbolic specification means the structure has not been established there, not that the tensor has rank zero.
Three practical examples
- An image-processing team checks that a batch includes separate axes for examples, height, width and color channels.
- A forecasting engineer distinguishes a single feature vector from a batch containing one feature vector before passing it to a model.
- An analyst reduces measurements across time while deciding whether to retain a length-one time axis for later broadcasting.
Think of it this way
Think of an address made of several coordinates. A stored value might need a folder number, page number and row number: three coordinate positions.
This analogy explains the number of axes, not the number of available positions along each one. It also does not imply that tensors physically occupy separate folders or that all programming systems assign identical meanings to axis order.
Worked example
The same values in different arrangements
Suppose a monitoring dataset contains two devices, three times and four measurements per time. Store it with shape (2,3,4). Its rank is three and its element count is 2×3×4=24.
Selecting one device with an operation that removes the device axis gives shape (3,4), rank two and 12 elements. Taking a one-device slice that preserves the axis gives shape (1,3,4), rank three and the same 12 elements. These results contain the same selected measurements but have different structures.
Averaging the original tensor across its time axis gives shape (2,4), rank two and eight values. Keeping that reduced axis instead gives shape (2,1,4), rank three and eight values.
Finally, reshaping the original 24 values into shape (2,12) changes the rank to two. It does not automatically preserve the meaning of a separate time axis for subsequent operations; the code must deliberately interpret the combined coordinates.
Illustrative case study
A fictional equipment-monitoring company builds a model expecting batches with axes for device, time and measurement. During a demonstration with one device, a preprocessing step removes the device axis. The model receives a rank-two tensor instead of the expected rank-three structure.
An engineer restores the batch axis and adds checks for both shape and axis meaning. The team also tests the reverse situation: a file reader can add an extra length-one axis that the model does not expect.
The lesson is that having the correct number of stored values is insufficient. The organization of those values is part of the interface between data preparation and the model.
Common misconceptions and mistakes
- Rank equals the number of elements. Rank counts axes; size counts elements.
- A one-element vector is a scalar. Shape (1,) has rank one, while scalar shape () has rank zero.
- Rank-two tensors always have matrix rank two. A zero matrix still has two axes but linear-algebra rank zero.
- Equal ranks imply compatible shapes. Axes can have incompatible lengths or meanings.
- Adding a length-one axis duplicates data. It changes structure without necessarily adding stored values.
Business use
Tensor rank matters at handoffs between data collection, preprocessing and model execution. Many deployment failures arise because one component interprets an axis differently or removes an axis that another component expects.
Document axis names alongside expected shapes. A specification such as batch-by-time-by-feature communicates more than “rank three.” Include single-example and empty-batch cases where the application supports them, because edge cases can reveal assumptions concealed by ordinary batch sizes. Rank validation helps detect structural mistakes, while semantic checks ensure that the right values occupy the right axes.
People also ask
Does a scalar have rank one because it contains one value?
No. It has one element but zero axes.
Can transposing increase rank?
A transpose that only permutes existing axes preserves rank. Adding a new axis is a different operation.
Does changing every value to zero reduce tensor rank?
No. The shape still has the same number of axes, although linear-algebra matrix rank may change for a two-axis tensor.
Check your understanding
A tensor has shape (1,6,1). What are its rank and size, and what happens if both length-one axes are removed?
Answer: Its rank is three and its size is six. Removing those two axes gives shape (6,), rank one and the same six elements.
Related terms
- Tensor size
- Scalar
- Broadcasting
- Tensor shape
- axis
- matrix rank
Learn next
Read tensor shape and broadcasting. Write the axis meanings for a batch of color images, then compare the expected structures for one image, a batch of one image and a batch of several images.
More in Mathematics and optimization
- Absolute error loss - Absolute error loss measures a numerical prediction's error as the absolute difference between its predicted and reference values: L(y,y-hat)=|y-y-hat|.
- Active-set method - An active-set method is an optimization approach that works with a changing subset of constraints believed to be tight at the solution.
- Adjoint state method - The adjoint state method computes how a chosen objective changes when the parameters of a mathematical model change.
- Σ-algebra
- Algebraic connectivity - Algebraic connectivity is the second-smallest eigenvalue of a graph's Laplacian matrix.
- Algorithm IMED - Algorithm IMED, short for Indexed Minimum Empirical Divergence, is a method for choosing actions in a stochastic multi-armed bandit problem.
Sources
Google Machine Learning Glossary, rank (Tensor): https://developers.google.com/machine-learning/glossary#rank_tensor. Source-list programming definition.
TensorFlow, tf.rank: https://www.tensorflow.org/api_docs/python/tf/rank. Official operation reference. The monitoring shapes and scenarios in this entry are original examples; this sense is distinct from algebraic tensor-decomposition rank.
Previous: Rank (ordinality) Next: Rank test