Is a norm the same as a distance?

A norm measures one vector's size. Applying it to the difference between vectors creates a distance.

Is squared L2 length a norm?

No. It is useful in many objectives but fails the required linear scaling and triangle-inequality rules.

Which norm should I use?

Choose according to the mathematical problem and the practical meaning of size, rather than assuming Euclidean length is always the right criterion.

AI Glossary / Mathematics and optimization / Intermediate

Norm

Definition

A norm is a mathematical function that assigns a nonnegative size to a vector while satisfying three rules: only the zero vector has size zero, scaling a vector scales its norm by the absolute scale factor, and the norm of a sum cannot exceed the sum of the norms. Different norms express different meanings of size.

What it means

The familiar straight-line length of a vector is one norm, but not the only one. A vector of prediction errors can be measured by its total absolute error, its Euclidean size, or its largest absolute component. Each emphasizes a different aspect of the same errors.

Norms apply to objects in vector spaces, including matrices and functions under appropriate definitions. They are not interchangeable with every quantity casually called a magnitude or score. A loss function can be useful without satisfying the norm rules, and squared Euclidean length is a common example of such a useful non-norm quantity.

How it works

For a real vector x=(x_1,...,x_d), the L1 norm is sum_i |x_i|. The L2 or Euclidean norm is sqrt(sum_i x_i²). The infinity norm is max_i |x_i|. These are cases of familiar Lp norms for p≥1, with the infinity case interpreted separately.

A norm creates a distance through d(x,y)=‖x−y‖. It can therefore define neighborhoods, stopping tolerances, and the size of model updates. A norm penalty on parameters also expresses a preference about model size; changing the norm changes that preference.

Always specify which object and norm are intended. For a matrix, the induced infinity norm is the largest absolute row sum, while applying a maximum-entry rule gives a different quantity. Software options can therefore have different meanings for one-dimensional vectors and two-dimensional matrices.

Three practical examples

1. Error monitoring: use an infinity norm to inspect the largest absolute component of a vector of deviations when a componentwise tolerance matters.

2. Model fitting: penalize an L1 parameter norm when seeking a sparse solution in an appropriate regularized optimization problem.

3. Numerical algorithms: measure residual or update sizes with a stated norm to determine whether a computation meets its intended stopping criteria.

Think of it this way

Think of measuring a journey by straight-line distance, total movement along a grid, or the largest displacement in one direction. The same endpoints can produce different numbers because the measuring rules answer different questions.

The analogy explains why no single norm captures every practical concern. It does not mean arbitrary travel costs form a norm: one-way roads, fixed fees, and other constraints can violate symmetry or scaling properties. Mathematical assumptions must be checked rather than inferred from the word “distance.”

Worked example

Take x=(−3,4). Its L1 norm is 3+4=7, its L2 norm is sqrt(9+16)=5, and its infinity norm is four. Multiplying by −2 gives (6,−8), whose norms are fourteen, ten, and eight: each doubles, as absolute homogeneity requires.

Now compare error vectors a=(4,0) and b=(3,3). The L1 sizes are four and six, so a is smaller by total absolute error. The infinity sizes are four and three, so b is smaller by worst-component error. Neither ranking is contradictory; the norms express different criteria.

For the triangle inequality, choose u=(1,0) and v=(0,1). Their Euclidean norms are one and one, while ‖u+v‖_2=sqrt2≤2. By contrast, squared Euclidean length is not a norm: for scalar values u=v=1, the squared length of u+v is four, exceeding one plus one. It also scales quadratically rather than by the absolute scale factor.

Finally, the matrix with rows (1,−2) and (3,4) has maximum absolute entry four but induced infinity norm seven, from its second row's absolute sum. The word “maximum” needs a precise object-level definition.

Illustrative case study

Fictional case study: an operations dashboard reports the average size of forecast errors, while a downstream controller rejects any individual error above a strict tolerance. A team initially uses one aggregated norm-related score for both purposes.

Reviewers compare a few error vectors and discover that the aggregate can look acceptable while one component violates the controller's limit. The dashboard therefore distinguishes total, Euclidean, and worst-component summaries where each is relevant. This improves interpretability of the measurement rule; it does not establish a better forecasting model.

Common misconceptions and mistakes

Do not assume a larger norm is always worse. The interpretation depends on whether the vector represents error, signal, parameters, or another quantity.

Do not treat every expression labeled an “L0 norm” as a mathematical norm. Counting nonzero entries fails absolute homogeneity. Similarly, the usual Lp expression with 0<p<1 is not a norm because the triangle inequality fails in that form.

Do not ignore feature units. Combining quantities measured on incompatible scales can make a norm dominated by a numerical unit choice. Standardization or weighting must have a documented purpose and be fitted appropriately when learned from data.

Business use

Norms make statements such as “small error,” “limited change,” and “similar parameters” precise. Their business relevance comes from aligning the mathematical size measure with the requirement being monitored.

Specify the norm, units, any weights, and whether values are summed, averaged, or rooted. Compare residual size with problem scale rather than applying an unexplained universal threshold. Also distinguish a small residual from a small solution error: an ill-conditioned problem can amplify the relationship between them.

Check your understanding

Which worked error vector has the smaller worst-component error, a=(4,0) or b=(3,3)?

Answer: b has infinity norm three, compared with four for a. However, a has smaller L1 norm, showing that a ranking depends on the chosen criterion.

Related terms

Learn next

plot the two-dimensional unit balls for L1, L2, and infinity norms. Then compare candidate parameter or error vectors under all three, explaining which practical requirement each ranking would support.

Sources

Sources: Aalto Dictionary of Machine Learning, norm: https://aaltodictionaryofml.github.io/ADictML_Main.pdf ; NumPy, numpy.linalg.norm: https://numpy.org/doc/stable/reference/generated/numpy.linalg.norm.html . Primary norm axioms and vector/matrix norm conventions were read. Non-norm uses of L0, squared L2, and p<1 are distinguished explicitly. All numerical comparisons and the fictional case are original.

Previous: Nontrivial MVD Next: Normal Approximation To The Binomial Distribution