Is every similarity function an inner product?

No. It must satisfy the mathematical properties, not merely assign plausible scores.

Can two nonzero vectors have inner product zero?

Yes. They are orthogonal under that inner product.

Does an inner product determine a distance?

Yes. It induces a norm, and the distance between x and y is the norm of x−y.

AI Glossary / Mathematics and optimization / Intermediate

Inner product

Definition

An inner product is a rule that combines two vectors into a scalar while satisfying specific symmetry, linearity, and positivity properties. It provides a mathematical basis for lengths, angles, and orthogonality. The familiar dot product is one example, but other valid inner products can give the same vector space a different geometry.

What it means

For real coordinate vectors, the ordinary dot product multiplies matching components and sums the results. It measures alignment together with magnitude: large vectors can have a large dot product even when their directions are not especially close.
An inner product generalizes this operation. It can place different weights on directions or apply to objects such as functions, provided its defining properties hold. The choice determines what perpendicular means and how distances are measured. In AI applications, this makes representation and scaling choices important: a numerical score is meaningful only in relation to the geometry being used.

How it works

For real vectors, an inner product is symmetric, linear in each argument, and positive definite: the inner product of a vector with itself is nonnegative and is zero only for the zero vector. Its associated norm is the square root of that self-product.
In complex spaces, symmetry becomes conjugate symmetry, and one argument is conjugate-linear. Authors differ on which argument is linear, so state the convention. For real vectors, x-transpose M y defines an inner product when M is symmetric positive definite. A matrix with a negative direction or a nonzero zero-length direction does not satisfy the required positivity.

Three practical examples

A retrieval system can use vector dot products as scores, with normalization chosen according to its model and evaluation design.
A projection calculation can separate a vector into a component along a direction and an orthogonal remainder.
A weighted numerical model can define a geometry that gives some coordinate directions more influence than others.

Think of it this way

Think of measuring alignment with a ruler whose scale may differ by direction. A valid inner product supplies a consistent measuring system, including what counts as a right angle. The analogy is limited: an arbitrary scoring rule is not an inner product, and a learned similarity score does not automatically satisfy the mathematical axioms.

Worked example

Consider original real vectors x=(1,2) and y=(3,−1). Their ordinary dot product is 1 times 3 plus 2 times −1, giving 1. Their ordinary squared norms are 5 and 10, so their cosine similarity is 1 divided by the square root of 50, approximately 0.1414.
Now define a weighted inner product using M with diagonal entries 4 and 1 and zero off-diagonal entries. Then <x,y>_M=4(1)(3)+(2)(−1)=10. The weighted squared norms are <x,x>_M=8 and <y,y>_M=37, giving weighted cosine 10/sqrt(296), approximately 0.5812. The vectors did not change; the measuring geometry did.
To find the weighted projection of x onto the direction y, use coefficient c=<x,y>_M/<y,y>_M=10/37. The projection is (30/37,−10/37), leaving residual x−cy=(7/37,84/37). Its weighted inner product with y is 4(7/37)(3)+(84/37)(−1)=0, as required.
Using the ordinary dot product on that residual instead gives 21/37−84/37=−63/37. It is not ordinarily perpendicular to y. Orthogonality must be checked using the same inner product that defined the projection.
For a positivity counterexample, replacing M by a diagonal matrix with entries 1 and −1 gives the vector (0,1) a self-product of −1. That bilinear expression is not an inner product on this real vector space. Merely writing a matrix between two vectors does not establish a valid geometry.

Illustrative case study

In a fictional example, Ash Catalog ranks document vectors using dot products. A reviewer notices that some documents rank highly largely because their vector magnitudes are larger. The team compares its existing score with normalized cosine similarity on a held-out set of search tasks.
It does not assume normalization is automatically better: magnitude may carry useful information in the model. The resulting design note names the scoring rule and the evidence supporting it. This invented case illustrates the relationship between geometry and retrieval evaluation, without claiming a measured improvement or endorsing a particular embedding system.

Common misconceptions and mistakes

An inner product can be negative for two different vectors. Positivity concerns a vector paired with itself, not every pair. Similarly, orthogonal vectors need not be statistically independent variables; these are different concepts.
A dot product is not always cosine similarity. Cosine divides by both norms and is undefined for a zero vector unless software introduces a special handling rule. Changing units or feature scaling can also change dot products and the geometry of comparisons.
For complex numbers, forgetting conjugation can destroy positivity. Under the convention <x,y>=sum of conjugate(x_j) times y_j, the self-product of (i,0) is one. Multiplying i by itself without conjugation instead gives minus one, which cannot be a squared norm.

Business use

Inner products underlie many calculations in modeling, retrieval, and numerical analysis. Teams do not usually buy an inner product as a product feature, but they benefit from knowing which similarity and distance rules a system uses.
A useful technical explanation specifies normalization, feature scaling, and any weighting matrix. Test these choices against the actual task instead of interpreting a large raw score as universal semantic similarity. When implementing projections or optimization routines, keep the selected geometry consistent across formulas; mixing an ordinary norm with a weighted orthogonality condition can change the problem being solved.

Check your understanding

A proposed real inner product gives a nonzero vector a self-product of zero. Is it positive definite?

Answer: No. An inner product permits self-product zero only for the zero vector; the proposed rule may instead define a weaker semidefinite form.

Related terms

Dot product
Norm
Cosine similarity
Hilbert space
Kernel method
vector space
orthogonality
projection
positive-definite matrix

Learn next

Verify the weighted projection calculation and compare it with the ordinary projection coefficient 1/10. Then rescale the first coordinate of both vectors by two and observe how the ordinary dot product reproduces the chosen weighted inner product.

Dot product

Sources

Primary terminology source: Alexander Jung and collaborators, The Aalto Dictionary of Machine Learning, inner product entry: https://aaltodictionaryofml.github.io/ADictML_Main.pdf . Its defining properties and induced norm were consulted. The weighted geometry and projection examples are original. The complex example explicitly uses the conjugate-linear-first convention.

Previous: Inner fences Next: Inpainting