AI Glossary / AI and machine learning / Advanced

Einsum notation

Definition

Einsum notation is a compact way of writing operations on multi-dimensional arrays by naming their axes, so sums, products, transposes and contractions are expressed in one short instruction.

What it means

Arrays in modern models are not flat tables but tensors with many axes: a batch of images might have axes for batch, height, width and color channel. Operations over them, multiplying, summing, rearranging, quickly become unreadable as loops.

Einsum, short for Einstein summation, borrows a physicist's shorthand: label each axis with a letter, and the operation is described purely by how the labels combine. Labels repeated across inputs are summed over; labels appearing once are kept.

One string replaces pages of code: matrix multiplication, batch dot products, attention scores and transposes are all single Einsum expressions differing only in their letters.

The cost is a learning curve. An Einsum expression is illegible until the convention clicks, and a mistyped letter silently produces a wrong-shaped result that may not fail loudly.

For a business, Einsum sits inside the libraries your AI vendors and data teams use. You will not write it, but seeing it in documentation should read as array plumbing, not mystery.

The notation also makes a whole family of errors visible at a glance: if the output labels do not match what you intended to keep, the expression says so plainly, which is why tensor code written in Einsum is easier to review than the same logic buried in reshapes.

How it works

Label each input array's axes with letters: an array of shape batch-by-features might be labeled bf.

Write the operation as the input labels, an arrow, and the output labels: the letters alone define the computation.

Any label that appears in the inputs but not the output is summed away; any label shared between inputs is matched element by element.

Read a few canonical examples until the convention settles: matrix multiply, batch vector products, trace and transpose each have one-line forms.

Use a library's Einsum function, available in NumPy, TensorFlow, PyTorch and others, which parses the string and executes the implied operation efficiently.

Verify the result's shape against the output labels every time during development; the shape is the executable reading of the string, and a mismatch is the bug finder.

Three practical examples

• Attention mechanisms: computing query-key score matrices across a batch in one expression.
• Matrix multiplication: the two-letter shared-index case that makes the convention click for new readers.
• Tensor reshaping: rearranging axes of batched data without explicit loops or transpose calls.

Think of it this way

Think of knitting instructions: k2, p2, repeat. Cryptic until learned, then a whole jumper in a line. Einsum is the same economy for array mathematics, and just as opaque to the uninitiated.

The limit: a dropped stitch in a compact pattern ruins the whole row silently. A wrong axis letter does the same, so shape checks are the proof-reader.

Worked example

reading an attention one-liner

Suppose a data team inherits model code containing the expression that scores queries against keys across a batch, written as a single Einsum string. The example is illustrative.

The string reads: batch-query-dim times batch-key-dim gives batch-query-key, with the shared dim label summed away.

Recognizing the shared-letter-sum pattern tells the team this is a batched dot product, the heart of attention scoring, without stepping through any loops.

They add a shape assertion after the line, because a mistyped label would still run while producing nonsense.

Illustrative case study

A fictional analytics firm copy-pasted an Einsum expression between two models and changed the data layout in only one of them. The expression still ran, summed over the wrong axis, and returned plausible-looking numbers.

The error surfaced weeks later in a client report whose totals refused to reconcile. A one-line shape check would have caught it on day one.

The invented lesson: compact notation moves the failure from crashing to quietly wrong. Guard the shapes.

Common misconceptions and mistakes

• It is a different kind of mathematics. It is ordinary array arithmetic in compact notation; nothing new is computed, only said more briefly.
• It is faster by nature. It can be, since the library plans the operation, but the gain is mainly clarity and generality, not magic speed.
• You must master it to use AI libraries. Plenty of fine work avoids it; it becomes essential mainly when reading or writing tensor-heavy code.

Business use

Treat Einsum strings in code review as concentrated risk: one letter wrong means silently wrong results.

Require shape assertions beside Einsum expressions in any codebase your business depends on.

When reading vendor documentation, translate the string into the shared-letters-sum rule before trusting your understanding.

For teams maintaining tensor code, a short internal primer on the convention pays for itself the first time a subtle axis bug is caught in review rather than in a client report.

People also ask

Where does the name come from?

From Einstein's summation convention in physics, where repeated indices imply summation without writing the sum sign.

Which libraries support it?

NumPy, PyTorch, TensorFlow, JAX and most tensor libraries, with near-identical syntax.

Is it only for deep learning?

No; any multi-dimensional array work benefits, including statistics and scientific computing.

Check your understanding

In Einsum notation, what does a letter that appears in the inputs but not in the output mean?

Answer: That axis is summed away: the result accumulates over it, which is how products become dot products and contractions.

Related terms

• Broadcasting
• tensor
• matrix multiplication
• attention mechanism
• NumPy
• axis

Learn next

AI for Business Owners helps you ask the right questions when technical documentation stands between you and a decision: https://aiforbusinessandlife.com/academy/ai-for-business-owners. [C2]

More in AI and machine learning

• Dynamic epistemic logic - Dynamic epistemic logic studies how knowledge and beliefs change when information is communicated or events are observed.
• Eager learning - Eager learning builds a general predictive model from training examples before new prediction requests arrive.
• Early stopping - Early stopping is ending a model's training at the point where it performs best on fresh data, before it starts memorizing the training set.
• Elastic-Net - Elastic-Net is a regression method that combines the two classic penalty types, L1 and L2, so it selects variables like the first and stabilizes estimates...
• ELMo - ELMo, short for Embeddings from Language Models, is a method for producing word representations that change with the surrounding text.
• Embodied agent - An embodied agent perceives and acts through a body situated in an environment.

Sources

[S2] Google, Machine Learning Glossary. Full URLs are in the source ledger. Verified 11 October 2026.

Previous: Eigenvector Next: Elastic map