Is a Gram matrix always square?

Yes. It compares a set of vectors with itself, giving one row and column per vector.

Can it contain negative values?

Yes. Off-diagonal inner products can be negative, although diagonal squared lengths are nonnegative for the usual real inner product.

Does it uniquely determine the original vectors?

Generally not in fixed coordinates. Different vector configurations related by suitable transformations can share the same pairwise inner products.

AI Glossary / Computer vision and multimodal AI / Intermediate

Gram matrix

Definition

A Gram matrix records the pairwise inner products of a set of vectors. Each entry measures how two vectors align and scale relative to one another under the chosen inner product. In neural style transfer, Gram matrices of feature maps summarize relationships between feature channels across an image.

What it means

A list of vectors may describe observations, image features or mathematical basis functions. The Gram matrix compares every vector with every other vector, including itself. Its diagonal entries are squared vector lengths, while its off-diagonal entries describe pairwise inner products.

In a style-transfer setting, each feature channel can be treated as a vector over spatial positions. Taking channel inner products summarizes which responses tend to occur together. This can capture useful texture-related statistics without keeping a separate entry for every spatial location.

The same mathematical object appears in many areas beyond image style. Its meaning depends on what the vectors represent and how they are scaled. A Gram matrix is not automatically a matrix of probabilities or a complete description of an image's content.

How it works

If the vectors form the columns of a real matrix X, their Gram matrix is G = XᵀX. If they form rows, the corresponding construction is XXᵀ. Stating the arrangement prevents dimension confusion.

For real vectors, G is symmetric and positive semidefinite. Its diagonal is nonnegative. Off-diagonal entries can be negative when vectors point in opposing directions; they are not constrained to the range of a probability.

In neural style transfer, a feature matrix may arrange channels as rows and spatial positions as columns. The channel Gram matrix is then FFᵀ, sometimes normalized. A loss compares such matrices between generated and style images. Normalization and feature choice affect the scale and interpretation of that loss.

Three practical examples

1. Style transfer: A system compares feature-channel Gram matrices to encourage similar texture statistics between a generated image and a style reference.

2. Kernel methods: Pairwise kernel evaluations form a kernel Gram matrix used by methods that work through similarities in an implicit feature space.

3. Linear algebra: A researcher uses a Gram matrix to study relationships and dependence among a collection of vectors or basis functions.

Think of it this way

Think of a table that records how strongly every pair of instruments appears together across a set of musical passages. The table captures relationships between channels of information while leaving out some details about the order of the passages.

The analogy is limited because an inner product is a precise numerical calculation, not necessarily a statistical correlation coefficient. Vector length and scaling affect the values, and the entries need not be normalized.

Worked example

Take vectors v₁ = [1, 2] and v₂ = [3, 4]. Their inner products are v₁·v₁ = 5, v₁·v₂ = 11, v₂·v₁ = 11 and v₂·v₂ = 25. The Gram matrix is [[5, 11], [11, 25]].

Now swap the two coordinate positions in both vectors, giving [2, 1] and [4, 3]. The Gram matrix stays the same. In the feature-map interpretation, a common permutation of spatial positions leaves these channel inner products unchanged.

This illustrates why the matrix does not retain a complete spatial layout. The observation applies at the feature-matrix level; arbitrarily shuffling image pixels can change the features extracted by a convolutional network, so it need not leave the image's computed style representation unchanged.

Illustrative case study

Fictional case: Willow Creative builds a texture-transfer experiment. Two outputs have similar Gram-based style losses, yet one preserves the intended object layout more clearly. The team recognizes that its style statistic alone does not enforce the desired content arrangement.

It evaluates a separate content objective and inspects generated text and small structures. The developers also standardize feature normalization before comparing loss values across image sizes. The case illustrates complementary objectives and the limits of a summary statistic, not a claim that Gram matrices capture an artist's complete style.

Common misconceptions and mistakes

A Gram matrix is not necessarily a covariance matrix. Covariance involves centering and a particular normalization; raw inner products need not include either.

An off-diagonal value is not automatically cosine similarity. Cosine similarity additionally accounts for vector norms.

Matching Gram matrices does not guarantee identical vectors, images or spatial arrangements. The representation deliberately loses information.

Positive semidefinite does not mean every entry is positive. It describes the quadratic form associated with the matrix. Also, different feature extractors and normalizations produce different Gram statistics, so a loss value is not universally comparable across implementations.

Business use

Gram matrices provide compact relational summaries that can support style objectives, similarity-based methods and mathematical analysis. They are useful when pairwise feature relationships matter more than retaining every raw coordinate.

Teams should document vector orientation, feature selection, centering and normalization. For creative applications, evaluate content fidelity and user intent separately from the style statistic. For analytical applications, verify that the chosen inner product and feature scaling match the meaning of similarity required by the task.

Check your understanding

Two feature matrices have the same channel Gram matrix after a common permutation of spatial positions. Does the matrix preserve the exact location of each feature response?

Answer: No. Those pairwise sums are unchanged by that shared permutation. They summarize channel relationships rather than retaining a full spatial map, which is why additional content information may be needed.

Related terms

Inner product
Feature map
Covariance matrix
Neural style transfer
kernel matrix
positive semidefinite matrix
style loss

Learn next

Calculate a Gram matrix for two short vectors and compare it with their normalized cosine similarities. Then center the vectors and observe how the values change. In a style-transfer example, distinguish the channel-correlation objective from the separate requirement to preserve content.

Sources

Primary research: Leon A. Gatys, Alexander S. Ecker and Matthias Bethge, “A Neural Algorithm of Artistic Style,” 2015, https://arxiv.org/abs/1508.06576 ; author paper: https://arxiv.org/pdf/1508.06576 . The paper was accessed for the feature-correlation context. The vector calculation and spatial-permutation explanation are original mathematical illustrations, not benchmark claims.

Previous: Grafting (decision trees) Next: Grammar