AI Glossary / Probabilistic machine learning / Advanced

Representational similarity analysis

Definition

Representational similarity analysis compares how different systems distinguish the same set of items or conditions. It first summarizes pairwise differences between each system's response patterns in a representational dissimilarity matrix, then compares those matrices to study shared or different representational structure.

What it means

Two neural networks can have different numbers of units, and a brain recording can have entirely different measurement channels. Directly matching individual coordinates may be difficult or inappropriate. RSA instead asks whether the same pairs of items are represented as relatively similar or dissimilar.
The comparison concerns relationships among items, not necessarily the identity of individual features or mechanisms. It can connect computational models, neural measurements, and behavioral judgments. In AI work, it can also help compare layers, model versions, or embeddings on a common evaluation set.

How it works

Choose the same conditions for each system and extract one response pattern per condition, using a clearly defined averaging or estimation procedure. Select a dissimilarity measure, such as Euclidean distance or one minus correlation, and calculate it for every pair of conditions.
Place the pairwise values in a symmetric representational dissimilarity matrix, or RDM. For ordinary self-distances its diagonal is zero. Compare corresponding off-diagonal entries, usually taking only one triangle to avoid counting each pair twice. Pearson or rank correlation answers different questions about agreement, so the comparison measure should be chosen deliberately.
Uncertainty assessment must respect the data structure. Pairwise distances sharing an item are not independent observations. Repeated measurements, subject or condition resampling, and appropriately designed permutation procedures can help support inference. A visually similar pair of heatmaps is exploratory evidence rather than a complete statistical test.

Three practical examples

• Model comparison: a vision team compares how two image encoders organize the same products, even when their embedding dimensions differ.
• Layer analysis: a researcher asks whether early and late network layers emphasize different relationships among a fixed set of inputs.
• Human-model comparison: an experiment compares model dissimilarities with behavioral judgments or measured neural response patterns, using an analysis that accounts for repeated observations and noise.

Think of it this way

Think of comparing two maps by their tables of distances between the same towns. Their coordinate systems can differ while the relationships between towns remain alike. But matching distance tables does not establish that the maps were drawn by the same process, and a correlation comparison may even ignore an overall difference in scale.

Worked example

Consider an original example with four conditions A, B, C, and D. System X represents them on a line at 0, 1, 3, and 7. Using Euclidean distance and pair order AB, AC, AD, BC, BD, CD gives the RDM's unique entries as (1, 3, 7, 2, 6, 4).
System Y uses two-dimensional vectors (0, 5), (0, 7), (0, 11), and (0, 19). Its corresponding distances are (2, 6, 14, 4, 12, 8), exactly twice X's distances. Pearson correlation between the two distance vectors is 1, despite the different coordinates and scale.
Now let system Z represent the same labeled conditions at 0, 3, 1, and 7. Its distances are (3, 1, 7, 2, 4, 6). The distance ranks for X are (1, 3, 6, 2, 5, 4), and for Z they are (3, 1, 6, 2, 4, 5). Squared rank differences sum to 10, giving Spearman correlation 1 - 6(10)/[6(6^2 - 1)] = 5/7, approximately 0.714.
These toy calculations describe agreement, not statistical significance. Six pair values arise from only four conditions, and shared conditions make those values dependent.

Illustrative case study

In a fictional project, Cedar Search updates a product-image encoder and wants to understand whether its internal organization has changed. The team fixes a representative set of images, extracts both embeddings, and compares their RDMs using a distance measure appropriate to the retrieval system.
A high overall agreement conceals a smaller subgroup whose relationships changed substantially. The analysts inspect that subgroup and run separate retrieval tests before deciding whether the update is acceptable. They repeat the analysis with additional images. This case illustrates using RSA to guide investigation rather than treating one similarity score as proof of unchanged product behavior.

Common misconceptions and mistakes

High RSA agreement does not show that two models use the same computation, have identical features, or achieve equal task accuracy. The result depends on the chosen conditions and dissimilarity measure. Pearson correlation of RDM entries can be perfect despite an overall distance-scale difference, as the example demonstrates.
Including both matrix triangles duplicates information. Treating every pair as an independent sample can exaggerate confidence. Selecting a layer, distance measure, or condition subset because it produces an attractive result on the same evaluation data also risks overfitting the analysis. Correlation-based pattern distances and correlations between RDMs are separate analytical choices.

Business use

Use RSA when the question concerns changes in representational relationships. Document the input set, extraction layer, preprocessing, distance measure, and matrix-comparison statistic. Pair it with task-level evaluation so that a representational finding has an operational interpretation.
Include repeated runs or independent evaluation conditions where feasible. Review important subgroups instead of relying only on a single aggregate coefficient. If comparing noisy human measurements with deterministic model outputs, account for measurement reliability when interpreting the ceiling on observable agreement. Avoid claiming that a model reproduces human reasoning solely because its RDM correlates with behavioral data.

People also ask

Must the two representations have the same dimension?

No. They need corresponding conditions, but their feature dimensions can differ.

Is RSA the same as clustering?

No. Clustering may summarize an RDM, whereas RSA compares representational relationships across systems or hypotheses.

Does a correlation of one mean identical embeddings?

No. The example has perfect distance correlation despite different coordinates and an overall scaling difference.

Check your understanding

With four conditions, how many unique off-diagonal pairwise comparisons does a symmetric RDM contain?

Answer: Four times three divided by two, or six. This count does not mean six statistically independent observations, because several pairs share the same conditions.

Related terms

• Embedding
• Euclidean distance
• Multidimensional Scaling
• Permutation test
• Representational dissimilarity matrix
• correlation distance
• rank correlation
• neural representation
• representational geometry

Learn next

Construct all three RDMs from the example and verify the correlations from one triangle. Then compare Euclidean and correlation-based pattern dissimilarities on a small multichannel dataset to see how the analytical question changes.

More in Probabilistic machine learning

• Regularization path - A regularization path is the sequence of fitted model solutions obtained as the strength of a regularization penalty varies.
• Regularized discriminant analysis - Regularized discriminant analysis is a classification approach that stabilizes estimated class covariance matrices through shrinkage.
• REINFORCE - REINFORCE is a family of stochastic gradient methods that adjusts a policy using sampled rewards or returns multiplied by the gradient of the log...
• Residual flow - A residual flow is a normalizing-flow model built from invertible residual transformations of the form f(x) = x + g(x).
• Riemann manifold Hamiltonian Monte Carlo - Riemann manifold Hamiltonian Monte Carlo is an MCMC method that uses a position-dependent metric to adapt Hamiltonian sampling dynamics to local features of...
• Robust logistic regression - Robust logistic regression refers to extensions of logistic regression designed to reduce sensitivity to specified departures from ordinary modeling...

Sources

Kriegeskorte, Mur and Bandettini, Representational Similarity Analysis—Connecting the Branches of Systems Neuroscience, original research article and methods: https://www.frontiersin.org/journals/systems-neuroscience/articles/10.3389/neuro.06.004.2008/full Original article RDM construction and statistical-inference passages independently consulted, including the dependence among shared-item dissimilarities.

Previous: Representational harm Next: Reproducing kernel Hilbert space