Standard symmetric SNE does not require class labels.

Does it preserve all distances?

No. It matches probabilistic relationships and can distort global geometry.

Why use joint probabilities?

They provide one symmetric pair representation and a simpler objective gradient than the original conditional formulation.

AI Glossary / Probabilistic machine learning / Advanced

Symmetric stochastic neighbor embedding

Definition

Symmetric stochastic neighbor embedding is a method for arranging high-dimensional observations in a low-dimensional map by matching symmetric probabilities assigned to pairs of observations. It replaces the original SNE collection of conditional-probability comparisons with one comparison between joint pair distributions. In its standard form, map similarities use a Gaussian kernel.

What it means

The name describes a relationship between two observations: the probability assigned to the ordered pair A, B equals that assigned to B, A. This removes the directional distinction present in conditional neighbor probabilities, where A can consider B a close neighbor without B assigning the same probability to A.

A symmetric map objective does not make every aspect of the method symmetric. Its Kullback - Leibler divergence still has a direction. The map is primarily a tool for exploring neighborhood relationships, and its axes do not automatically have business meanings or measurement units. Interpreting an apparent group requires returning to the original observations.

How it works

Begin with distances between observations and convert each observation's distances to conditional neighbor probabilities. Nearby observations receive larger probabilities, with local bandwidths controlling the scale of neighborhood comparison.

For n observations, combine the two directions using p(i,j) = [p(j given i) + p(i given j)] divided by 2n. Self-pair probabilities are zero. The resulting probabilities sum to one across all ordered nonself pairs.

Place observations in the display and assign each pair an unnormalized similarity exp(minus squared map distance). Normalize these similarities across all ordered nonself pairs to obtain q(i,j). Adjust the coordinates to reduce the sum of p(i,j) times log[p(i,j)/q(i,j)].

A useful implementation audit checks probability normalization before optimizing coordinates. Accidentally summing over unordered pairs while retaining ordered-pair constants changes the objective's scale and can create inconsistent gradients. Several initializations and neighborhood settings should be compared because a convenient objective does not imply a unique, globally optimal display.

Three practical examples

1. A text-analysis team explores whether document embeddings place related support topics near one another, then reads representative documents before naming groups.

2. A product researcher compares maps produced from different feature representations, investigating whether apparent neighborhoods survive sensible preprocessing changes.

3. A machine-learning instructor uses symmetric SNE to isolate the effect of joint affinities before introducing the heavier-tailed map similarities used in t-SNE.

Think of it this way

Think of people submitting lists of preferred conversation partners. A one-way nomination can be strong even when it is not reciprocated. Symmetrization combines both nominations into one shared relationship weight, and the seating arrangement tries to respect these weights. The seating chart is an interpretation of relationships, not a factual ranking of every person.

Worked example

Consider three observations, A, B and C. Stipulate valid conditional probabilities: A assigns 0.8 to B and 0.2 to C; B assigns 0.6 to A and 0.4 to C; C assigns 0.3 to A and 0.7 to B. These numbers are a teaching input, not a claim about a fitted dataset.

The symmetric probabilities are p(A,B) = 1.4/6 = 7/30, p(A,C) = 0.5/6 = 1/12, and p(B,C) = 1.1/6 = 11/60. Each reverse pair has the same value. The three listed values sum to one half, so all six ordered pairs sum to one.

Now place the three map points at the corners of an equilateral triangle. Equal distances give equal Gaussian similarities, so each ordered pair has q = 1/6.

Its divergence is twice the sum of (7/30) log(1.4), (1/12) log(0.5), and (11/60) log(1.1). This is approximately 0.0764 using natural logarithms. The nonzero result shows that equal map relationships do not reproduce the stipulated relationships. It does not establish which layout is globally optimal.

Illustrative case study

Fictional case: Alder Insights maps survey-response embeddings for an internal research workshop. Analysts initially interpret a separated island as a distinct customer segment. They repeat the display with several neighborhood scales and discover that the island's membership changes substantially.

The team keeps the map as a discussion aid, checks representative responses directly, and evaluates any proposed segmentation in the original feature space. Symmetric SNE helps generate questions, while independent evidence determines whether a proposed grouping deserves operational use. No customer decisions are automated from map coordinates.

Common misconceptions and mistakes

A common mistake is treating symmetric SNE as another name for t-SNE. Both use symmetric joint affinities, but t-SNE changes the low-dimensional similarity distribution to a heavy-tailed Student distribution.

Another mistake is assuming that symmetric affinities make KL(P||Q) equal KL(Q||P). They do not. Symmetrizing pair directions is different from symmetrizing a divergence.

Clear-looking gaps are not statistical evidence of distinct populations. Symmetrization also does not by itself eliminate the crowding difficulties of representing many-dimensional neighborhoods in two dimensions. Preprocessing, distance choice and optimization settings remain consequential.

Business use

Business value lies in exploratory analysis, representation comparison and analyst communication. A map can reveal observations worth investigating or suggest questions about a dataset. Teams should document the feature representation, scaling, neighborhood settings and random initialization, and retain access to original records. Evaluate business decisions using appropriate held-out evidence rather than visual attractiveness alone.

Check your understanding

If p(A,B) equals p(B,A), must KL(P||Q) equal KL(Q||P)?

Answer: No: pair symmetry does not remove divergence direction.

If only three unordered probabilities sum to one half, is normalization wrong?

Answer: Not when the objective counts their three reverse pairs as well.

Related terms

Stochastic neighbor embedding
Dimensionality reduction
Perplexity
t-SNE
Kullback–Leibler divergence
neighborhood preservation and crowding problem

Learn next

Next, compare Gaussian and Student map affinities at increasing distances. Then examine how local bandwidth selection and preprocessing affect neighborhoods before interpreting a visualization.

Sources

Primary source: Laurens van der Maaten and Geoffrey Hinton, Visualizing Data using t-SNE, Section 3.1, which presents symmetric SNE and distinguishes its Gaussian map similarities from t-SNE: https://www.jmlr.org/papers/volume9/vandermaaten08a/vandermaaten08a.pdf

The conditional-probability example and fictional case above are original teaching material.

Independent review consulted Section 3.1, including Equation 3, symmetrized conditional probabilities and the directed KL objective.

Previous: Symmetric relationships Next: Symmetrically Connected Network