Is Sinkhorn a neural network?

No. It is a matrix-scaling algorithm, though its computations can be used inside differentiable learning systems.

Does entropy remove the marginal constraints?

No. Standard balanced entropic transport still matches the prescribed row and column totals.

Is a Sinkhorn divergence the same as the regularized objective?

No. Common debiased versions subtract self-comparison terms; their properties depend on the precise definition and assumptions.

AI Glossary / Probabilistic machine learning / Advanced

Entropically regularized optimal transport

Definition

Entropically regularized optimal transport finds a mass-transfer plan that balances low transport cost with a preference for a more spread-out plan. Adding an entropy term makes many discrete transport problems amenable to efficient matrix-scaling methods, especially Sinkhorn iterations, while changing the solution relative to unregularized optimal transport.

What it means

Optimal transport compares distributions by asking how mass can move between their locations at minimum cost. A transport plan specifies how much goes from each source location to each destination. In balanced transport, its row and column totals must match the supplied source and destination masses.

Entropy regularization discourages concentrating all mass on a few pairings. It creates a trade-off between matching cheaply and distributing the coupling more broadly. The “mass” may represent probability or feature weight rather than physical goods. A soft statistical coupling should not automatically be interpreted as an executable delivery schedule or proof of a causal relationship between matched items.

How it works

One convention minimizes transport cost plus epsilon times the sum of P_ij log P_ij, subject to nonnegative entries and fixed marginals. Here epsilon is positive, and 0 log 0 is defined as zero. Because this sum is negative entropy, larger epsilon favors higher entropy.

For positive marginals and finite costs, the solution can be represented by scaling a kernel whose entries are exp(−cost/epsilon). Sinkhorn iterations alternately adjust row and column scaling to meet the marginal constraints.

Small epsilon approaches the unregularized problem but can cause numerical difficulties; stabilized or log-domain computations are often useful. Different libraries report different quantities: the transport cost of the regularized plan, the full regularized objective, and a debiased Sinkhorn divergence are not interchangeable. Always identify the convention used.

Three practical examples

1. An image-comparison system couples color histograms, allowing nearby colors to match at lower cost than distant colors. Regularization controls how diffuse the correspondence becomes.

2. A researcher aligns feature distributions from two datasets using a soft transport plan. The cost representation determines whether those correspondences are substantively meaningful.

3. A generative-model experiment uses a transport-based training loss. It checks the effect of the entropy scale and whether self-comparison bias is addressed by the chosen divergence.

Think of it this way

Think of distributing people among equally sized workshop tables. Minimizing only walking distance may concentrate pairings in a few obvious assignments; an added preference for variety produces softer mixing while respecting table capacities. The analogy is limited because entropy concerns the distribution of numerical mass, not a universal social benefit, and real people cannot necessarily be split fractionally across tables.

Worked example

Consider two source locations and two destinations, each with masses 0.5 and 0.5. The cost matrix has zero diagonal costs and off-diagonal costs of 2. Let epsilon = 1, using natural logarithms and the objective convention stated above.

By symmetry, a feasible plan has diagonal entries a and off-diagonal entries b, where a + b = 0.5. The scaled exponential kernel implies b/a = exp(−2), approximately 0.1353. Thus a = 0.5/(1 + exp(−2)), approximately 0.4404, and b is approximately 0.0596.

Each row and column sums to 0.5. The transport-cost component is 4b, approximately 0.2384. Unregularized optimal transport instead puts 0.5 on each diagonal and zero elsewhere, with transport cost zero.

The regularized plan therefore pays additional transport cost to gain entropy. Both input distributions are identical here, yet the regularized plan's transport-cost component is positive. This demonstrates why that component should not automatically be called a metric with zero self-distance. The example does not calculate a debiased Sinkhorn divergence or compare full objectives under another entropy convention.

Illustrative case study

Fictional case: Palette Harbour compares product photographs through color histograms. Its developer initially chooses a large entropy scale because calculations are stable, but notices that distinct color arrangements receive overly diffuse matches. She evaluates several scales and checks marginal residuals as well as retrieval quality.

The team records whether its dashboard reports raw transport cost or a debiased divergence. It also acknowledges that a color histogram discards spatial layout, so two very different pictures may still look similar under this representation. This hypothetical project reports no measured speed-up or accuracy gain.

Common misconceptions and mistakes

Regularization changes the optimization problem; it is not merely a faster exact solver for the original objective at finite epsilon. More entropy does not mean that the input distributions themselves have become more uncertain.

Small epsilon is not always numerically easy, and stopping too early can leave inaccurate marginal totals. A visually plausible matrix is not sufficient evidence of convergence.

Balanced transport assumes equal total mass. If the application requires creating, discarding or leaving mass unmatched, an unbalanced or partial formulation may be appropriate. Finally, entropy regularization cannot repair a cost matrix that encodes the wrong notion of similarity.

Business use

For business applications, this method is useful when distribution comparisons or soft correspondences are more informative than comparing bins independently. It is particularly attractive when many related transport problems must be computed.

Specify the units and scale of costs before choosing epsilon: rescaling costs without rescaling the entropy coefficient changes the trade-off. Test sensitivity to regularization and stopping tolerances, and report the exact returned statistic. Keep numerical speed separate from decision quality. A faster comparison is valuable only if its representation and coupling support the intended task.

Check your understanding

If every cost and epsilon in the worked example are both multiplied by 10, does the kernel exp(−cost/epsilon) change?

Answer: No. The ratio is unchanged, so the optimal plan is unchanged for the same marginals, although objective values scale under this convention.

Related terms

Optimal transport
Entropy
transport plan
Sinkhorn algorithm
Wasserstein distance
coupling
cost matrix
Sinkhorn divergence

Learn next

Verify row and column sums after alternating scaling steps on a small positive matrix. Then compare unregularized cost, regularized cost and debiased divergence on identical distributions.

Sources

Primary sources: Cuturi, Sinkhorn Distances: Lightspeed Computation of Optimal Transport, formulation and matrix-scaling discussion: https://papers.neurips.cc/paper_files/paper/2013/file/af21d0c97db2e27e13572cbf59eb343d-Paper.pdf ; POT project API, Sinkhorn implementation conventions: https://pythonot.github.io/all.html#ot.sinkhorn . Independent review checked Cuturi Section 4, exponential scaling and entropy convention. The symmetric two-by-two calculation and fictional case are original; no historical benchmark result is reproduced.

Previous: Entropic vector Next: Entropy