AI Glossary / Probabilistic machine learning / Intermediate
Potts model
Definition
A Potts model is a model of interacting variables that each take one of several categorical states. Its characteristic pairwise interaction distinguishes whether neighboring variables have the same label or different labels. In common machine-learning applications, positive disagreement penalties encourage nearby variables to share labels while allowing boundaries where other evidence supports them.
What it means
Imagine assigning a category to every pixel in an image. Treating pixels independently can produce isolated, inconsistent labels. A Potts interaction adds a preference for neighboring pixels to agree, without assuming that the category names have a numerical ordering.
For example, a disagreement between labels A and B can receive the same penalty as a disagreement between A and C. The model is therefore different from a penalty based on the numerical distance between label codes. The classic interaction is simple, but the collective distribution over all labels can still be computationally difficult.
How it works
One common energy formulation is E(z) = sum_i D_i(z_i) + sum_(i,j) w_ij I(z_i != z_j). Here D_i is a local cost, the second sum runs over graph edges, and nonnegative w_ij penalizes disagreement. A corresponding probability model has p(z) = exp[-E(z)]/Z, where Z normalizes over every possible labeling.
The equivalent agreement-reward formulation uses indicators of equality in the exponent, with a constant adjustment that disappears into normalization. State the sign convention: a positive disagreement penalty encourages agreement, while other couplings can have different effects.
Finding the most probable labeling minimizes energy, but calculating marginal probabilities requires summing over alternatives. These are separate inference problems. A small graph can be enumerated; larger problems need methods suited to their structure. The existence of graph-cut approaches does not mean every multi-label Potts problem has an easy exact solution.
Three practical examples
- Image segmentation: combine local evidence for each pixel's class with a penalty for unnecessary label changes between neighbors.
- Categorical map smoothing: encourage nearby map cells to have coherent categories while retaining locally supported boundaries.
- Discrete sequence analysis: penalize changes between neighboring categorical states when a piecewise-constant description is a defensible assumption.
Think of it this way
Think of neighboring tiles that prefer matching colors, while each tile also has its own reason to choose a particular color. Matching preferences can remove isolated patches, but sufficiently strong local reasons can preserve an edge. The metaphor does not mean all neighboring real-world objects should share a class; the agreement preference is a modeling assumption that can be wrong.
Worked example
Consider an original two-node model with labels A, B, and C, no local costs, and one disagreement penalty w = log(2). There are nine joint labelings: three matching pairs and six mismatching pairs.
Each matching pair has energy 0 and unnormalized weight 1. Each mismatching pair has energy log(2) and weight exp[-log(2)] = 1/2. Therefore Z = 3 × 1 + 6 × 1/2 = 6.
The total probability of a match is 3/6 = 1/2. Any particular matching pair has probability 1/6; any particular mismatching pair has probability 1/12. If the penalty were zero, all nine labelings would be equally likely and the match probability would be 3/9 = 1/3.
The penalty increases agreement but does not make agreement certain. Conditional on the first node being A, the second node has weights 1, 1/2, and 1/2 for A, B, and C, giving probabilities 1/2, 1/4, and 1/4. Its marginal label distribution remains uniform because the toy model treats all labels symmetrically.
Illustrative case study
Fictional case: Meadow Vision labels surfaces in inspection images. Independent pixel predictions produce scattered single-pixel patches, so the team adds a Potts disagreement term to its labeling energy. It compares several strengths on annotated evaluation images.
The team discovers that stronger smoothing can remove both unwanted speckles and small genuine regions. It therefore examines boundary quality and small-region retention rather than choosing the smoothest image by appearance alone. The model's graph and local costs are documented with the selected penalty. This scenario is invented; it does not assert a real inspection improvement or establish one penalty as universally suitable.
Common misconceptions and mistakes
A Potts model is not merely any model with categorical labels: the characteristic equality-versus-disagreement interaction matters. It does not inherently know which labels are semantically similar unless the formulation adds such structure.
The most probable joint labeling is also not the same as independently choosing the most probable marginal label at every node. Finally, a smooth-looking answer does not prove either good model assumptions or successful optimization. Over-smoothing and approximate-inference errors are distinct possibilities.
Business use
For business applications involving categorical images, maps, or sequences, the model offers an interpretable way to trade local evidence against label coherence. This can be useful when isolated changes are unlikely but real boundaries still matter.
Choose an evaluation that reflects the cost of deleting small regions or shifting boundaries. Document whether the output is a best labeling, a marginal probability, or an approximation to either. A low energy under the chosen model alone is not evidence that categories match reality.
People also ask
How is it related to the Ising model?
The two-state Potts model is equivalent to an Ising interaction after suitable parameter scaling and an additive energy constant.
Must neighbors agree?
No. Finite penalties favor agreement but allow disagreement, especially when local evidence offsets the penalty.
Are label numbers meaningful distances?
Not in the basic Potts interaction. It tests equality, so renaming categories consistently leaves that interaction unchanged.
Check your understanding
In the worked example, does each matching pair have probability one half?
Answer: No. One half is the total probability across all three matching pairs. Each particular matching pair has probability one sixth.
Related terms
- Markov random field
- Ising model
- Categorical variable
- Marginal probability
- energy function
- partition function
- pairwise potential
- graph cut
- MAP inference
Learn next
Enumerate the two-node example and verify normalization. Add a local preference at one node to see how evidence competes with agreement. Then distinguish energy minimization from probability estimation before studying approximate inference on larger graphs.
More in Probabilistic machine learning
- Posterior predictive check - A posterior predictive check compares observed data with replicated data generated from a fitted Bayesian model.
- Posterior predictive p-value - A posterior predictive p-value is a tail probability comparing a chosen feature of replicated data from a fitted Bayesian model with the corresponding...
- Potential scale reduction factor - The potential scale reduction factor, commonly associated with R-hat, is a family of diagnostics for comparing multiple Markov chain Monte Carlo runs.
- Preconditioned stochastic gradient descent - Preconditioned stochastic gradient descent modifies a stochastic gradient with a matrix before using it to update model parameters.
- Probabilistic circuit - A probabilistic circuit represents a probability distribution through a computational graph of simple functions, weighted sums, and products.
- Probabilistic inference by automatic differentiation - Probabilistic inference by automatic differentiation uses derivatives of a suitable probability computation to recover quantities such as marginal...
Sources
Boykov, Veksler and Zabih, Fast Approximate Energy Minimization via Graph Cuts, original 2001 paper, energy formulation and Potts interaction: https://www2.cs.sfu.ca/CourseCentral/821/li/material/source/Boykov01.pdf Boykov, original publication and algorithm context: https://www.csd.uwo.ca/~yboykov/Abstracts/pami01-abs.shtml Boykov-Veksler-Zabih Section 7 Equation 18 and multi-label complexity discussion were independently consulted.
Previous: Potts diagram Next: Powell's dog leg method