Is the model a neural network?

Not inherently. It is a probabilistic interaction model, although it has mathematical connections with energy-based neural models.

Can all interactions be positive?

Yes. Other models permit negative or mixed interactions, which can create competing preferences.

Why is the partition function difficult?

It sums weights across all configurations. There are 2 to the power N configurations for N binary variables, although special structures can permit more efficient calculations.

AI Glossary / Probabilistic machine learning / Intermediate

Ising model

Definition

The Ising model is a probability model for interacting binary variables, traditionally represented as spins taking values −1 or +1. Pairwise interaction terms favor agreement or disagreement between connected variables, while individual field terms favor particular values. It originated in statistical physics and also informs probabilistic machine learning.

What it means

Independent binary variables cannot describe a situation in which neighboring outcomes tend to occur together. The Ising model adds a simple way to express those dependencies.

In its physical interpretation, spins interact on a lattice and respond to an external magnetic field. In a statistical application, the same mathematical structure can represent binary labels on a graph. The graph need not literally describe magnetic material.

An Ising model specifies probabilities for entire configurations, not just a separate probability for each variable. This joint view is powerful, but it creates a computational challenge: the number of possible configurations doubles whenever another binary variable is added. Choosing a configuration, sampling configurations and learning interaction strengths are different tasks.

How it works

One common convention assigns a configuration a score equal to the sum of interaction terms J_ij x_i x_j over edges, plus field terms h_i x_i. Its probability is proportional to the exponential of this score.

Positive J_ij favors equal signs because their product is +1; negative J_ij favors opposite signs. Positive h_i favors x_i = +1. A partition function sums the unnormalized weights over all configurations and makes probabilities add to one.

Physics often writes the distribution as exp(−energy/temperature), with suitable units. Statistical notation may absorb temperature into the coefficients. Check the sign convention before comparing formulas. Conditional updates such as Gibbs sampling can avoid explicitly computing the global partition function, although efficient mixing is not guaranteed.

Three practical examples

An image model encourages neighboring binary pixel labels to agree while allowing observations to provide local evidence.
A researcher uses an Ising graph to study associations among binary measurements, without interpreting every edge as a causal effect.
A simulation project uses a small Ising system to compare exact enumeration with approximate sampling methods.

Think of it this way

Think of a row of switches joined by preferences: some connections reward matching settings, while others reward opposite settings. The whole arrangement receives a score from all those preferences together.

The switches are an analogy for random variables, not decision-making agents. The model does not imply that a switch physically changes its neighbor, and conflicting connections may prevent every local preference from being satisfied simultaneously.

Worked example

Calculating a complete two-spin distribution

Take two spins with no individual fields and a single interaction J = log 2. Under the positive-score convention, the unnormalized weight is exp(J x1 x2).

The configurations (+1, +1) and (−1, −1) each have weight exp(log 2) = 2. The configurations (+1, −1) and (−1, +1) each have weight exp(−log 2) = 1/2. The partition function is therefore 2 + 2 + 1/2 + 1/2 = 5.

The two agreeing configurations each have probability 0.4, and the two disagreeing configurations each have probability 0.1. Agreement has total probability 0.8.

Each individual spin is still equally likely to be +1 or −1. Marginal balance therefore does not imply independence. Given that the second spin is +1, the first is +1 with probability 0.4/(0.4 + 0.1) = 0.8. Their expected product is 0.8 − 0.2 = 0.6, which is also their correlation here because both means are zero and both variances are one.

Illustrative case study

Fictional case: smoothing a binary inspection map

A fictional engineering team labels each location in a simple inspection grid as one of two material conditions. Isolated noisy observations make the initial map unstable.

The team experiments with an Ising-style neighborhood prior combined with an observation model. It checks whether smoother labels preserve real narrow boundaries or erase them. The coupling strength is selected using separate validation examples with known labels.

The report distinguishes the prior assumption that nearby labels often agree from evidence that a particular location has a particular condition. A visually smoother map is not automatically a more accurate map. No accuracy gain is presumed in this fictional example.

Common misconceptions and mistakes

An Ising model is not limited to two variables. Each variable is binary, but the system can contain many variables.

Strong positive interactions encourage agreement; they do not make all variables deterministically identical at finite parameter values. Individual fields and competing interactions also affect the result.

A graph edge expresses a direct interaction in the specified model, not automatically a real-world causal relationship. Conversely, variables without a direct edge can still be marginally associated through other variables. The familiar physical phase-transition results depend on lattice, dimension and limiting assumptions; they should not be applied indiscriminately to every small statistical graph.

Business use

This model is useful when binary dependencies are central and a graph structure is defensible. It can provide an interpretable baseline or a component inside a larger probabilistic system.

Document variable coding, sign conventions, edge construction and the role of local evidence. Use exact enumeration on tiny examples to validate inference code before scaling. For larger models, report approximation and sampling diagnostics alongside predictions. Check whether pairwise binary interactions are adequate, since richer categories, higher-order relationships or changing conditions may require a different model.

Check your understanding

If J becomes zero in the worked two-spin model and the fields remain zero, what is the agreement probability?

Answer: One half. All four configurations have equal probability 1/4, and two of them agree.

Related terms

Learn next

Enumerate a three-spin chain, then compare marginal and conditional probabilities. Study Gibbs updates and inspect how strong interactions can slow movement between high-probability regions.

Sources

Primary teaching sources: Cosma Shalizi, Markov Random Fields: https://stat.cmu.edu/~cshalizi/dst/20/lectures/22/lecture-22.html

David Tong, Statistical Physics, section 5.2: https://www.damtp.cam.ac.uk/user/tong/statphys/five.pdf

The two-spin probabilities and inspection-map case are original teaching examples.

Previous: Ishikawa diagram Next: Isolates