Does a CRF require input features to be independent?

No. It can use overlapping and dependent input features while modeling a conditional distribution over outputs.

Are all CRFs linear chains?

No. Linear chains are a common special case; other graph structures support different relationships among output variables.

Does the best output need probability above 0.5?

No. It only needs a higher probability than every alternative. With many possible outputs, the largest probability can be much smaller than one-half.

AI Glossary / Graphs and symbolic AI / Intermediate

Conditional random field

Definition

A conditional random field, or CRF, is a probabilistic model for predicting related output variables given observed inputs. It represents dependencies among outputs through a graphical structure and defines a conditional distribution over complete output assignments. A familiar example is a sequence-labeling model that predicts the labels of neighboring words jointly rather than classifying every word independently.

What it means

The conditional distribution is usually written P(Y given X), where X is the observed input and Y is the output structure. The model need not separately describe how likely the input itself is.

In a linear-chain CRF, neighboring output labels interact along a sequence. Input features can provide evidence for individual labels or label transitions. A general CRF can use a different graph structure, so the term does not refer exclusively to a chain of words.

The purpose of modeling output dependencies is to assess compatible combinations. A label that looks strongest in isolation may belong to a less plausible overall sequence once relationships with nearby labels are considered.

How it works

The model assigns nonnegative compatibility factors to combinations of input features and output labels. In a common log-linear form, factors are exponentials of weighted feature scores. Multiplying the relevant factors gives an unnormalized score for a complete output assignment.

A normalization term sums those scores over all possible output assignments for the given input. Dividing by it produces a conditional probability distribution. This global normalization differs from separately normalizing each local next-label decision.

Training adjusts parameters using labeled data, often by maximizing a regularized conditional likelihood. Prediction may seek the single most probable full labeling or the marginal probability of a particular label at a particular position; these are different tasks.

For a linear chain, dynamic programming can perform important inference tasks exactly, including Viterbi-style decoding and forward-backward calculations. More general graph structures can make exact inference much harder, requiring different algorithms or approximations.

Three practical examples

1. A text-processing model identifies spans such as organization names while accounting for relationships between neighboring labels.

2. A document application labels successive lines as headings, body text, or other roles, using both line features and neighboring role compatibility.

3. An image model assigns related region labels using a graph that connects neighboring regions, with the inference method chosen for that graph structure.

Think of it this way

Think of a CRF as scoring a cast of roles together. Each participant has evidence favoring particular roles, but the compatibility of the whole arrangement also matters.

The analogy has limits: the model uses explicit mathematical factors and a defined label space. It does not possess a general understanding of roles, and a high-scoring arrangement can still be wrong when its learned features or assumptions are unsuitable.

Worked example

Consider two output positions, each labeled A or B. For one fixed input, the first position has unary factors A = 2 and B = 1. The second has A = 1 and B = 3. Give matching neighboring labels a pair factor of 4 and different labels a pair factor of 1.

The four sequence scores are AA: 2 times 1 times 4 = 8; AB: 2 times 3 times 1 = 6; BA: 1 times 1 times 1 = 1; and BB: 1 times 3 times 4 = 12.

The normalization term is 8 + 6 + 1 + 12 = 27. Therefore P(BB given this input) is 12/27, approximately 0.444. BB is the most probable full labeling, although its probability is below one-half because probability is distributed across four alternatives.

Choosing the strongest unary label separately would produce AB. The pair factor changes the preferred joint result. These invented factors demonstrate inference; they are not fitted parameters or a claim about a real language task.

Illustrative case study

Fictional case: Maple Documents experiments with labeling adjacent lines. Its team initially evaluates only separate line scores and overlooks the model's transition factors.

A reviewer constructs the two-position example above and shows why local choices need not maximize the full sequence score. The evaluation is revised to report both whole-sequence predictions and per-position results, with each measure clearly labeled.

The case illustrates a distinction in decoding and evaluation. It does not claim that stronger neighboring-label agreement is always appropriate or that using a CRF automatically improves document extraction.

Common misconceptions and mistakes

Do not describe every CRF as a hidden Markov model. A CRF directly models the conditional output distribution, while an HMM conventionally specifies a joint generative model of states and observations.

Do not confuse compatibility factors with probabilities before normalization. The worked example uses factors greater than one without violating probability rules.

Do not assume the most likely label at each position necessarily forms the most likely complete labeling. Also, global normalization does not guarantee calibrated confidence, and a graph connection does not establish a causal relationship.

Business use

For business teams, CRFs are relevant when outputs have a structure that independent predictions fail to capture. Examples include coherent spans, ordered document roles, and linked region labels.

The value of the approach depends on the labeling scheme, training examples, features, and inference cost. Review performance at the level that matters to the workflow: a correct individual token may not be enough if the required entity span is incomplete. Compare confidence estimates and error patterns on representative data rather than relying on the model family name.

Check your understanding

A complete labeling has unnormalized score 12 and all labeling scores sum to 27. Is its probability 12 or 12/27?

Answer: It is 12/27. Scores become probabilities only after division by the normalization term for that input.

Related terms

Learn next

Recalculate the toy example after changing the same-label factor to 1 and observe how the preferred labeling changes. Then compare whole-sequence decoding with marginal prediction and study the forward-backward algorithm for a linear chain.

Forward-backward algorithm

Sources

Sutton and McCallum, An Introduction to Conditional Random Fields: https://arxiv.org/pdf/1011.4088 . The authors' primary tutorial passages on conditional modeling, normalization, and inference were consulted. The factor calculation and fictional document case are original. Historical software listings are not used as current implementation recommendations.

Previous: Conditional probability table Next: Conditioning