Is it the same as a hidden Markov model?

No. A CRF models labels conditional on the input, whereas a conventional HMM specifies a joint generative model of observations and hidden states.

Does linear chain mean labels are independent?

No. Neighboring label interactions are precisely what the chain represents.

Must it enumerate every sequence?

No. Suitable dynamic-programming algorithms exploit the chain structure for exact normalization and decoding in the standard finite-label setting.

AI Glossary / Natural language and speech / Advanced

Linear chain CRF

Definition

A linear chain CRF is a conditional random field that assigns probabilities to sequences of labels using a chain of dependencies between neighboring label positions. Given an observed input, such as a sentence, it evaluates complete candidate label sequences. This allows the preferred label at one position to depend on how it fits with labels nearby, rather than treating every position independently.

What it means

CRF stands for conditional random field. Conditional means that the model represents probabilities of outputs given the input. It does not have to model how the input itself was generated. Linear chain describes the arrangement of label dependencies; it does not mean that the input must be a straight line of numbers or that every component of a larger system must be linear.

For example, labels can identify the grammatical category of each word or mark parts of a named entity. The chain provides a structured way to combine evidence about individual positions with preferences about adjacent labels.

How it works

For a fixed input, local factors assign compatibility values to candidate labels and neighboring label pairs. Multiplying the factors gives an unnormalized weight for a complete sequence. Dividing by the sum of all candidate-sequence weights, called the partition function, produces a probability distribution.

In a standard log-linear formulation, learned feature weights determine these factors. Features can use contextual properties of the observed input; the chain restriction concerns interactions among output labels. Training adjusts parameters using labeled sequences, commonly with a regularized conditional likelihood objective.

Dynamic programming avoids listing every full sequence explicitly. Forward computations obtain the normalization quantity, forward-backward calculations obtain marginal probabilities, and Viterbi decoding finds a highest-probability complete sequence. These solve related but distinct problems: selecting the most probable label separately at each position is not generally the same as selecting the most probable complete sequence.

Three practical examples

1. Named-entity labeling: A document system considers neighboring entity tags while identifying spans that may name organizations, people, or locations.

2. Grammatical tagging: A language pipeline combines evidence about each word with patterns among adjacent grammatical categories.

3. Structured record extraction: A team labels ordered text segments as fields such as identifier, description, or other text, after defining a suitable annotation scheme for its documents.

Think of it this way

Think of choosing a row of tiles where each tile must suit its position and fit sensibly beside its neighbors. The best row need not contain the individually preferred tile at every location. The analogy captures compatibility, but the model uses numerical factors learned or specified for a task; it does not possess an independent understanding of what a sensible document should mean.

Worked example

Consider an original two-position example with labels A and B. At position one, unary factors are A=3 and B=1. At position two, they are A=1 and B=2. Give a neighboring pair factor of 3 when the labels match and 1 when they differ. Start and end factors are 1.

The four sequence weights are AA: 3×1×3=9; AB: 3×2×1=6; BA: 1×1×1=1; and BB: 1×2×3=6. Their sum is 22, so the probabilities are respectively 9/22, 6/22, 1/22, and 6/22. AA is the most probable complete sequence, with probability about 0.409.

Choosing independently from the unary factors would produce AB, since A is preferred at the first position and B at the second. The pair factor changes the joint choice. The factors themselves are not probabilities and need not sum to one. These values were chosen for teaching and independently enumerated; they are not parameters learned from real text.

Illustrative case study

Fictional case: Cedar Forms labels text in incoming service records. An independent classifier sometimes proposes inconsistent neighboring field labels, so the team evaluates a chain model using the same annotation definitions and held-out documents.

Reviewers examine complete field spans as well as individual label accuracy. They check unusual form layouts separately, because a strong preference for familiar transitions can also reinforce a wrong interpretation. The comparison is a proposed evaluation workflow, not a claim that the CRF improved a real organization's performance.

Common misconceptions and mistakes

Do not confuse the factor scores with normalized sequence probabilities. Normalization is over complete candidate outputs for the given input, not merely over the labels at each position.

A chain model also does not imply that input features can inspect only one word. Contextual observations can inform local factors while the label interactions retain chain structure. Finally, favorable transition scores are not automatically hard constraints: if certain tag sequences must be impossible, the implementation needs an explicit and consistently applied constraint policy.

Business use

A linear chain CRF is relevant when a business problem involves ordered labels whose neighboring relationships matter. Before adopting it, define the output scheme, obtain consistent annotations, and compare it with an appropriate baseline on representative data.

Measure errors at the level the workflow actually consumes. A nearly correct set of token labels may still produce a wrong address or incomplete organization name. Also distinguish confidence in the whole sequence from confidence in one label. Neither quantity is automatically calibrated for an operational decision without evaluation.

Check your understanding

In the worked example, why does AA beat the independently preferred AB?

Answer: Its matching-label pair factor raises its full sequence weight to 9, exceeding AB's weight of 6. Both share the same normalization denominator.

Related terms

Learn next

Work through a three-position Viterbi table, then compute a marginal probability for one position. Compare the information those answers provide before studying how conditional likelihood training learns factor weights.

Marginal probability

Sources

Sources: Sutton and McCallum, An Introduction to Conditional Random Fields, Sections 2.3 and 4.1: https://homepages.inf.ed.ac.uk/csutton/publications/crftut-fnt.pdf . Michael Collins, Log-Linear Models, MEMMs, and CRFs, conditional normalization and parameter estimation: https://www.cs.columbia.edu/~mcollins/crf.pdf . Relevant primary mathematical passages were read. The finite two-label example is original and independently checked; it illustrates inference, not model training or a benchmark.

Previous: Linear belief function Next: Linear classifier