Must every labeler use output-transition scores?

No. Contextual token classifiers can predict labels without an explicit structured output layer.

Can multiple words form one extracted item?

Yes. Schemes such as BIO represent a multiword span through several aligned labels.

Is a word label automatically a label for every subword piece?

No. Training and evaluation need a declared mapping policy; different valid implementations can handle continuation pieces differently.

AI Glossary / Natural language and speech / Advanced

Sequence labeling

Definition

Sequence labeling assigns a label to each unit in an ordered input sequence, producing an aligned label sequence of the same length at the chosen unit level. In NLP, the units are often words or tokens, and labels can represent parts of speech, entity boundaries, or other properties. The surrounding sequence can help determine each label.

What it means

A sentence-level classifier might label an entire message as positive. A sequence labeler instead assigns information to individual positions. For example, it can mark which words belong to a person's name and which belong to an organization's name.
One label per input unit does not mean each decision must be independent. A contextual model can use neighboring input, and a structured decoder can account for relationships between output labels. Conversely, a sequence-labeling task does not require one particular neural architecture. The defining feature is the alignment between the selected input units and their labels.

How it works

First define the units and annotation scheme. A training example pairs an input sequence with a corresponding label sequence. A model learns how input evidence supports labels and, in some designs, how labels fit together.
Hidden Markov models, conditional random fields, recurrent networks, and Transformer-based taggers are possible approaches. Structured decoding can choose the highest-scoring complete label sequence rather than taking the best local label at every position. Its benefit depends on the model and task; it is not automatically superior in every setting.
Subword tokenization requires an explicit alignment policy when reference labels belong to words. Special tokens and padding also need appropriate treatment. Evaluation should recover the intended word or span units rather than silently changing what is being scored.

Three practical examples

A part-of-speech tagger labels each word according to its use in context. The same spelling can receive different labels in different sentences.
A document tool uses boundary-and-type tags to identify company and person mentions. Several token labels can combine into one extracted entity span.
A semantic-role system labels tokens relative to a supplied predicate, marking argument spans and roles. A sentence with multiple predicates may require several aligned analyses of the same input.

Think of it this way

Think of placing a small card under every item in a row. Each card describes the item above it, but neighboring cards may need to form a coherent pattern. Counting the cards checks alignment. Checking each card's content and the pattern across the row checks the actual analysis.

Worked example

Take two abstract input positions and two possible labels, A and B. At both positions, supply local potentials of 0.6 for A and 0.4 for B. Independently choosing the larger local value produces AA.
Now add pair potentials: AA=0.1, AB=0.9, BA=0.8, and BB=0.2. Define a complete path's unnormalized score as its two local potentials multiplied by its pair potential. The four path scores are AA: 0.6 × 0.6 × 0.1 = 0.036; AB: 0.6 × 0.4 × 0.9 = 0.216; BA: 0.4 × 0.6 × 0.8 = 0.192; BB: 0.4 × 0.4 × 0.2 = 0.032. The best complete path is AB, not AA.
These are invented potentials, not a measured model's probabilities. Their sum is 0.476; normalizing gives AB a path probability of 0.216/0.476, approximately 45.4%, under this toy distribution. The most likely path need not exceed 50%.
For a separate alignment illustration, Mara works at Cedar Labs has five words. The BIO labels B-PER, O, O, B-ORG, I-ORG also number five and encode two mentions: Mara and Cedar Labs. The number of extracted spans need not equal the number of token labels.

Illustrative case study

Fictional team RecordMap builds a named-entity tagger for service notes. Its word-level annotations look correct, but a tokenizer divides some organization names into several subword pieces. A careless alignment step shifts later labels, producing misleading training examples.
The team traces labels back to original word identifiers and documents how continuation pieces, special tokens, and padding are handled. They inspect reconstructed mentions and score entity boundaries as well as individual tags. A separate test checks that long documents split into chunks do not lose or duplicate mentions at the joins.
This fictional case illustrates data alignment and evaluation, not a reported production improvement.

Common misconceptions and mistakes

Sequence labeling is different from assigning one class to an entire sequence and from freely generating an output of arbitrary length. Those tasks may use similar encoders but have different output structures.
Do not assume that aligned outputs imply independent predictions. Also avoid assuming that every possible tag sequence is meaningful: an inside tag may need a compatible earlier boundary under the chosen BIO convention. The scheme and decoding policy must be explicit.
High token accuracy can conceal poor span extraction when most tokens receive the outside label. Excluding padding is necessary, but it does not by itself ensure that word, subword, and entity-level scores are being compared fairly.

Business use

Sequence labeling can turn unstructured text into searchable annotations while preserving links to the source. Specify the unit, label inventory, boundary convention, and downstream use before evaluating a model. Review complete extracted spans and relevant error categories, not only a single aggregate score. Consistent alignment makes failures traceable and helps distinguish a model error from a preprocessing or annotation mismatch.

Check your understanding

Why does the toy structured model choose AB?

Answer: Its complete score, 0.216, exceeds all other path scores after pair potentials are included.

Why is AA chosen by the independent rule?

Answer: A has the larger local potential at both positions.

How many mentions do the five BIO labels encode?

Answer: Two, despite having one label for each of five words.

Related terms

Learn next

Create a small aligned tagging dataset and document its unit and boundary conventions. Compare independent local decoding with a simple structured decoder, then inspect both token-level decisions and reconstructed spans. Keep data alignment checks separate from model-quality measurements.

Sources

Jurafsky and Martin, Speech and Language Processing, Chapter 18, sequence labeling, BIO representation, HMMs, and CRFs: https://web.stanford.edu/~jurafsky/slp3/18.pdf Hugging Face Transformers, Token classification, word/subword label alignment and evaluation: https://huggingface.co/docs/transformers/tasks/token_classification The path potentials, BIO sentence, and RecordMap scenario are original illustrations.

Previous: Sequence form Next: Sequence model