Does every word have only one possible tag?

No. Some spellings have multiple established grammatical uses; the tagger uses context to choose among them.

Is tagging the same as parsing?

No. Tagging assigns categories to words, while parsing identifies structural relationships or groupings.

Can a tagger work on unfamiliar words?

Depending on its method, it can use morphology and context to predict a category, but unfamiliar terms remain important evaluation cases.

AI Glossary / Natural language and speech / Beginner

Part-of-speech tagging

Definition

Part-of-speech tagging is the task of assigning grammatical category labels to words in context, using a defined tagset. Labels can identify nouns, verbs, adjectives, and other categories. A tagger must often distinguish between several possible categories for the same written form. The output is a sequence of category assignments, rather than a complete grammatical or semantic analysis of the sentence.

What it means

Knowing a word's possible categories is different from choosing its category in a particular use. “Book” can be a noun or a verb, so a dictionary lookup that returns both possibilities does not finish the tagging task. Context helps identify which established use is intended.

The annotation unit also matters. A written token can contain more than one syntactic word in some languages or conventions. A project should therefore specify tokenization and word segmentation before interpreting the expected tag sequence. Punctuation and other non-word symbols may receive their own tags under the selected scheme.

How it works

A pipeline identifies the units to label and applies a tagging model or rules. Evidence can include the word form, affixes, nearby words, and broader context. Sequence models can also account for relationships among neighboring labels, rather than choosing every label without considering the others.

Training commonly uses text with reference annotations. The tagger learns how observed evidence relates to the scheme's categories and then predicts tags for unseen text. Different methods include probabilistic sequence models and neural models, but they must still follow the same declared output definitions to be compared meaningfully.

Evaluation aligns predictions with reference labels and measures agreements. Overall accuracy is useful but incomplete: common easy cases can dominate it, while rare ambiguous categories cause important downstream errors. Inspect confusion patterns, unfamiliar vocabulary, and representative document genres. A correct category label does not automatically establish a correct lemma, dependency relation, or word sense.

Three practical examples

1. Lemmatization support: A pipeline uses grammatical category information to help choose the appropriate base form for an ambiguous word.

2. Terminology extraction: A document tool considers noun and adjective sequences as candidate technical expressions, with additional checks before accepting them as terms.

3. Parser preparation: A language system uses predicted categories as evidence for subsequent structural analysis, while tracking whether tagging mistakes propagate into that analysis.

Think of it this way

Think of tagging as attaching grammatical labels to the components of a sentence before examining how they connect. The labels help organize the next stage, but attaching them correctly is itself a contextual task. The analogy should not suggest that categories are visible physical properties or that the labels provide a complete assembly diagram or interpretation.

Worked example

Consider the original sentence “They book rooms,” ignoring punctuation. Under a simple broad-category reference, the labels are PRON, VERB, and NOUN. A toy tagger predicts PRON, NOUN, and NOUN because it treats “book” as a noun regardless of context.

Two of the three predictions match, giving token accuracy 2/3, or about 66.7%, on this tiny example. For the VERB category, the prediction has no true positives and one false negative. The single NOUN prediction at “book” is a false positive for NOUN, while “rooms” supplies a true positive for that category.

Now consider “The book arrived,” whose reference categories are DET, NOUN, and VERB. The same spelling “book” correctly receives NOUN here. A fixed word-to-tag lookup cannot handle both uses correctly if it must always choose one category for that spelling. These hand-designed cases illustrate contextual disambiguation and error counting; they are not a benchmark of a real tagger.

Illustrative case study

Fictional case: Ash Catalogues builds a terminology extractor for product documentation. Its team discovers that action descriptions are sometimes proposed as product terms because an ambiguous verb is tagged as a noun.

Reviewers collect representative examples and distinguish tagging errors from mistakes in the extraction rules. They evaluate on separately annotated documents and retain difficult cases for discussion under the chosen guidelines. The scenario demonstrates a diagnostic workflow without claiming a measured improvement or an actual deployment.

Common misconceptions and mistakes

Do not confuse a tagger's output with a dictionary's list of possible categories. Tagging assigns categories to occurrences under a scheme. Nor should a high overall accuracy be assumed to mean that every category or document type is equally well handled.

Comparisons can also be misleading when one system uses broad categories and another uses fine-grained tags. Tokenization mismatches must be resolved before counting label agreements. Finally, reference annotations can contain errors or difficult convention-dependent cases; review them carefully instead of assuming either the model or the reference is always correct.

Business use

Part-of-speech tagging can support language workflows where grammatical categories help identify useful patterns. Its value depends on the downstream task, not simply on adding another linguistic stage to a pipeline.

Businesses should test the actual text they handle, including abbreviations, fragments, spelling variation, and specialist vocabulary. Track errors that change search results or extracted records, and document the tagset version and segmentation conventions. When updating a model, compare both tagging behavior and the outputs users rely on so an apparent local gain does not conceal a new downstream problem.

Check your understanding

Why is labeling “book” as NOUN in every sentence inadequate for the worked examples?

Answer: “Book” is a verb in “They book rooms” and a noun in “The book arrived. The same spelling requires different assignments in those contexts.

Related terms

Learn next

Tag several sentences containing one ambiguous word, then construct a small confusion table. Separate errors caused by segmentation, category ambiguity, and annotation conventions before comparing tagging methods.

Sources

Sources: Jurafsky and Martin, Speech and Language Processing, Chapter 18, sequence labeling and grammatical tagging: https://web.stanford.edu/~jurafsky/slp3/18.pdf . Universal Dependencies morphology, word segmentation, and contextual category distinctions: https://universaldependencies.org/u/overview/morphology.html . Universal category inventory: https://universaldependencies.org/u/pos/ . Relevant primary passages were read. Toy predictions, counts, and the fictional case are original; no runtime experiment or general accuracy claim is presented.

Previous: Part of speech Next: Partial Area Under the ROC Curve