AI Glossary / Natural language and speech / Advanced
Yarowsky algorithm
Definition
The Yarowsky algorithm is an iterative bootstrapping method for word-sense disambiguation. Starting from a small amount of sense-indicating seed evidence, it learns contextual rules and uses confident predictions to label additional examples.
What it means
The name commonly refers to David Yarowsky's 1995 method. It exploits two tendencies: nearby expressions can strongly indicate a word's sense, and repeated uses within one document often share a sense. These are useful regularities, not universal laws.
The target is disambiguation among established sense categories, not unrestricted discovery of new senses. A system distinguishing financial and river meanings of “bank” already has those categories in view.
Although the original paper calls the method unsupervised, its seed collocations connect examples to senses. In contemporary terminology, such a setup is often discussed as weakly supervised or semi-supervised bootstrapping. The important practical question is what information enters through the seeds, not which label sounds most autonomous.
How it works
Collect occurrences of an ambiguous word with their contexts. Seed cues label a small subset. A decision-list classifier learns contextual indicators, ranked by their strength, and applies the strongest relevant rule to an occurrence.
Confidently labeled examples enlarge the training material, allowing new contextual cues to be learned in later rounds. Discourse consistency can help filter or extend labels. Yarowsky explicitly treats that tendency as overridable by strong local evidence, rather than forcing every occurrence in a document to match.
The loop continues until a stopping condition is met. Predicted labels remain fallible, so evaluation must be separate from the examples used to expand the training set. Self-generated confidence is not an independent quality check.
Three practical examples
- A researcher reproduces a historical sense-tagging experiment to study how reliable seed phrases spread labels through a corpus.
- A terminology team tests whether a small collection of high-precision contextual cues can label additional examples for review.
- A student compares a decision list with a model that combines many features, observing how conflicting contextual evidence is handled.
Think of it this way
Think of sorting unlabeled workshop parts using a few clearly identified examples. Once one reliable feature is learned, it helps identify more parts, which may reveal further useful features.
The analogy is limited because linguistic senses are not always sharply separated physical categories. One mistaken starting example can also encourage a misleading rule. The process gains reach through reuse of evidence, not through automatic creation of ground truth.
Worked example
Following one bootstrapping round
Consider an invented collection of 100 occurrences of “bank”. The task distinguishes a financial institution from the land beside a river. Carefully selected seed contexts label six financial and four river examples, leaving ninety unlabeled.
A first classifier proposes labels for twenty additional examples above a chosen confidence threshold. The next training round therefore has thirty labeled examples: ten seeded and twenty predicted. The number thirty is a coverage count, not a claim of thirty correct labels.
A separate manual review finds that eighteen of the twenty additions are correct. Their observed precision is 18 / 20 = 90%. If the two wrong additions share a misleading contextual word, retraining can strengthen that mistake and spread it further.
The team should inspect the faulty cue and test subsequent rounds on held-out examples. This simplified walk-through demonstrates label expansion and error propagation; it does not reproduce the original paper's smoothing, rule scores or experimental results.
Illustrative case study
A fictional archive team wants to distinguish technical and everyday uses of “cell” in a specialist document collection. It starts with a few apparently clear seed expressions and runs a small bootstrapping experiment.
Review shows that a broad topic word appears around both senses. The team removes that weak cue, narrows the context relationship and retains uncertain cases for human inspection. It also checks documents that deliberately compare different meanings, where one-sense-per-discourse is unsuitable.
The archive measures accuracy on independently reviewed examples after each round. It does not use the growth of the automatically labeled pool as its success criterion. This is a fictional evaluation scenario, not a claim of performance for the original algorithm.
Common misconceptions and mistakes
- Treating the one-sense tendencies as exceptionless language rules.
- Assuming more automatically labeled examples necessarily mean better labels.
- Confusing disambiguation with inventing an unknown sense inventory.
- Hiding the supervision introduced by choosing sense-specific seeds.
- Interpreting confidence learned from a biased seed set as a calibrated probability of correctness.
- Copying a historical benchmark result into a promise about a new language, corpus or sense distinction.
Business use
The Yarowsky algorithm is valuable for understanding how limited initial knowledge can be expanded through contextual evidence. Its business relevance is often educational or methodological: it exposes both the appeal and risks of bootstrapping labels.
Before using a related workflow, define the intended senses and examine whether the corpus supports them. Choose seeds for precision and representativeness rather than convenience alone. Keep source documents and the rule responsible for each predicted label available for inspection.
Review rare senses and mixed-sense documents, because a dominant category can overwhelm the expansion process. Compare the effort saved with the cost of correcting propagated mistakes. If the labels will guide a consequential application, validate that downstream task rather than assuming a sense-tagging score translates directly into operational benefit.
People also ask
Does the algorithm need every training example labeled?
No. Its purpose is to expand from limited seed evidence using a larger unlabeled collection.
What is a collocation here?
It is a contextual relationship between words, including positional or grammatical relationships. It need not be an idiom.
Does a document always use an ambiguous word in one sense?
No. Comparisons, quotations and topic shifts can mix senses. The original method treats discourse consistency as evidence, not an absolute requirement.
Check your understanding
A bootstrapping system doubles its labeled pool, but no independently labeled test set is checked. What has improved for certain?
Answer: Only the number of assigned labels has increased. Accuracy, coverage of rare senses and downstream usefulness remain unestablished; errors may also have expanded.
Related terms
- Word sense disambiguation
- Self-training
- Decision lists
- Collocation (remote sensing)
- Weak supervision
- bootstrapping
- seed labels
- decision list
- collocation
- semantic drift
Learn next
Read self-training and decision lists. Design a tiny ambiguous-word experiment with two seed cues, then deliberately include a misleading cue and a mixed-sense document. Explain how independent review could detect error propagation.
More in Natural language and speech
- Word-sense induction - Word-sense induction is the automatic discovery of different uses or senses of a word from the contexts in which it occurs.
- Word shape - Definition: Word shape is a representation of a token's character pattern, such as uppercase letters, lowercase letters, digits and punctuation.
- Wordpiece - Definition: WordPiece is a subword tokenization method that represents text using a fixed vocabulary containing whole words and smaller word pieces.
- Zero anaphor - Definition: A zero anaphor is an unspoken or unwritten referring expression whose interpretation is supplied by linguistic context.
- Zero-width - Definition: In regular expressions, zero-width describes a match or assertion that consumes no characters.
- Zipf - Mandelbrot law - The Zipf - Mandelbrot law is a rank - frequency model that extends Zipf's law by adding a shift to rank.
Sources
David Yarowsky, Unsupervised Word Sense Disambiguation Rivaling Supervised Methods (1995): https://aclanthology.org/P95-1026.pdf. The primary paper describes seed collocations, decision-list learning and optional discourse-based label propagation, and distinguishes disambiguation from induction. The bank counts and archive case are original teaching examples, not reproduced experimental results.
Previous: Yale shooting problem Next: Yates analysis