Is a lexicon just a dictionary?

A dictionary is one familiar kind of lexical resource, but computational lexicons can store task-specific fields and links rather than reader-oriented explanations.

Can a lexicon include phrases?

Yes. Multiword expressions can be entries when the resource treats them as lexical units.

Does a lexicon understand context?

It supplies information that contextual processing can use; lookup alone does not resolve every ambiguity or interpret a complete statement.

AI Glossary / Natural language and speech / Beginner

Lexicon

Definition

A lexicon is a collection of words or other lexical units together with information about them. In natural language processing, it is a resource that a system can consult for properties such as grammatical category, meaning, pronunciation, or permitted complements. A lexicon may contain single words and multiword expressions. Its contents and level of detail depend on the task it supports.

What it means

A simple lexicon might associate product terms with approved spellings. A richer one could distinguish several senses of a word, identify their grammatical behavior, and link them to related concepts. The organizing unit is often a lemma, or conventional dictionary form, but a resource may also record inflected forms and their relationships to that lemma.

The word does not imply one universal database containing everything speakers know. Different languages, domains, and applications need different information. A pronunciation lexicon and a sentiment lexicon can contain overlapping words while recording very different properties. Their usefulness depends on the coverage and distinctions needed for a particular task.

How it works

A team first defines what counts as an entry and which fields are required. Entries might be organized by lemma and part of speech, by word sense, or by another explicit scheme. The team then adds information from an appropriate source and checks consistency, spelling variants, and ambiguous cases.

During processing, a system looks up a word or expression and retrieves candidate information. Context still matters: the same written form can be a noun in one sentence and a verb in another. Morphological analysis may connect an inflected form to a stored entry, while contextual analysis helps select among candidate senses.

Some lexicons also describe grammatical preferences. A verb can permit certain complements and exclude others; a mere label saying “verb” does not capture all of that behavior. Maintenance is therefore more than adding strings. It includes defining distinctions, documenting provenance, handling missing entries, and updating domain-specific meanings without silently changing established records.

Three practical examples

1. Speech production: A pronunciation lexicon provides candidate pronunciations for names and specialized vocabulary, with a separate policy for unfamiliar terms.

2. Document tagging: A domain lexicon helps recognize equipment names and multiword expressions in maintenance reports, while context checks prevent ordinary uses from being misclassified.

3. Semantic search: A lexical resource connects word senses with related meanings so a search system can consider appropriate alternatives rather than matching spelling alone.

Think of it this way

Think of a cabinet of language reference cards. Each card contains selected information about an expression, and some cards point to related cards. This captures the lookup function, but the cabinet cannot interpret every sentence by itself. Choosing the relevant card and understanding the surrounding statement still require contextual processing, and the cabinet may have gaps.

Worked example

Consider an original toy lexicon that uses lemma-plus-category as its entry key. It contains four entries: “record” as a noun, “record” as a verb, “inspection” as a noun, and “carry out” as a multiword verb expression.

There are four entries but only three distinct written headwords. The first two entries share a spelling while representing different grammatical uses. The final entry contains two space-separated words, yet the resource treats it as one lexical expression. Counting spaces would therefore not reproduce the entry count.

In “They record an inspection,” the system should select the verbal entry for “record.” In “The record describes an inspection,” it should select the nominal entry. A lookup that returns both candidates has retrieved possibilities; it has not completed disambiguation. These invented examples demonstrate resource design, not measured performance on a corpus.

Illustrative case study

Fictional case: Larch Maintenance creates a lexicon for an assistant that reads workshop notes. Its first version stores equipment names, common abbreviations, and approved expansions. Reviewers notice that “press” can identify equipment or describe an action, so they add distinct entries and examples instead of forcing one interpretation everywhere.

The team records unresolved uses for review and evaluates recognition on separately annotated notes. It also distinguishes missing terminology from incorrect sense selection. The case illustrates a maintenance workflow; no real deployment, accuracy gain, or commercial result is claimed.

Common misconceptions and mistakes

A lexicon is not the same as a corpus: a corpus contains language samples, while a lexicon organizes lexical information. Nor is it necessarily identical to a model's token vocabulary, whose units can include word fragments.

Do not assume that every entry has exactly one word, one meaning, or one pronunciation. Entry conventions must be stated before counts are compared. A missing entry does not show that an expression is invalid, and finding an entry does not prove that its listed sense is appropriate in the current sentence.

Business use

Businesses can use lexicons to make terminology handling more consistent and inspectable. A useful resource can support search, extraction, translation preparation, or speech applications, especially where specialist vocabulary matters.

Before investing, define the target task and test whether lexical coverage is a significant source of errors. Track ambiguous matches, missing entries, and maintenance effort separately. Assign responsibility for updates and retain evidence for additions so the resource remains understandable as products, language, and document practices change.

Check your understanding

A resource stores separate noun and verb entries for “record.” Does that mean it contains two different written headwords?

Answer: No. It contains two entries sharing one written headword under its stated entry convention.

Related terms

Learn next

Compare a lexicon with a small corpus. Identify which information belongs in an entry and which information depends on a particular occurrence. Then explore word-sense disambiguation to understand why lookup and interpretation are separate steps.

Sources

Sources: NLTK Book, Chapter 2, lexical resources and entries: https://www.nltk.org/book/ch02.html . NLTK Book, Chapter 8, lexical properties and verb complements: https://www.nltk.org/book/ch08.html . Princeton WordNet overview, word senses and lexical relations: https://wordnet.princeton.edu/ . Relevant primary explanatory passages were read. The supplied AIMA glossary page did not expose a matching entry in retrieval; this article uses the broader NLP resource sense supported by these sources. Examples are original.

Previous: Lexicographic optimization Next: Library classification