Can NER find names it has never seen?

Some models generalize from context and form, but accuracy on unfamiliar names must be tested.

Does NER always include dates and money?

No. Those categories are common in some schemes but absent from others. The schema determines coverage.

Can entities overlap or nest?

Some tasks permit nested mentions; others use flat, nonoverlapping spans. The model and evaluation must support the chosen policy.

AI Glossary / Natural language and speech / Beginner

Named entity recognition

Definition

Named entity recognition, or NER, identifies spans of text that belong to specified entity categories and assigns a type to each span. Common categories include people, organizations, and locations; particular applications may also cover dates, amounts, products, or other domain-specific expressions.

What it means

NER turns part of an unstructured sentence into a labeled observation. From a message about a meeting, it may identify a person's name and a company name. It does not automatically determine the person's unique identity, their relationship with the company, or whether the statement is true.

The category inventory is part of the task definition. One dataset may separate geopolitical entities from other locations; another may combine them. A scientific application may need proteins or materials that a general-purpose news model was never trained to recognize.

The system also needs boundaries. Recognizing “Harbor” inside “Harbor Analytics” may be incomplete if the intended organization name includes both words. A correct type attached to the wrong span is still an error under strict evaluation.

How it works

An NER model can assign labels to tokens, predict spans directly, or generate structured outputs. In a common BIO scheme, B marks the beginning of a mention, I marks its continuation, and O marks text outside the target categories. Entity types are attached to the B and I labels.

Training examples show the intended boundaries and types. Context helps distinguish a name from an ordinary word and resolve names that can refer to different kinds of things. Capital letters and name dictionaries are useful clues in some settings, but they are not sufficient rules across languages and formats.

The output should retain offsets into the original text. Later stages can perform entity linking, coreference, or relationship extraction using those mentions. These additional tasks must not be silently attributed to NER alone.

Three practical examples

Document search: index recognized company and product names so users can filter relevant passages.
Support processing: identify a named device model and an order-date expression using an application-specific schema.
Review preparation: highlight possible personal names for a human reviewer, while recognizing that NER alone does not guarantee complete removal of identifying information.

Think of it this way

Think of NER as placing labeled bookmarks over relevant names in a document. The bookmark identifies a span and its category; it does not supply the complete biography or business record behind it. The analogy is limited because models can miss spans, choose wrong boundaries, or misclassify an unfamiliar name.

Worked example

Use the fictional sentence “Mira Shah joined Harbor Analytics in Lisbon.” With categories PERSON, ORGANIZATION, and LOCATION, the intended spans are “Mira Shah,” “Harbor Analytics,” and “Lisbon.” A BIO annotation starts each span with B-PER, B-ORG, or B-LOC; “Shah” and “Analytics” receive their respective I labels.

Suppose a model predicts “Mira Shah” as PERSON, “Harbor” as ORGANIZATION, and “Lisbon” as LOCATION. Under exact-span-and-type scoring, two of its three predictions are correct. Precision is 2/3 and recall is 2/3, so F1 is approximately 66.7%.

A token-level score would answer a different question and could appear higher. The NER result also does not establish which Mira Shah is meant or assign a verified registration number to Harbor Analytics. Those require additional evidence and linking steps.

Illustrative case study

Fictional case: Amber Index uses a news-trained NER model for technical service logs. It recognizes staff names but misses many equipment model names. The team initially interprets the empty product field as evidence that no product was mentioned.

Reviewers discover the schema mismatch and annotate relevant product examples with a clear boundary policy. They evaluate the product category separately and preserve uncertain cases for review. This is an illustrative adaptation process, not a claim of measured performance or complete recognition.

Common misconceptions and mistakes

NER is not the same as entity linking: labeling a name as a person does not resolve it to a unique database record. It is not full coreference resolution, which can connect names with descriptions and pronouns.

Do not assume all names begin with capital letters or that every capitalized word is a name. Do not infer missing facts from a type label. Also avoid comparing scores from incompatible schemas, nested-span policies, or boundary conventions as if they measured exactly the same task.

Business use

Define the categories needed by the workflow before choosing or evaluating a model. Provide annotation examples that cover abbreviations, compound names, unfamiliar products, and ambiguous uses. Evaluate precision, recall, and F1 by category as well as overall.

Keep test documents representative of actual input, including layout and transcription errors where relevant. Store the source span and model version so a reviewer can trace extracted fields. If a downstream process treats “no entity detected” as an important decision, test missed entities explicitly rather than assuming an empty result means absence.

Check your understanding

A system labels “Harbor Analytics” as ORGANIZATION. Does that prove it has linked the text to the correct registered business?

Answer: No. NER supplies the span and type. Identifying a particular real-world record is a separate linking or verification task.

Related terms

Learn next

annotate a short document with a fixed category list and compare both boundaries and types with another reader. Then add unique entity identifiers in a separate step. This makes the distinction between recognition and linking visible before introducing more complex models.

Sources

Sources: Tjong Kim Sang and De Meulder (2003), Introduction to the CoNLL-2003 Shared Task: Language-Independent Named Entity Recognition, task schema and exact-match evaluation: https://aclanthology.org/W03-0419.pdf . Jurafsky and Martin, Speech and Language Processing, Chapter 18, NER and BIO labeling: https://web.stanford.edu/~jurafsky/slp3/18.pdf . The sentence, scores, and Amber Index case are original illustrations.

Previous: Name - value pair Next: Named graph