AI Glossary / Computer vision and multimodal AI / Intermediate

Open set recognition

Definition

Open set recognition is a recognition setting in which a system must identify supported classes while allowing inputs from previously unseen classes to be rejected as unknown. It addresses the fact that the world can contain more categories than those represented during training.

What it means

A classifier trained to distinguish mugs, bottles and plates may later receive a photograph of a stapler. In a closed-set task, the available answers may force it to choose one of the three labels. An open-set system needs an additional possibility: none of the supported categories is sufficiently justified.

Unknown means unknown relative to the system's supported classes, not unknown to humanity. It also does not name the new object. Rejecting a stapler as unknown is different from successfully identifying it as a stapler.

This distinction separates rejection from category discovery and learning. An open-world recognition workflow can go further by reviewing unfamiliar examples and adding new categories over time. The ability to reject alone does not provide that complete workflow.

How it works

A system represents the input, evaluates its relationship to known classes and applies a rule for accepting a known label or rejecting the input. Methods differ: they may use feature distances, bounded acceptance regions or models of how known-class scores behave.

A threshold on an ordinary classifier's maximum score is a possible baseline, but unsupported inputs can still receive high scores. The important question is how the complete rule performs when actual unknown classes are introduced.

Evaluation therefore includes known-class examples and held-out categories treated as unknown. It measures both successful recognition of knowns and rejection of unknowns, including the trade-off between rejecting too much and accepting too much.

A finite test cannot represent every future unknown. Report which categories and operating conditions were tested. Broader claims require stronger evidence than a single convenient set of obviously different images.

Three practical examples

A parts-recognition station sends unsupported component types to manual review instead of assigning the closest familiar label.
A wildlife-image workflow distinguishes its supported species while allowing other animals to remain unclassified.
A document-routing system flags an unfamiliar document type rather than forcing it into one of the established queues.

Think of it this way

Think of a shop assistant who knows a catalog and can say, “This does not look like one of our listed products.” That answer can prevent an incorrect stock assignment without identifying the item precisely.

The analogy has limits. A model's rejection is a decision rule operating on evidence, not self-awareness or a comprehensive understanding of everything it does not know.

Worked example

Evaluating knowns and unknowns separately

Consider an invented test containing 200 examples from supported classes and 100 examples from unsupported classes. For the known examples, suppose the system assigns 170 correct labels, assigns 10 wrong known labels and rejects 20 as unknown. For the unknown examples, it rejects 85 and incorrectly accepts 15 as known.

The known-class correct-recognition rate is 170/200 = 85%. The unknown-rejection rate is 85/100 = 85%. Unknown false acceptance is 15/100 = 15%. Among the 195 accepted examples, only 170 have correct labels, so accepted-label accuracy is approximately 87.2%.

Now imagine raising the rejection threshold. Unknown false acceptance may decrease, but more genuine known examples may also be rejected. That trade-off affects review workload and the cost of mistaken automatic decisions.

Reporting only the accuracy on accepted known examples would hide both unknown acceptance and rejected knowns. The useful report keeps the categories of outcomes visible. These are illustrative rates, not algorithm benchmarks.

Illustrative case study

Illustrative case study - fictional

A fictional repair center trains a visual tool on six connector types. Staff later encounter a seventh type that resembles one of the originals. The tool's ordinary classifier gives it a confident familiar label, creating a risk of incorrect routing.

The team introduces an open-set evaluation using additional unsupported connectors, including visually similar ones. It compares acceptance rules and decides which rejected items require staff review. Repeated unfamiliar examples are collected for a separate, authorized update process.

The center does not treat each rejected item as a newly discovered class: some are simply blurred images of known connectors. The case illustrates the distinction between rejection, investigation and learning, without a measured outcome.

Common misconceptions and mistakes

A high confidence score from a closed-set classifier is not proof that the input belongs to a known class.
An unknown output does not identify the new category or establish that all rejected examples share one category.
Rejecting difficult known examples is not the same error as accepting unknowns. Both need measurement.
An “other” class trained on a few examples does not exhaust all possible future categories.
Open-set recognition and out-of-distribution detection overlap, but distribution changes can also occur within known classes, such as a new camera style.

Business use

Businesses benefit from making unsupported inputs visible when automated labels drive routing, retrieval or record updates. The value depends on what happens after rejection: a useful system needs a clear review path and a way to track repeated failure patterns.

Choose thresholds using representative known and unknown validation data, keeping a separate test set for evaluation. Estimate the resulting review volume as well as error rates. When the supported catalog changes, reassess both recognition and rejection behavior rather than assuming the earlier threshold remains suitable.

Check your understanding

A system rejects a new connector as unknown. Has it demonstrated that it can recognize the connector's exact type?

Answer: No. It has declined the supported labels. Naming or learning the new type requires additional evidence and processing.

Related terms

Novelty detection
Zero-shot learning
closed-set classification
out-of-distribution detection
open-world recognition
rejection threshold

Learn next

Design a small evaluation with supported classes and held-out classes, including some visually similar unknowns. Record correct known labels, wrong known labels, rejected knowns and accepted unknowns separately. Examine how each count changes when the acceptance rule changes.

Sources

Sources: Scheirer and colleagues, Toward Open Set Recognition: https://www.wjscheirer.com/papers/wjs_tpami2013_openset.pdf ; Bendale and Boult, Towards Open Set Deep Networks: https://arxiv.org/abs/1511.06233 ; Bendale and Boult, Towards Open World Recognition: https://arxiv.org/abs/1412.5687 . Relevant original-paper text and the latter research abstracts were reviewed; no universal rejection guarantee is asserted.

Previous: Open-loop Next: Open-source artificial intelligence