Does class incremental learning require zero stored examples?

No. Some settings permit a bounded replay memory, while others prohibit storing old examples. The constraint must be stated.

Is it the same as recognizing never-seen classes?

No. It concerns learning new classes as training information arrives. Zero-shot recognition is a different capability, though systems may combine them.

Does increasing the number of outputs solve forgetting?

No. Adding labels creates capacity to express new predictions, but shared representations and decision boundaries can still change adversely.

AI Glossary / Probabilistic machine learning / Intermediate

Class incremental learning

Definition

Class incremental learning is a continual-learning setting in which a model learns additional classes over time while remaining able to distinguish the classes encountered earlier. At evaluation, it must choose among the relevant accumulated classes without being told which learning episode supplied the example. Retaining old knowledge while acquiring new distinctions is central to the task.

What it means

A classifier may begin by recognizing two kinds of objects and later receive examples of two more. Simply replacing its original training objective with the new two-class problem does not satisfy class incremental learning if it loses the ability to recognize the earlier classes.
The model must also compare old classes with new ones. Preserving a separate classifier for each episode is insufficient when the system is not given the episode identity at prediction time.
This differs from domain incremental learning, where the input conditions may change while the output problem remains the same, and from task incremental learning, where task identity is supplied under the evaluation protocol.

How it works

Define the sequence of class arrivals, the data available at each stage, and the permitted memory or model growth. These choices materially affect the problem's difficulty.
Update the representation and classifier to accommodate the new classes while preserving useful information about previous ones. Methods may retain selected examples for rehearsal, replay generated examples, constrain parameter changes, preserve previous outputs through distillation, or combine several mechanisms.
After each stage, evaluate on all classes learned so far using the actual prediction conditions. Do not provide a hidden episode label that narrows the choices unless that information is part of the intended setting.
Track both acquisition and retention. High accuracy on newly introduced classes can coexist with severe forgetting of older classes, and an overall score can hide that imbalance.

Three practical examples

An image classifier learns additional product categories as a catalog grows, while continuing to recognize earlier categories.
A support-routing system adds new issue classes but must still distinguish them from existing issue types.
A document classifier incorporates newly defined document categories without assuming that every incoming document belongs to the latest training batch.

Think of it this way

Think of a librarian learning new subject categories while retaining the ability to classify books into the old categories. The librarian must compare the full set of relevant categories, not merely remember which shelf was introduced most recently.
The analogy has limits: a model's shared parameters can change in ways that disrupt earlier decisions, and its retained examples may be tightly limited. Human memory should not be assumed to describe the mechanism of a learning algorithm.

Worked example

Consider an evaluation. After stage one, a classifier distinguishes classes A and B with 90 correct predictions out of 100 balanced test examples. Stage two introduces C and D.
After the update, evaluate without giving the stage identity. On the same 100 old-class examples, it gets 60 correct. On 100 balanced new-class examples, it gets 88 correct. The combined accuracy is (60 + 88) / 200 = 74%.
Reporting only the new-class result of 88% would conceal the fall in old-class accuracy from 90% to 60%, a decrease of thirty percentage points. The combined result also depends on the test-set weighting; these groups have equal size by construction.
Now suppose a test procedure tells the model whether to choose from {A,B} or {C,D}. That removes the need to distinguish across the two groups and changes the evaluation scenario. Its result should not be compared directly with the four-class result as though they measure the same task.

Illustrative case study

In a fictional project, Willow Retail expands an image-labeling system from two product families to four. The first update appears successful because the newest categories score well, but staff find that older products are increasingly assigned to new labels.
The team introduces a fixed evaluation protocol across all learned categories and records the amount of old training data retained by each method. It evaluates the effect of class order as well as final accuracy.
This invented case illustrates how incomplete evaluation can hide forgetting; it reports no measured improvement.

Common misconceptions and mistakes

Do not equate class incremental learning with ordinary minibatch training on a shuffled dataset containing every class from the outset.
Do not assume that remembering old class names means preserving the ability to recognize them.
Do not provide episode identity at test time while claiming the harder setting that requires choosing across old and new classes.
Do not compare methods without reporting differences in replay memory, access to old data, model size, and training budget.
Finally, historical experimental failures of a particular method are not a proof that one broad family of methods can never work in every class incremental setting.

Business use

Class incremental learning is relevant when an organization's label set expands and repeatedly training from the full historical dataset is impractical or outside the chosen workflow. The aim is useful adaptation with controlled loss of earlier capabilities.
Specify how new categories enter the system, what retained information is allowed, and how success is measured across the full label set. Compare with reasonable reference approaches, including retraining where feasible. The tradeoff includes memory, computation, update latency, and prediction quality; avoiding a full retraining cycle is not sufficient by itself.

Check your understanding

A model scores 95% when told which two-class episode an example belongs to, but it has never been evaluated over all six learned classes. Does that establish six-class incremental performance?

Answer: No. The episode information restricts the decision and can hide confusion between classes from different episodes. Evaluation must match the claimed setting.

Related terms

Continual Learning
catastrophic forgetting
task incremental learning
domain incremental learning
rehearsal
knowledge distillation
class imbalance

Learn next

create an evaluation table with one row per training stage and one column per class group. Use it to separate learning new categories from retaining old ones, and document the test-time information available to the model.

Sources

Primary source: Van de Ven and Tolias, Three Scenarios for Continual Learning, scenario definitions and method discussion: https://arxiv.org/pdf/1904.07734 The counts and fictional case are original illustrations. Historical results from that paper are not presented as universal limits on current methods.

Previous: Class-based n-gram Next: Class interval