Is it the same as continual learning?

It is a more specific situation combining sequential learning with scarce labeled examples for new additions.

How does it differ from generalized few-shot learning?

Generalized few-shot evaluation recognizes base and novel classes together; an incremental setting additionally emphasizes successive additions and retention across sessions.

Must every weight be updated?

No. Some methods mainly change prototypes or classifier parameters. The allowed update mechanism is part of the method.

AI Glossary / Prompting and human-AI interaction / Advanced

Incremental Few-shot Learning

Definition

Incremental few-shot learning is learning new categories or capabilities in successive stages from small labeled datasets while retaining useful performance on what was learned earlier. This entry focuses on the widely studied few-shot class-incremental setting, in which a classifier must recognize all classes encountered so far.

What it means

A business may already have a classifier for established products but receive only a handful of labeled examples of a newly introduced product. Adding that product should not require pretending the earlier products no longer exist.

The challenge combines limited evidence about new classes with preservation of old knowledge. Training aggressively on the new examples can overfit them or disturb the existing classifier. Refusing to change anything can leave the new classes poorly represented.

The word incremental refers to the sequence of additions. Few-shot refers to the small number of labeled examples available for each new class. Neither word guarantees a particular accuracy or means that the model started without substantial previous training.

How it works

A typical experiment begins with a base training stage containing many examples of established classes. Later sessions introduce disjoint new classes with a small support set per class. Evaluation after each session asks the model to choose among all classes seen to date.

Methods differ. A system may preserve a learned representation and add class prototypes, retain selected old examples, constrain parameter changes, or use a teacher model to discourage loss of earlier behavior. Each choice carries memory, computation and data-access assumptions.

Tao and colleagues formalized a setting in which the full old training datasets are unavailable during incremental stages. Other protocols can allow different memories, so comparisons must state exactly what is retained. Keeping a representation fixed can protect its parameters while still allowing new classifier choices to change old predictions.

Three practical examples

A visual catalog adds new packaging categories using a few labeled photographs, while continuing to recognize earlier packages.
A maintenance classifier receives a small set of examples of a newly defined sound pattern and is tested against both old and new categories.
An archive expands its document taxonomy one department at a time, retaining a test collection for previously introduced document types.

Think of it this way

Think of adding new entries to a field identification guide. A useful revision teaches the new distinctions without making the old pages misleading. Reference photographs cannot represent every lighting condition or viewpoint.

The analogy highlights two separate questions: whether the new entry generalizes beyond its references, and whether adding it makes earlier identifications more confusing.

Worked example

Consider an original teaching example with four established classes and two new classes. Each new class supplies five labeled training images, so the incremental training set contains ten images. These training images are excluded from evaluation.

A balanced test set contains twenty fresh images per class. There are eighty old-class and forty new-class test images. Before expansion, the old classifier correctly recognizes 72 of the eighty old images, giving 90% old-class accuracy.

After expansion, every test prediction chooses among all six classes. Suppose the new system correctly labels 68 old images and 30 new images. Old-class accuracy is 68/80 = 85%; new-class accuracy is 30/40 = 75%; combined accuracy is 98/120, approximately 81.7%.

The old-class decline is five percentage points. Reporting only the new-class 75% result would conceal it. Reporting only combined accuracy would also obscure the different results for old and new classes.

This arithmetic evaluates an invented outcome; it does not predict what a particular algorithm will achieve. Multiple incremental sequences and repeated support-set selections would be needed to assess stability. The test distribution may differ from production traffic.

Illustrative case study

Fictional case: Alder Components wants to add recently introduced parts to an image classifier. Its analysts reserve separate photographs for testing and choose five training photographs per new part from distinct capture sessions.

They compare a fixed-feature prototype method with a model that updates some learned weights. The review includes older parts that resemble the additions, because these may acquire new competitors even when their own stored representations remain unchanged.

Before accepting an update, the team documents permitted memory, class order and performance after each session. It also routes unfamiliar objects for review instead of assuming every object belongs to a registered class. This fictional case claims no measured improvement.

Common misconceptions and mistakes

New-class accuracy alone establishes success: retention of earlier classes is also part of the task.
Five examples means five examples in the model's entire history: the base representation may have used extensive prior training.
Freezing features guarantees unchanged predictions: expanding the set of candidate classes can change decisions.
Remembering all historical data is always allowed: benchmark and operational constraints differ.
Few-shot learning is necessarily prompting: this setting often involves trained representations and classification rules, rather than examples inserted into a language-model request.

Business use

This approach is relevant when categories expand faster than labeled datasets can be collected. Its business value depends on whether the organization can supply representative examples, preserve trustworthy evaluation data and manage update quality.

Compare the maintenance burden with periodic full retraining or a simpler retrieval-based system. Track old-class, new-class and overall performance alongside memory usage and update time. A small training set reduces annotation volume but can make results sensitive to mistaken labels and narrow capture conditions. Record those assumptions when presenting results to decision makers.

Check your understanding

A system improves accuracy on two new classes but falls sharply on the earlier classes. Has it demonstrated successful incremental few-shot learning?

Answer: Not on that evidence alone. The evaluation must account for both acquiring the additions and retaining earlier capability under the stated protocol.

Related terms

Few-shot learning
Class incremental learning
Continual Learning
Representation learning
catastrophic forgetting
prototype classifier
knowledge distillation
support set

Learn next

Design a small sequence of class additions. Separate support examples from test examples, document memory access, and evaluate against every class introduced so far. Repeat with different support selections and inspect which older classes become confused with the new ones.

Sources

Tao and colleagues, Few-Shot Class-Incremental Learning, CVPR 2020; problem setting and preservation challenge: https://arxiv.org/html/2004.10956

Zhang and colleagues, Few-Shot Incremental Learning With Continually Evolved Classifiers, CVPR 2021; alternative representation/classifier strategy: https://arxiv.org/abs/2104.03047

The accuracy calculation and Alder Components case are original hypothetical illustrations. Protocol assumptions should accompany any empirical comparison.

Previous: Incremental encoding Next: Incremental heuristic search