Does instance selection always improve accuracy?

No. It may improve, preserve or reduce accuracy depending on the data, method and retained examples.

Is random sampling a form of selecting instances?

It selects a subset, but specialized instance-selection methods use task-related criteria beyond random choice.

Does the method create new training examples?

Selection itself retains existing examples. Creating new representatives is usually discussed as prototype generation or another transformation.

AI Glossary / AI and machine learning / Intermediate

Instance selection

Definition

Instance selection is the process of choosing a subset of existing examples to retain for a learning task. It can reduce storage or computation, remove suspected noise, or preserve useful representatives while discarding examples considered redundant for the chosen method.

What it means

An instance is one observation or record, such as one labeled product image or one row of measurements. Selecting instances changes which rows are used, rather than which input features appear in each row.

The idea is especially important in instance-based learning, where prediction depends directly on stored examples. A nearest-neighbor classifier may become cheaper to use if a smaller reference set preserves the distinctions needed for classification.

Different selection goals can conflict. Removing redundant interior points may preserve a boundary, while removing suspicious boundary points may smooth noise. A method designed for one goal should not be assumed to achieve the other automatically.

Instance selection is also distinct from creating synthetic prototypes. Selection retains a subset of original examples. A method that constructs new representative vectors performs a different kind of data reduction, even if both approaches reduce storage.

How it works

Define the downstream learner and the purpose of reduction. A useful subset for one distance measure or classifier may be unsuitable for another, so selection should be connected to the intended task.

Apply a selection criterion using the training data. A method may examine neighborhood relationships, classification consistency or the effect of removing an example on other training cases. Some procedures add useful cases; others start with all cases and remove them.

Keep selection inside the training process. If cross-validation is used, perform the selection separately within each training fold rather than using held-out labels to choose which examples survive.

Evaluate both predictive behavior and resource savings. Inspect rare classes, difficult regions and changes in class balance. A smaller dataset is not a successful result if it discards the observations needed for important cases.

Three practical examples

1. A nearest-neighbor application reduces a large reference collection to lower memory and query costs. The team measures how much neighbor-based prediction changes after selection.

2. A research dataset contains suspected mislabeled examples. An editing procedure identifies candidates for removal, but unusual cases are reviewed rather than automatically equated with errors.

3. A training experiment compares random subsampling with a task-aware selection method. It evaluates whether the added selection cost produces a better accuracy-storage tradeoff.

Think of it this way

Think of choosing a compact set of examples for a training manual. Repeated examples may add little, but a rare exception can be essential. The analogy is limited because algorithmic selection uses a specified criterion and may fail to recognize the practical importance of an unusual case unless that importance is represented in the objective and evaluation.

Worked example

Suppose a reference set contains 1,000 examples, and an instance-selection method retains 250. The storage count is reduced by 750 examples, or 75 percent.

On an independent evaluation set of 200 cases, the full reference set makes 12 errors and the selected set makes 14. Their error rates are 6 percent and 7 percent. The reduction therefore comes with a one-percentage-point increase in observed error in this invented comparison.

Whether that tradeoff is acceptable depends on error costs, uncertainty and which cases changed. If the two additional errors affect a critical rare class, an acceptable average rate may conceal an unacceptable local loss. Query speed must also be measured rather than inferred directly from the percentage reduction.

Illustrative case study

Fictional case: Rowan Parts uses a nearest-neighbor baseline to classify component images. Many stored examples are nearly identical, so engineers investigate reducing the reference set.

They evaluate several selection methods and retain a separate test collection that includes uncommon components. One method produces a compact set but removes most examples of a rare class, prompting a review of its selection objective.

The team reports storage, latency and class-specific performance together. This invented case illustrates why representative coverage matters; it does not claim that any particular reduction algorithm guarantees the best subset.

Common misconceptions and mistakes

Mistake: Confusing instance selection with feature selection. One chooses examples; the other chooses variables or attributes.

Mistake: Assuming every isolated example is noise. Rare but legitimate cases may be especially important to preserve.

Mistake: Selecting instances using the entire dataset before evaluating on held-out cases. Information from the evaluation data can leak into the selection process and make results optimistic.

Business use

Instance selection can reduce the operating cost of example-based systems and support data-quality investigation. Its value should be judged through the full pipeline, including the time needed to choose and maintain the subset.

Keep traceability to original records and document why examples were excluded. Reassess the subset as new classes or operating conditions appear. A compact reference collection that worked last year may no longer represent the population the system now serves.

Check your understanding

A reduction method keeps only 10 percent of examples and preserves average accuracy, but loses most rare-class cases. Is the average enough to approve it?

Answer: No. Coverage and performance for important subgroups must be assessed before judging the tradeoff acceptable.

Related terms

Instance-based learning
Data editing
Feature Selection
nearest-neighbor classifier
prototype selection
class imbalance
training-data leakage

Learn next

Learn nearest-neighbor classification and cross-validation. Remove one example from a small reference set, observe which predictions change, and distinguish redundant examples from rare examples that preserve an important boundary.

Sources

Wilson and Martinez, Reduction Techniques for Instance-Based Learning Algorithms: https://www.researchgate.net/publication/226428802_Reduction_Techniques_for_Instance-Based_Learning_Algorithms. Primary research text consulted for reduction goals and methods; numerical comparison and fictional case are original.

Previous: Instance-based learning Next: Instantaneously trained neural networks