Is this classification?

No. Regression combines numerical targets; classification predicts categories.

Does larger k always improve results?

No. In the example it changes 18 to approximately 15.33. Whether that helps depends on the unknown target and the data-generating pattern.

AI Glossary / Statistics and probability / Advanced

Nearest Neighbors Regression

Definition

Nearest neighbors regression predicts a numerical outcome by combining outcomes from similar training observations. Similarity is defined through a distance measure on input features. A common version averages the outcomes of the k closest observations; another uses observations within a chosen radius. Contributions can be equal or weighted by distance.

What it means

The method bases a new prediction on nearby recorded cases. Its usefulness therefore depends on what nearby means and whether those cases provide relevant evidence. Two customers might be close in age but far apart in purchasing behavior. Choosing a distance representation is part of designing the model, not merely a technical search setting.

How it works

Store training inputs and their numerical targets. For a query, calculate distances, select its neighborhood, and compute the chosen weighted average. With positive weights w, the prediction is the sum of w times each target divided by the sum of w. Inverse-distance weighting emphasizes closer observations. Exact matches require an explicit zero-distance convention.

Three practical examples

A service team predicts handling time from similar historical requests. Features must be available when the request arrives; including its eventual resolution code would leak information from the future.
A facilities team predicts hourly energy demand from weather and occupancy measurements. It tests on later periods rather than randomly scattering adjacent hours across training and testing.
A repair workshop estimates labor hours using comparable completed jobs. It records the distance to the nearest examples so staff can recognize a query that has little support, even when the software still returns a number.

Think of it this way

Imagine estimating a task's duration by looking at a few previous tasks judged similar. The estimate changes if you consult one colleague's case, several cases, or give the most comparable case extra influence. The procedure is transparent only when the criteria for similarity and the selected cases are visible.

Worked example

Consider an original training table with one input x and an outcome y. Its four observations are (1,10), (2,14), (4,22), and (7,50). Use ordinary absolute distance on x. The numbers are teaching data, not results from a business deployment.

For query x = 3, the distances to the four observations are 2, 1, 1, and 4. With k = 2, the selected neighbors have outcomes 14 and 22. Their equal-weight prediction is (14+22)/2 = 18. Both are equally close, so inverse-distance weighting also gives 18.

Increase k to three. The additional observation has outcome 10. Equal weighting gives (10+14+22)/3 = 46/3, approximately 15.33. Inverse-distance weights are 1/2, 1, and 1, so the prediction becomes (5+14+22)/(0.5+1+1) = 16.4. Changing neighborhood size and changing weights are distinct choices.

Move the query to x = 3.5 and retain k = 2. The selected inputs remain 4 and 2, but their distances become 0.5 and 1.5. The equal-weight prediction stays 18. Inverse-distance weights are 2 and 2/3, producing (2 times 22 + (2/3) times 14)/(2+2/3) = 20. A one-neighbor prediction would be 22. These are three valid outputs from different specified rules, not three estimates from the same fitted configuration.

Now query x = 10. The two closest recorded inputs are 7 and 4. Their average outcome is 36. With inverse-distance weights 1/3 and 1/6, the estimate is (50/3+22/6)/(1/3+1/6), approximately 40.67. Neither calculation extends the observed upward trend beyond 50. A nonnegative weighted average stays between its selected target values; requesting a prediction outside the training-input range does not create a trend-extrapolating model.

Finally, examine feature scaling in a separate two-input example. The query is (0,0); candidate A is (1,100) with target 10, and candidate B is (2,0) with target 20. Raw Euclidean distances are approximately 100.005 and 2, so a one-neighbor model selects B. Divide the second coordinate by 100: the distances become the square root of 2 and 2, so it selects A. No target changed. The new scale changed the meaning of closeness and therefore changed the prediction from 20 to 10.

Illustrative case study

This fictional case concerns Cedar Repairs, an imaginary workshop exploring duration estimates. Its prototype initially ranks past jobs using several measurements in their original units. A reviewer discovers that one large-scale feature dominates the distance calculation, much as the second coordinate dominates the raw example above.

The team compares justified scaling choices and neighborhood sizes using held-out jobs. Any scaling parameters learned from data are fitted using training data only. Staff receive the prediction together with the selected comparable jobs and their distances. Unusual jobs are reviewed separately. The case describes a proposed evaluation workflow and does not claim a measured reduction in costs or completion time.

Common misconceptions and mistakes

A close neighbor is close under the selected representation, not necessarily similar in every business-relevant sense. Adding irrelevant features can change rankings even when those features provide no useful outcome information.

A model that returns a number for every query has not demonstrated reliable coverage. Radius-based selection can find no neighbors, requiring a defined fallback. Ties at the boundary of a fixed-size neighborhood also require attention: different tie handling can select different outcomes.

Business use

Review error size, error direction, and coverage together. In the four-point example, a query at 10 gets a plausible-looking result despite being beyond every training input. A business interface could expose that distance and request review. The appropriate trigger should come from operational evaluation, rather than an arbitrary universal cutoff.

Check your understanding

Why does the inverse-distance estimate at 3.5 equal 20 rather than 18?

Answer: The observation with outcome 22 receives three times the weight of the observation with outcome 14.

Why did scaling reverse the selected neighbor?

Answer: It changed the relative contribution of the two input coordinates to distance.

Related terms

Learn next

Recalculate the four-point example with k equal to one, two, and three. Then document how your implementation handles exact matches, boundary ties, and queries with no radius-selected neighbors.

Sources

scikit-learn contributors, Nearest Neighbors, official user guide, regression section and tie warning: https://scikit-learn.org/stable/modules/neighbors.html . All numerical examples and the Cedar Repairs case are original illustrations; they do not reproduce a reported experiment.

Previous: Nearest neighbor search Next: Nearest Shrunken Centroid