Is it always a weighted average?

No. It can fit a local model, such as a weighted linear regression, around each query.

What does bandwidth control?

It controls the scale over which examples influence a query, according to the chosen weighting function.

Can a prediction be unsupported by nearby data?

Yes. Sparse regions or narrow neighborhoods can leave too little information or even zero total weight.

AI Glossary / Mathematics and optimization / Intermediate

Locally weighted learning

Definition

Locally weighted learning makes a prediction by emphasizing training examples relevant to the query being answered. Relevance is commonly determined by distance or similarity in feature space. The method may average nearby targets or fit a local model, with weights that change when the query changes.

What it means

The central idea is that different parts of the input space may need different local approximations. Instead of insisting that one simple model describe every region equally well, the method gives greater influence to examples near the current query.
Locally weighted learning includes more than a weighted average. A local constant fit produces an average, while a local linear fit estimates a trend around the query. Nearest-neighbor regression is closely related, using selected nearby examples to make a prediction.
The word local refers to the chosen feature representation and distance, not necessarily physical location. Two customers can be near each other in a feature space while living far apart. Whether that mathematical neighborhood is useful depends on whether its features capture relevant similarities.

How it works

Represent the query as q and training inputs as xᵢ. Choose a distance and a weighting rule, often with a bandwidth controlling how rapidly influence decreases with distance. For a local constant prediction, compute ŷ(q)=Σwᵢ(q)yᵢ/Σwᵢ(q), provided the denominator is positive.
For a local linear fit, choose coefficients that minimize weighted residual error near q and evaluate the fitted model at q. The weights usually depend on the query, so another query can produce another fitted model.
A narrow neighborhood can follow local variation closely but may contain too little information. A broad neighborhood uses more examples but can smooth over important changes. Feature scaling, neighborhood size, and local-model complexity therefore need evaluation together. The learning procedure must also define what happens when no suitable nearby examples exist.

Three practical examples

A numerical prediction service can estimate an outcome from comparable historical cases using a documented similarity rule.
An analyst can fit a smooth local trend to measurements whose relationship changes across the input range.
A robotics research model can approximate behavior near a current state from nearby recorded experiences, while separately addressing control requirements.

Think of it this way

Think of asking the most relevant nearby examples for advice, giving some more influence than others. The analogy captures query-specific weighting, but examples do not reason or judge relevance themselves. The distance, weights, and local model are chosen mathematically, and a nearby example can still be misleading.

Worked example

Consider an original one-dimensional training set: inputs 0, 1, 2, and 4 with targets 1, 3, 5, and 20. Use the triangular weighting rule wᵢ(q)=max(0,1−|xᵢ−q|/h), where h is a positive bandwidth.
At q=1.5 with h=2, the weights are 0.25, 0.75, 0.75, and 0. Their sum is 1.75. The weighted target sum is 0.25×1+0.75×3+0.75×5=6.25. The local constant prediction is 6.25/1.75=25/7, approximately 3.571.
At q=3.5 with the same bandwidth, only the inputs 2 and 4 receive positive weights: 0.25 and 0.75. The prediction becomes 0.25×5+0.75×20=16.25. The same stored dataset therefore gives different effective weighting at different queries.
Return to q=1.5 but reduce h to 0.4. Every training input is at least 0.5 away, so all weights are zero. The weighted-average formula is undefined; a practical procedure needs an explicit fallback, such as expanding the neighborhood or declining the prediction.
Finally, use a local linear fit at q=1.5 with h=2. The three positive-weight points lie exactly on y=1+2x, so that line has zero weighted residual error and predicts 4 at the query. This differs from the local constant result. Choosing the local model matters as well as choosing the weights.

Illustrative case study

In a fictional equipment-service project, analysts predict job duration from similar past jobs. They initially use raw distance across machine age, job size, and a coded machine category. Review shows that arbitrary category codes and mismatched units dominate the neighborhood selection.
The team revises the representation, checks retrieved neighbors for several queries, and compares predictions on held-out cases. It also marks queries with little nearby support for additional review. This hypothetical workflow illustrates relevance design and coverage checking, without claiming measured prediction gains or a deployed system.

Common misconceptions and mistakes

Do not assume locally weighted means geographically nearby or automatically personalized in a meaningful way. Do not treat an arbitrary category code as a sensible distance coordinate. Feature scaling can radically change which examples are considered close.
A nonnegative normalized weighted average stays within the range of the targets receiving positive weight. A local linear fit need not share that property, particularly near boundaries or during extrapolation. Also, memory-based methods can shift substantial computation to prediction time. Calling a method nonparametric does not mean it has no tuning choices or computational costs.

Business use

Locally weighted learning can be attractive when historical comparisons are useful and relationships vary across the input space. Review the meaning of similarity, local data coverage, prediction latency, and how stored data will be updated. An explanation based on neighboring examples is only as useful as the quality of those comparisons.
Evaluate performance across dense and sparse regions rather than reporting only one overall average. A method can work well for common queries while having little evidence for unusual ones. Keep neighborhood selection and bandwidth tuning within the training process so evaluation data do not silently influence the chosen model.

Check your understanding

Two examples with targets 10 and 20 have weights 1 and 3. What is their local constant prediction?

Answer: (1×10+3×20)/(1+3)=17.5. Normalization by total weight is essential.

Related terms

Nearest Neighbors Regression
Kernel regression
Local regression
Lazy learning
Weighted least squares
Nearest-neighbor regression
bandwidth
distance metric
feature scaling
nonparametric learning

Learn next

Next, compute predictions at two queries using the same weighting rule, then change the bandwidth. Compare a local constant fit with a local line and inspect what each does near the edge of the available data.

Sources

Primary research source: Christopher G. Atkeson, Andrew W. Moore, and Stefan Schaal, Locally Weighted Learning, original 1997 paper, distance-weighted averaging and local-model sections: https://www.qou.edu/ar/sciResearch/pdf/distanceLearning/locallyWeighted.pdf . Original publication record: https://publications.ri.cmu.edu/locally-weighted-learning . The triangular-weight example is original.

Previous: Locally adaptive regression kernel (LARK) Next: Locally weighted regression