What happens when the score is exactly zero?

In the convention used here, the signed margin is zero and an update occurs. Other implementations must document their tie handling.

Is the learning rate a probability?

No. It controls the size of the parameter change.

Does the convergence theorem guarantee a particular separator?

No. Different presentation orders can lead to different separating parameters, and their future performance can differ.

AI Glossary / Deep learning / Intermediate

Perceptron learning rule

Definition

The perceptron learning rule is an error-driven procedure for adjusting a linear classifier's weights and bias. When an example is misclassified, or lies on the decision boundary under a common convention, the rule moves the parameters toward a score consistent with its label.

What it means

The rule explains how a perceptron learns rather than what it computes after training. The classifier forms a weighted sum; the learning procedure changes those weights using labeled examples.

It is an online-style algorithm because it can update after one example at a time. A training pass may revisit every example, since correcting one case can change predictions for others. The order of presentation can affect the final separator.

The classic guarantee is conditional: a finite, linearly separable training set can be separated after finitely many standard updates. This does not mean every dataset is separable, that the best possible separator is found, or that future mistakes disappear. Practical stopping rules are still necessary when those assumptions do not hold.

How it works

Use labels y in {−1, +1}, input x, weights w and bias b. Compute the score s = w·x + b. In a common margin-based convention, update whenever y × s is zero or negative.

For learning rate eta greater than zero, the update is w_new = w + eta × y × x and b_new = b + eta × y. Otherwise leave the parameters unchanged. The bias can also be handled by adding a constant input feature.

A positive example adds a scaled input vector; a negative example subtracts it. This increases the signed score of the example that triggered the update. Other examples may improve or worsen.

Different presentations use labels 0 and 1 or express the update through prediction error. Their threshold and scaling conventions must be kept consistent. The classic rule is not the same as fitting logistic probabilities, nor does it require differentiating through a hard threshold.

Three practical examples

A student records every mistaken classification and draws the boundary after each update.
A text-classification experiment makes sequential updates from sparse labeled feature vectors and compares performance with another linear model.
A monitoring prototype examines whether a stream of persistent mistakes reflects drifting inputs, contradictory labels or insufficient features.

Think of it this way

Think of moving a straight divider after discovering a misplaced item. A correction moves the divider in a direction that helps that item, but may put another item on the wrong side. Rechecking earlier items is therefore part of the process.

The analogy is geometric. The algorithm does not deliberately negotiate among cases, and it cannot bend a straight divider around an intrinsically nonseparable pattern. It also cannot decide whether a disputed label is trustworthy.

Worked example

Correcting one example can disturb another

Take invented weights w = (1, −1), bias b = 0, and a positive example x = (2, 3). Its score is 2 − 3 = −1, so the update is triggered. With eta = 0.5, add 0.5 × (2, 3) to the weights and 0.5 to the bias.

The new weights are (2, 0.5), and the new bias is 0.5. The same example now scores 4 + 1.5 + 0.5 = 6, correctly positive.

However, consider a negative example z = (1, 2). Before the update its score was 1 − 2 = −1, correctly negative. Afterwards its score is 2 + 1 + 0.5 = 3.5, incorrectly positive. The rule has not failed: it made a local correction, and the training process must revisit affected cases. One successful update is not evidence of convergence.

Illustrative case study

Fictional example: Elm Document Services experiments with a perceptron classifier for two document queues. The team logs updates during training and sees that several records repeatedly cause opposite parameter changes.

A review finds two records with identical feature vectors but different labels. Some differences were lost during preprocessing, while one pair was labeled inconsistently. A deterministic classifier receiving identical inputs cannot assign them different classes based on information it never sees.

The team corrects the annotation error, revisits the feature representation and imposes a documented training limit. It evaluates the selected model on held-out records rather than waiting indefinitely for a mistake-free pass. Where overlap remains, it treats the result as a limited classifier rather than claiming the convergence theorem applies.

The lesson is that repeated updates can reveal a data or representation problem. Increasing the iteration count is not a substitute for understanding that problem.

Common misconceptions and mistakes

Every training example changes the weights. The basic rule updates only on specified errors or boundary cases.
More epochs guarantee convergence. The classic guarantee requires separability and the relevant algorithmic assumptions.
Convergence finds the maximum-margin boundary. The ordinary perceptron can find a separator without maximizing margin.
An update that fixes one case fixes the dataset. Other predictions can change in the opposite direction.

Business use

The rule provides a compact baseline for learning from labeled vectors and a useful way to inspect the interaction between examples and representation. Its limited state and simple updates can make experimentation inexpensive.

For an operational decision, specify feature scaling, label conventions, presentation order, stopping conditions and evaluation data. Compare error patterns with other simple classifiers. If the business needs probability estimates, a bare perceptron score is insufficient. If data changes over time, retaining an online update mechanism also requires a policy for trustworthy labels and for detecting harmful adaptation.

Check your understanding

An update correctly classifies the example that triggered it but causes an earlier negative example to become positive. Must the implementation be wrong?

Answer: No. A perceptron update targets the current example. Subsequent passes and evaluation are needed because the shared boundary affects other examples too.

Related terms

Learn next

Read perceptron and linear separability. Work through a short labeled sequence, recording the score, update decision, weights and bias after every example. Recheck all earlier predictions after each change.

Sources

Cornell perceptron lecture, algorithm and convergence: https://www.cs.cornell.edu/courses/cs4780/2018fa/lectures/lecturenote03.html. Cornell course slides with signed-margin update and assumptions: https://www.cs.cornell.edu/courses/cs3780/2026sp/slides/lec4.pdf. scikit-learn implementation documentation: https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.Perceptron.html. Numerical cases and the company example are original illustrations.

Previous: Perceptron Next: Perceptual computing