Is logistic regression a linear classifier?

In its usual binary form with affine log-odds, it has a linear decision boundary for a fixed probability threshold.

Does linear mean there is no intercept?

In common machine-learning usage, linear classifiers often include an intercept; strictly, their scores are then affine.

Can a linear classifier handle nonlinear patterns?

It can do so through an appropriate transformed feature representation, but not every pattern is separable in the original features.

AI Glossary / Mathematics and optimization / Intermediate

Linear classifier

Definition

A linear classifier assigns categories using linear or affine scores of the input features. In a binary task, a common rule predicts one class when wᵀx+b is nonnegative and the other when it is negative. Its decision boundary is a hyperplane in the feature space used by the classifier.

What it means

With two features, the boundary is a line; with three, it is a plane. In higher dimensions, the same idea is called a hyperplane. The coefficients determine the boundary's orientation, while the intercept shifts its position.
Linear refers to the score and boundary structure in the chosen features. The final category is discrete, so the complete classification function is not a linear map in the algebraic sense. Also, a linear classifier applied to transformed features can create a curved boundary in the original inputs.
Different training methods can produce linear classifiers. Their fitting objectives, margins, regularization, and probability interpretations can differ. The phrase describes a family of decision structures rather than a single algorithm or guarantee about performance.

How it works

Represent each example with a feature vector x. Training estimates weights w and an intercept b according to an objective and any regularization. At prediction time, compute the score s=wᵀx+b and apply the stated threshold rule.
For several classes, a common construction computes one affine score per class and chooses the largest. Pairwise score ties occur on hyperplanes. Tie handling should be specified where it matters.
Some linear classifiers provide scores that can be converted to estimated probabilities under their model, while others provide margins without a probability interpretation. A raw score of two is not automatically twice as likely as a score of one. Feature scaling must also be consistent between training and use because weights are tied to the chosen representation.

Three practical examples

A document classifier can combine word-based features into scores for routing categories.
A simple image-recognition baseline can classify vectors of predefined image features using a linear decision boundary.
An operational model can combine measured attributes into a category score, then evaluate whether its threshold suits the intended review workload.

Think of it this way

Think of a weighted checklist whose total determines which side of a cutoff an example falls on. The analogy captures additive scoring. It does not imply that the weights are causal effects, that features are independent, or that the score is a calibrated probability.

Worked example

Consider an original two-feature classifier with score s(x)=2x₁−x₂−3. Predict class A when s≥0 and class B otherwise. The boundary satisfies x₂=2x₁−3.
For P=(3,1), the score is 6−1−3=2, so P receives A. For Q=(1,1), it is 2−1−3=−2, so Q receives B. For R=(2,1), it is 4−1−3=0, so the stated tie rule assigns A. A different tie rule would only change what happens exactly on the boundary.
Multiplying every coefficient, including the intercept, by a positive constant leaves these class decisions unchanged. For example, 10x₁−5x₂−15=5s(x) has the same sign. Therefore the raw score magnitude depends on scaling. The signed Euclidean distance to the boundary is s/√5 for the original weights, so P lies 2/√5, approximately 0.894, from it on the A side.
Now consider binary inputs (0,0), (0,1), (1,0), and (1,1), with A assigned to the unequal pairs and B to the equal pairs. A single affine boundary cannot realize all four decisions under the stated sign convention. The two A requirements sum to w₁+w₂+2b≥0, while the two B requirements sum to w₁+w₂+2b<0, a contradiction. This is the familiar exclusive-or pattern and illustrates a limitation of linear separation in the original two features.

Illustrative case study

In a fictional help-desk project, analysts begin with a linear text classifier because it provides a manageable baseline and fast scoring. They inspect examples near its boundary and discover that two request types share many words but differ through context.
The team tests whether better features resolve the confusion and compares an alternative model on the same held-out examples. It does not assume a more complex model will necessarily help, nor that the linear model's coefficient list supplies a complete explanation of each request. This is an illustrative workflow without reported deployment results.

Common misconceptions and mistakes

Do not confuse linear classification with linear regression: their output tasks and training objectives can differ. Do not assume a linear boundary in transformed features is a straight line in raw input coordinates. A polynomial feature such as x₁x₂ can change the available decision shapes.
Do not compare coefficient magnitudes without considering feature units and preprocessing. Correlated features complicate interpretation, and a large coefficient does not establish a causal effect. Finally, linear separability of training examples is not proof of reliable predictions on future examples.

Business use

Linear classifiers can be useful baselines when the chosen features contain enough information for an additive scoring rule. Their practical appeal can include compact models and inexpensive prediction, although the actual cost depends on feature construction and dimensionality.
Teams should evaluate errors relevant to the use case, document the threshold, and inspect performance across meaningful subsets. If the boundary cannot capture important interactions, consider whether feature design or another model family addresses the problem. Keep probability calibration separate from the question of whether the classifier's labels are accurate.

Check your understanding

A binary classifier uses s=3x₁+x₂−4 and predicts A when s≥0. What does it predict at (1,2)?

Answer: s=3+2−4=1, so it predicts A.

Related terms

Learn next

Next, draw a two-feature decision boundary and calculate scores on both sides. Then compare linear classifiers trained with different losses to see why identical boundary forms do not imply identical fitting procedures or probability outputs.

Decision boundary

Sources

Primary terminology reference: Alexander Jung and contributors, The Aalto Dictionary of Machine Learning, linear classifier entry: https://aaltodictionaryofml.github.io/ADictML_Main.pdf . The score calculations, boundary-distance calculation, and exclusive-or contradiction are original demonstrations.

Previous: Linear chain CRF Next: Linear constraints