Why is it called regression if it classifies?

It models a probability through a regression structure; classification is obtained by applying a decision rule.

Can it handle more than two classes?

Related multinomial formulations and multiclass strategies are available.

Does a coefficient show a constant probability change?

No. Without interactions, it gives a constant log-odds change per feature unit, holding other features fixed.

AI Glossary / Statistics and probability / Beginner

Logistic regression

Definition

Logistic regression is a statistical and machine-learning method for modeling the probability of a categorical outcome. Its basic binary form estimates the probability of one of two outcomes, such as renewed versus not renewed. It combines input features into a linear score and transforms that score with the logistic function. Despite its name, it is widely used for classification, while its underlying output is an estimated probability rather than a compulsory yes-or-no decision.

What it means

The binary model makes the log-odds of the outcome linear in the chosen features. If p is the probability of the designated positive outcome, its odds are p/(1 - p), and its log-odds are the natural logarithm of that ratio.

Probability itself is not linear in the inputs. An equal increase in the score has a larger probability effect near the middle than near zero or one. Features can include encoded categories, transformations, and interactions, so the model need not be limited to straight relationships in the original raw measurements.

How it works

Define the positive outcome and the information available when predictions will be made. Encode features consistently, split data appropriately, and estimate coefficients from labeled observations. A common objective minimizes binary negative log likelihood, also called binary cross-entropy, optionally with a regularization penalty.

For features x, calculate z = intercept plus the sum of each coefficient times its feature value. Then p = 1/(1 + exp(-z)). With finite coefficients, the mathematical output lies strictly between zero and one.

Evaluate probabilities on held-out data using calibration and suitable predictive metrics. If an action requires a class label, choose a threshold based on the decision's costs and constraints. The threshold is separate from fitting the probability model; 0.5 is a convention, not a universal business optimum.

Three practical examples

A subscription service estimates the probability that an account will renew using information available before the renewal date.
A manufacturing team predicts whether an inspected component will fail a predefined quality criterion.
An AI application uses a logistic model as a transparent baseline for routing requests to human review, assessing both missed cases and review workload.

Think of it this way

Think of a mixing board that adds weighted signals into one score, followed by a dial that converts the score into a probability. Moving the score by one unit always multiplies the odds by the same factor, but the probability dial moves by different amounts depending on where it started.

Worked example

Consider an original teaching model for an event probability: z = -2 + 0.8x + 0.6r, where x is a measured activity score and r is an indicator taking value 0 or 1. These coefficients are supplied for illustration, not estimated from real customer data.

For x = 2 and r = 1, z = -2 + 1.6 + 0.6 = 0.2. The probability is 1/(1 + exp(-0.2)), approximately 0.5498. Increasing x to 3 while keeping r fixed gives z = 1 and probability approximately 0.7311, an increase of about 18.12 percentage points.

The corresponding odds rise from exp(0.2), approximately 1.2214, to exp(1), approximately 2.7183. Their ratio is exp(0.8), approximately 2.2255. This is an odds multiplier, not a probability multiplier. At a threshold of 0.6, the first case receives the negative label and the second the positive label. At a threshold of 0.5, both receive the positive label even though the probability estimates have not changed.

Illustrative case study

Fictional case: Willow Subscriptions builds a renewal model to help plan support outreach. Analysts remove features that would become known only after renewal and evaluate predictions on a later period. They compare the logistic baseline with a more complex model using probability quality and operational outcomes.

The team finds that the simpler model is useful for estimating workload but that some customer groups need better calibration. It adjusts the evaluation and outreach policy before deployment. A coefficient associated with renewal is not treated as proof that changing that feature will cause customers to renew; outreach effectiveness requires a separate causal evaluation.

Common misconceptions and mistakes

An odds ratio is not a risk ratio or a percentage-point change. A positive coefficient does not establish causation, and its interpretation depends on feature coding and the other included terms. Correlated features can make individual coefficients unstable.

Probability outputs are not automatically well calibrated. Class reweighting, sampling changes, or population shifts can affect their interpretation. Complete separation can prevent an ordinary unpenalized fit from having finite coefficient estimates. Regularization can help stabilize fitting, but it does not repair data leakage or an incorrectly defined outcome.

Business use

Logistic regression is useful when businesses need a reasonably compact probability model, a practical baseline, or interpretable associations. It can work well with structured features and may be easier to maintain than a more complex alternative.

Assess errors in the context of the action: a false alarm and a missed event may have very different costs. Monitor calibration, subgroup performance, and changes in feature distributions. Store the feature preparation steps with the model so production inputs have the same meaning as training inputs. Simplicity supports review, but it does not remove the need for validation.

Check your understanding

1. What is the first worked probability?

Answer: Approximately 0.5498. 2.

Does exp(0.8) multiply the probability?

Answer: No; it multiplies the odds for a one-unit increase in x under this model. 3.

Does changing a classification threshold refit the model?

Answer: No; it changes the action rule applied to the existing probabilities.

Related terms

Learn next

Practice converting probabilities to odds and log-odds. Then study model calibration, class imbalance, interaction terms, and threshold selection. Compare logistic regression with a simple baseline before adding more complicated features or algorithms.

Sources

Google, Machine Learning Crash Course, “Calculating a probability with the sigmoid function.” https://developers.google.com/machine-learning/crash-course/logistic-regression/sigmoid-function — primary educational explanation of score-to-probability conversion. Scikit-learn, “Logistic regression.” https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression — primary documentation on binary and multinomial formulations, fitting, and regularization. The worked example and fictional case are original.

Previous: Logistic loss Next: Logit