Can a logit be negative?

Yes. Negative binary logits correspond to probabilities below 0.5; multiclass raw scores can also be negative.

Is a logit the same as a log probability?

No. Binary logit uses ln[p/(1 − p)], while log probability uses ln(p).

Why are raw neural-network outputs called logits?

The terminology extends the binary pre-sigmoid score to vectors of pre-softmax class scores.

AI Glossary / Natural language and speech / Intermediate

Logit

Definition

The logit of a probability p is its natural log odds: logit(p) = ln[p/(1 − p)], for 0 < p < 1. In machine learning, “logits” also commonly means the raw scores supplied to a sigmoid or softmax transformation before probabilities are produced.

What it means

The two usages are connected but should not be collapsed. For a binary model with one score z and probability p = sigmoid(z), z is exactly the logit of p. It represents the log odds of the designated positive outcome against its complement.

For a multiclass softmax model, each raw score is also conventionally called a logit. An individual score is not generally the binary log odds of that class against all other classes. Instead, differences between class scores equal the log probability ratios between those classes.

Logits can be negative, zero or positive and need not sum to one. A raw score of two is not a 200% probability. The associated transformation and comparison set are necessary to interpret it.

How it works

For a binary probability, divide p by 1 − p to obtain odds, then take the natural logarithm. A probability of 0.5 gives odds of one and logit zero. Probabilities above 0.5 have positive logits, while those below 0.5 have negative logits.

The inverse is the logistic function p = 1/(1 + exp(−z)). This relationship lets a model express a bounded probability through an unbounded score. The endpoint probabilities zero and one correspond to limiting values of negative and positive infinity rather than finite real logits.

For softmax, pi = exp(zi)/Σj exp(zj). Dividing pi by pj cancels the shared denominator, giving ln(pi/pj) = zi − zj. Adding the same constant to every score leaves the probabilities unchanged.

That invariance is why an isolated multiclass logit lacks an absolute probability interpretation. Its meaning depends on the other scores and on any later transformations, such as temperature scaling or masking.

Three practical examples

A binary classifier exposes a score that an analyst converts into an estimated probability using the documented sigmoid transformation.
A language-model tool inspects token logits to understand which vocabulary alternatives are favored before sampling.
A debugging report compares score differences between two labels instead of treating a large raw multiclass value as self-explanatory confidence.

Think of it this way

Think of contestants receiving preliminary scores that still need an interpretation rule. In a binary logistic setup, one score is measured against a defined alternative. In a softmax setup, the relative scores across contestants determine the normalized result.

The analogy does not imply that logits are human judgments or that the resulting probabilities are deserved or correct. They are model quantities, and their calibration and practical reliability require separate evaluation.

Worked example

Comparing binary and multiclass meanings

Start with a binary probability of 0.80. Its odds are 0.80/0.20 = 4, so its logit is ln(4) ≈ 1.3863. Applying the logistic function to that value returns 0.80.

Now take a three-class softmax vector of raw logits [0, ln(2), ln(3)]. Exponentiating produces [1, 2, 3], with total six. The class probabilities are therefore [1/6, 2/6, 3/6]. The third class has probability 0.5 even though its raw logit is ln(3), not zero.

Its binary log odds against the other two classes combined is ln(0.5/0.5) = 0. This differs from its raw multiclass logit. Meanwhile, the log probability ratio of class three to class two is ln((3/6)/(2/6)) = ln(3/2), exactly their raw-score difference ln(3) − ln(2).

Adding ten to all three raw logits preserves the same probabilities. This checked example shows why the context of the term matters.

Illustrative case study

A fictional document-classification team compares outputs from two model versions. One version reports much larger raw multiclass scores, and a dashboard labels it more confident without calculating the resulting distribution.

An engineer demonstrates that adding a constant to every score can create that appearance while leaving softmax probabilities unchanged. The team revises the dashboard to display normalized estimates and selected score differences, with the transformation documented.

It then evaluates prediction accuracy and calibration on reviewed examples instead of inferring quality from raw score magnitude. The lesson is that logits are useful for understanding model computation, but comparisons need a consistent definition. The scenario describes a mathematical pitfall rather than a measured defect in a named product.

Common misconceptions and mistakes

A logit is already a probability. It is a transformed probability or an unnormalized model score, depending on context.
Every multiclass logit equals that class's binary log odds. Generally it does not.
Larger absolute softmax logits always mean greater confidence. A common shift changes no probabilities.
A zero raw multiclass logit guarantees probability 0.5. The other class scores still determine normalization.

Business use

Teams encounter logits in model integrations, diagnostic exports and generation controls. Documentation should specify whether a field contains binary log odds, multiclass scores or log probabilities; these quantities support different calculations.

Before building a confidence display, confirm the transformation and any subsequent modifications. A probability derived from logits still needs calibration checks for the intended use. Avoid comparing absolute score magnitudes across models or label sets without a justified method. Changes in vocabulary, class definitions or temperature can alter how a score relates to the final output.

Check your understanding

A model adds five to every entry in a softmax logit vector. What happens to its class probabilities?

Answer: They stay the same, because the shared exponential factor cancels between each numerator and the normalization denominator.

Related terms

Learn next

Calculate both logit(p) and ln(p) for a binary probability. Then normalize a small score vector with softmax and verify that a common score shift changes neither its probabilities nor its pairwise differences.

Sources

Jurafsky and Martin, Speech and Language Processing, Chapter 4: https://web.stanford.edu/~jurafsky/slp3/4.pdf. Binary logit and pre-softmax score terminology. SciPy, logit documentation: https://docs.scipy.org/doc/scipy/reference/generated/scipy.special.logit.html. Mathematical definition and endpoints. Multiclass identities and examples are directly checked from the softmax formula.

Previous: Logistic regression Next: Logit lens