Can logistic loss be zero?

It approaches zero as a finite-label prediction becomes infinitely confident and correct; for a finite logit, it remains positive.

Is logistic loss the same as binary cross-entropy?

Yes, after matching label encoding and using the sigmoid relationship between logits and probabilities.

Does minimizing it guarantee calibrated probabilities?

No. The population objective favors correct probabilities under suitable conditions, but finite data, model restrictions, weighting, and optimization affect actual calibration.

AI Glossary / Mathematics and optimization / Intermediate

Logistic loss

Definition

Logistic loss is a smooth loss function for binary prediction that penalizes confidence assigned to the wrong outcome. With a label y∈{−1,+1} and real score s, it is log(1+exp(−ys)). Equivalently, with labels coded as zero or one and a sigmoid probability, it is binary cross-entropy.

What it means

The product ys is the signed margin. A large positive margin means the score points strongly toward the observed class and produces a small loss. A large negative margin means a confident error and produces a large loss. A score of zero gives loss log2 under either binary label.
Unlike classification accuracy, logistic loss uses the strength of the prediction. Two models can make the same class decisions but receive different losses because their probabilities differ. It therefore assesses an aspect of probabilistic prediction that a correct-or-incorrect count discards.
The loss is used beyond ordinary logistic regression: a neural network can also produce a binary logit and be trained with this objective. The loss's name does not determine the architecture of the model that produces the score.

How it works

For a real logit s, compute p=1/(1+exp(−s)), the modeled probability of the positive class. With a target t∈{0,1}, the loss is −t log p−(1−t)log(1−p). Setting y=2t−1 yields the margin form log(1+exp(−ys)).
Training usually sums or averages these per-example losses, possibly adding regularization or explicit weights. The derivative with respect to s is p−t, giving the optimizer a direction for correcting the score. With an affine score, the ordinary unregularized objective is convex in its coefficients; this does not make a deep network's parameter objective convex.
Numerically stable implementations evaluate the combined logit-loss expression without naively exponentiating very large values. The distinction between logits and probabilities is essential when choosing a software loss function.

Three practical examples

A binary document classifier can train on logistic loss while producing a probability-like score for the positive category.
A multi-label system can apply a separate binary logistic loss to each tag, with the target structure stated explicitly.
An evaluation report can compare probabilistic forecasts using average log loss alongside threshold-based metrics such as precision and recall.

Think of it this way

Think of a prediction score that charges more for being confidently wrong than cautiously wrong. The analogy captures the loss's behavior, but the charge is a mathematical logarithmic penalty, not automatically a financial cost. It also does not mean uncertainty should always be increased: confident correct predictions receive lower loss.

Worked example

Consider an original positive example with y=+1. If the logit is s=2, the positive-class probability is approximately 0.880797 and the logistic loss is log(1+exp(−2)), approximately 0.126928. At s=0, the probability is 0.5 and the loss is approximately 0.693147. At s=−2, the probability is approximately 0.119203 and the loss is approximately 2.126928.
Now evaluate two examples, one positive and one negative, while predicting logit 2 for both. The losses are approximately 0.126928 and 2.126928, giving a mean of 1.126928. A model that predicts logit zero for both has mean loss 0.693147. With a tie rule that assigns zero to the positive class, both models classify only one of these two examples correctly, yet their average losses differ substantially.
Using the zero-one target convention, a positive example with p=0.9 has loss −log0.9≈0.105361. A negative example given that same positive probability has loss −log0.1≈2.302585. The larger penalty reflects confidence in the wrong outcome.
Finally, for t=1 and s=2, the score derivative is p−t≈−0.119203. A small gradient-descent update to this score alone moves it upward, reducing the loss for that example. In an actual shared-parameter model, updates reflect all contributing examples and the parameterization, so one example's preferred direction need not improve every other prediction.

Illustrative case study

In a fictional subscription-analytics project, two classifiers have similar accuracy at a chosen threshold. One assigns extremely high probabilities to several incorrect predictions. The team compares their average logistic losses and inspects those cases rather than treating the equal accuracy numbers as interchangeable performance.
It also evaluates calibration and the operational consequences of threshold choices. The hypothetical example shows how log loss complements decision metrics; it does not claim that optimizing it alone establishes a useful business policy or a measured return.

Common misconceptions and mistakes

Do not put zero-one labels directly into log(1+exp(−ys)) without converting the encoding: y=0 would incorrectly give a constant log2. Do not apply a sigmoid before a loss interface that expects logits, because that changes the input meaning.
Logistic loss is not classification accuracy, and a lower training loss does not guarantee better held-out performance. Class weighting changes the objective and can change probability interpretation. Also, separable data can make unregularized logistic-regression coefficients grow without a finite minimizing coefficient vector, even though the loss approaches its infimum.

Business use

Use logistic loss when the probability assigned to binary outcomes matters, and pair it with metrics tied to the intended decisions. State whether the report uses a sum or mean, the logarithm base, any class weights, and which examples were evaluated.
Inspect very large losses because a few confident mistakes can influence the average strongly. Determine whether those cases reflect mislabeled data, distribution change, or genuine model weakness. If the model will support thresholded actions, select and evaluate the threshold separately rather than assuming the training loss specifies the correct action rule.

Check your understanding

For y=−1 and s=−2, is the signed margin positive or negative?

Answer: Positive, because ys=2. The score supports the observed negative class, giving loss approximately 0.126928.

Related terms

Learn next

Next, convert three logits to probabilities and calculate their losses under both possible labels. Then compare accuracy and average log loss on the same predictions to understand what each metric retains and discards.

Log Loss

Sources

Primary sources: The Aalto Dictionary of Machine Learning, logistic loss definition: https://aaltodictionaryofml.github.io/ADictML_Main.pdf . PyTorch, BCEWithLogitsLoss, official formula and numerical-stability explanation: https://docs.pytorch.org/docs/2.14/generated/torch.nn.BCEWithLogitsLoss.html . Numerical examples use natural logarithms.

Previous: Logistic function Next: Logistic regression