AI Glossary / Deep learning / Advanced

Sigmoid

Definition

A sigmoid is an S-shaped mathematical function. In machine learning, the term usually refers to the logistic sigmoid, sigma(z) = 1/(1 + exp(minus z)). It maps any finite real input to a value strictly between zero and one in exact arithmetic. It is used in logistic regression, neural-network activations, and gating mechanisms.

What it means

A model can produce a raw score with no fixed upper or lower bound. The logistic sigmoid converts that score into a bounded value. In a suitably trained binary classifier, the result is interpreted as the model's estimated probability of the positive class.
That interpretation comes from the model and training objective, not simply from placing a number between zero and one. A sigmoid applied to an arbitrary score does not create a calibrated probability. In a hidden layer or gate, its output may instead represent an activation or an amount of information to pass onward.

How it works

The logistic sigmoid increases smoothly as its input increases. At zero it returns 0.5. Large positive inputs approach one, and large negative inputs approach zero. It preserves score order because it is strictly increasing.
Its derivative is sigma(z) times (1 minus sigma(z)), reaching a maximum of 0.25 at zero and becoming small near the extremes. This saturation can make gradient-based learning through repeated sigmoid activations difficult in some settings.
When p = sigma(z), z equals the natural logarithm of p divided by (1 minus p): the log-odds. Thresholding p is a separate decision step. Software should use numerically stable implementations, since extreme values can cause overflow or rounding in a naive exponential calculation. Finite-precision outputs may round to zero or one even though the exact mathematical function does not reach them for finite inputs.

Three practical examples

• A course-resource classifier transforms a learned score into an estimated probability that a page needs a particular tag. The team separately checks calibration and chooses a decision threshold.
• A multilabel model assigns one sigmoid output to each possible topic. Several topics can receive high values because these outputs are not required to sum to one.
• A gated sequence model uses sigmoid values to regulate a numerical information flow. A gate value near one is not a prediction that an external event will occur with near certainty.

Think of it this way

Imagine a smooth dimmer that approaches its darkest and brightest settings gradually. Around the middle, a small movement changes the brightness noticeably; near either end, the same movement has less effect. The analogy describes sigmoid's shape and saturation, not whether a model's score is trustworthy.

Worked example

Suppose a teaching classifier produces raw score z = 0.8 for a fictional article. The sigmoid output is 1/(1 + exp(minus 0.8)), approximately 0.6900. If interpreted by the trained classifier as a probability, that is a model estimate of about 69%, not a guarantee that the article belongs to the positive class.
At a decision threshold of 0.5, the article would be classified positive. At a threshold of 0.7, it would be classified negative. The equivalent raw-score threshold for 0.7 is log(0.7/0.3), approximately 0.8473, which is above 0.8.
For a second article with z = minus 0.8, the output is approximately 0.3100. These complementary values illustrate sigma(minus z) = 1 minus sigma(z). They do not imply that the two articles are complementary events. The first article's local derivative is approximately 0.6900 times 0.3100 = 0.2139.

Illustrative case study

Fictional Maple Content uses a sigmoid-output model to prioritize articles for metadata review. Staff initially treat a displayed score of 0.9 as evidence that nine out of ten such articles must need correction.
The evaluation team groups comparable predictions on held-out data and compares them with observed outcomes. It discovers that the probability interpretation needs further assessment under the new publishing workflow. The team reports calibration separately from ranking performance and selects a review threshold based on available capacity and error costs. This invented case illustrates why a bounded output alone does not establish reliability.

Common misconceptions and mistakes

Sigmoid is not a hard step function: it produces continuous values rather than only zero and one. A score of zero maps to 0.5, not zero. Independent sigmoid outputs do not generally form a probability distribution summing to one, which distinguishes their usual multilabel use from softmax over mutually exclusive classes.
A monotonic sigmoid cannot improve the ranking of fixed raw scores merely by transforming them. Saturation also does not mean the model is necessarily correct or confident in a well-calibrated sense. Finally, sigmoid may name a broader family of S-shaped curves; specify the logistic formula when that is what is intended.

Business use

Understanding sigmoid helps teams interpret binary prediction scores, discuss thresholds, and avoid confusing internal activations with decision certainty. It is particularly useful when a product presents a probability-like number to users or converts a score into an automated action.
Document what the positive class means and how the model was evaluated. Examine performance at the intended threshold and check whether probability estimates remain useful after a change in data. Keep the mathematical transformation, the learned model, and the operating decision separate in the explanation so users understand what each contributes.

People also ask

Is logistic regression linear or nonlinear?

Its score is linear in the specified features, while sigmoid transforms that score nonlinearly into a probability estimate.

Can sigmoid be used for regression?

It can constrain an output, but the target range and modeling assumptions must justify that choice.

Does a sigmoid output of 0.5 mean the input is neutral?

Only in relation to a defined model interpretation; mathematically it means the raw input to the function was zero.

Check your understanding

Why does raising the example threshold to 0.7 change the label?

Answer: The unchanged output of approximately 0.69 is below the new threshold.

Do three sigmoid outputs have to sum to one?

Answer: No, unless an additional construction imposes that condition.

Why can extreme sigmoid inputs complicate learning?

Answer: The derivative becomes small in saturated regions, reducing the gradient passed through the function.

Related terms

• Logistic regression
• Activation function
• Logit
• Log-odds
• Softmax
• Calibration
• Classification threshold
• Saturation (point operation)
• Vanishing gradient
• saturation
• multilabel classification

Learn next

Calculate sigmoid at zero, two, and minus two, then compare the derivatives at those inputs. Explain why changing a threshold alters decisions without changing the model's ranking of examples, and distinguish that operation from retraining or calibrating the model.

More in Deep learning

• RNN-T - RNN-T stands for recurrent neural network transducer.
• SELU Function - The SELU function, short for scaled exponential linear unit, is a neural-network activation function with a scaled linear positive branch and a saturating...
• Sensor interface layer - A sensor interface layer is the part of a robotic or other sensing system that receives measurements from sensors and presents them to downstream software...
• Sigmoid layer - A sigmoid layer applies the logistic sigmoid function to each element of an input tensor, producing outputs between zero and one for finite inputs in exact...
• Sigmoid perceptron - A sigmoid perceptron is a neural unit that forms a weighted sum of its inputs, adds a bias, and applies a sigmoid activation to produce a smooth output.
• Softplus Function - The Softplus function is a smooth activation function defined by softplus(x) = ln(1 + exp(x)).

Sources

Google Machine Learning Crash Course, sigmoid transformation and logistic-regression log-odds: https://developers.google.com/machine-learning/crash-course/logistic-regression/sigmoid-function Keras, sigmoid activation definition and output range: https://keras.io/api/layers/activations/

Previous: Siegel - Tukey test Next: Sigmoid layer