Is the forward pass softened?

Not necessarily. In a straight-through construction it can retain the hard value while only the backward rule changes.

Does it apply only to random operations?

No. It is also used with deterministic rounding and thresholds.

Is it the same as a score-function estimator?

No. Score-function methods differentiate an expected stochastic objective using a different identity; their bias and variance properties differ.

AI Glossary / Probabilistic machine learning / Intermediate

Straight-through estimator

Definition

A straight-through estimator is a training rule that substitutes a usable derivative for the derivative of a hard or discrete operation during backpropagation. The forward calculation still uses the hard operation, while the backward calculation passes a surrogate gradient through it. The simplest version treats the operation as having derivative one during the backward pass.

What it means

Rounding, binary thresholds, and discrete selections can block ordinary gradient-based training. Their outputs may remain unchanged over a range of inputs, giving zero derivatives almost everywhere, with derivatives undefined at switching boundaries.

A straight-through estimator deliberately changes the backward rule so earlier parameters receive a learning signal. It is a practical approximation, not the exact derivative of the hard forward computation. Different variants use different surrogate derivatives, such as an identity or a clipped function. The chosen variant is part of the method and can strongly affect training.

How it works

Compute the model output using a hard decision, such as a binary activation. Evaluate the downstream loss and obtain its derivative with respect to that activation. Instead of multiplying by the hard operation's true derivative, multiply by the selected surrogate derivative.

The resulting signal updates the underlying continuous parameter, often called a latent or full-precision parameter in quantized networks. The next forward pass applies the hard operation again, so the deployed behavior remains discrete.

This construction can be written with a stop-gradient operation: use the desired hard value in the forward pass while allowing derivatives from a chosen smooth expression to pass backward. The precise implementation must be checked because a seemingly equivalent expression may yield a different gradient. Straight-through estimates are generally biased and need not define the gradient of the original objective. Their usefulness is judged through the actual hard model's performance.

Three practical examples

1. Quantized training: use rounded or low-bit values during the forward pass while updating continuous underlying weights through a surrogate gradient. Evaluate the final quantized network, not only a floating-point counterpart.

2. Binary gates: train a model that selects whether a computation is active. Check the realized gate usage and system cost instead of assuming sparse activations automatically reduce runtime.

3. Discrete representations: learn compact binary codes while allowing gradients to reach their encoder. Measure retrieval or reconstruction with the discrete codes that will actually be stored and used.

Think of it this way

Think of adjusting a thermostat with a switch that is either off or on. Tiny adjustments may not change the switch position. A straight-through rule supplies a directional hint for the adjustment anyway. That hint can help cross a useful threshold, but it is not the physical slope of the switch's hard response.

Worked example

Consider an original deterministic example. Define h(a) = 1 if a is positive and 0 otherwise. The target is 1, and the loss is one half times (h minus 1) squared. Start with a = minus 0.2, so h = 0 and the loss is 0.5.

The true derivative of the loss with respect to a is zero in a neighborhood around minus 0.2 because the hard output remains zero there. Under the identity straight-through rule, however, the surrogate derivative of h with respect to a is 1. The backward signal is therefore (h minus 1) times 1 = minus 1.

With learning rate 0.3, gradient descent updates a to minus 0.2 minus 0.3 times minus 1 = 0.1. The next hard output is 1, and the loss becomes zero. With learning rate 0.1 instead, the updated value is minus 0.1; the hard output remains zero and the loss remains 0.5.

This example shows how a surrogate signal can move a parameter across a threshold and why a nonzero surrogate update need not immediately change the hard loss. It does not prove that the same rule improves an arbitrary network or is an unbiased gradient estimator.

Illustrative case study

Fictional case: Birch Devices trains a small classifier intended for a low-bit inference engine. Its team keeps continuous training weights but applies a quantization operation in the forward pass. It compares an identity straight-through rule with a clipped surrogate and records the exact clipping range.

Engineers validate exported quantized models on the actual inference system and inspect whether weights saturate near representable limits. They separate accuracy changes caused by training from differences in the deployment implementation. If the surrogate trains successfully but the exported model performs poorly, they investigate the mismatch rather than relying on the training curve. This fictional case contains no reported accuracy or speed improvement.

Common misconceptions and mistakes

A straight-through estimator does not make a discontinuous operation differentiable in the ordinary mathematical sense. It is not automatically unbiased and does not guarantee descent of the hard objective at each update. Forward values and backward derivatives must be distinguished. Also avoid assuming that all straight-through estimators use derivative one everywhere; clipped and smooth-surrogate variants behave differently.

Business use

The business value is enabling training for models with discrete operations that match deployment constraints. The tradeoff is a learning signal that may misrepresent the true objective, creating sensitivity to initialization, scale, and saturation. Teams should document both the forward operation and backward rule, validate the final discrete model, and compare against suitable alternatives. A smaller model file or fewer nominal operations does not alone establish practical performance gains.

Check your understanding

Answer: 1.

What is the true local derivative in the example at minus 0.2?

Answer: It is zero.

Answer: 2.

What gradient does the identity straight-through rule propagate?

Answer: It propagates minus 1 for the specified loss.

Answer: 3.

Why does the smaller update leave the hard loss unchanged?

Answer: The continuous parameter has not crossed the switching threshold.

Related terms

REINFORCE
Surrogate gradient
quantization-aware training
binary neural network
stop gradient
discrete latent variable
continuous relaxation

Learn next

Compare true and surrogate derivatives for a rounding function. Then study the effects of clipping and continuous latent weights, followed by alternative estimators for expected losses involving stochastic discrete variables.

Sources

Bengio, Léonard and Courville, Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation, discusses the straight-through heuristic and alternative gradient estimators: https://arxiv.org/pdf/1308.3432

The threshold calculation and Birch Devices scenario are original teaching examples; the scenario is fictional.

Independent review consulted Section 4, including Equation 13 and the bias caveat, in the original author manuscript.

Previous: Storage model Next: Straight-through processing