AI Glossary / Statistics and probability / Advanced
Rectified Gaussian distribution
Definition
In the clipped-normal sense used here, a rectified Gaussian distribution is the distribution of Y = max(0, X) when X is Gaussian. Negative values are reset to zero, while positive values remain unchanged. The resulting distribution combines a point mass at zero with a continuous positive component. The name also appears in other nonnegative Gaussian-related models, including energy-based formulations, so the transformation or density should always be checked rather than inferred from the label alone.
What it means
Rectification is not the same as discarding negative observations and renormalizing the positive remainder. Clipping keeps the probability associated with negative inputs but moves it to zero. Truncation conditions on the input being positive and redistributes probability over that retained region. Taking an absolute value is different again: it reflects negative values onto positive ones. These operations can produce very different zero probabilities, means and likelihoods despite all yielding nonnegative outputs.
How it works
Specify the original Gaussian mean mu and standard deviation sigma, with sigma positive, then apply the transformation max(0, X). The probability at zero is P(X ≤ 0), equal to the standard-normal cumulative probability at -mu/sigma. For positive y, the continuous density is the original Gaussian density at y; it is not renormalized to integrate to one by itself. For y at least zero, the cumulative probability is the original Gaussian cumulative probability at y. Below zero, it is zero.
Three practical examples
Example 1: A theoretical analysis models a Gaussian input passed through a rectified linear unit. Example 2: A measurement system reports negative noisy readings as zero, producing a clipped distribution under a Gaussian noise model. Example 3: A simulation studies a nonnegative quantity created by flooring a latent normal variable at zero. These examples require justification for the latent Gaussian assumption. Nonnegative data alone do not establish that a rectified Gaussian is an appropriate model.
Think of it this way
Think of a floor at zero. Values above the floor retain their positions, while all values below it are placed on the floor itself. A pile therefore accumulates exactly at zero. By contrast, truncation removes the below-floor observations and looks only at what remains. The pile is not a cosmetic detail: it changes how probabilities and likelihoods must be calculated.
Worked example
Let X have a standard normal distribution and define Y = max(0, X). Symmetry gives P(Y = 0) = 0.50. The positive continuous part also has total probability 0.50. The mean is 1/square root of (2 pi), approximately 0.39894. The second moment is 0.50, because exactly half of the standard normal's second moment lies on the positive side. Variance is therefore 0.50 - 1/(2 pi), approximately 0.34085. The probability Y exceeds one is P(X > 1), approximately 0.15866. If X were instead conditioned to be positive, the resulting half-normal mean would be square root of (2/pi), approximately 0.79788, and there would be no point mass at zero. The factor-of-two difference in means highlights the distinction between clipping and truncation.
Illustrative case study
Fictional case study: Silver Sensors exports negative background-corrected readings as zero. Analysts initially fit a positive truncated normal model and ignore the many zero observations. A reviewer identifies the export rule and proposes a model that includes the zero mass, while checking whether Gaussian latent errors are plausible. This fictional case illustrates matching a model to a measurement transformation. It does not claim that clipping is good measurement practice or that this distribution fits any real sensor.
Common misconceptions and mistakes
Common mistakes include normalizing the positive density as though no zero mass existed, treating rectification as an absolute-value transformation, and assuming all uses of the name denote the same distribution. A histogram spike at zero can have several causes, so it does not prove Gaussian clipping. Observed rounded zeros can also include small positive values. In multiple dimensions, componentwise rectification produces probability on boundary faces as well as in the positive interior, complicating a simple density-only description.
Business use
Business use: The concept helps analysts understand outputs from thresholded measurements, simulation rules and rectified model components. Record whether zero means a genuine value, a censored reading, rounding or an explicit transformation. Choose a likelihood that accounts for the actual observation process. Compare alternative nonnegative models and check sensitivity to the latent distribution. Explain the discrete mass separately from the continuous density when presenting charts or calculated probabilities.
People also ask
Is this the same as a truncated normal?
No; clipping moves negative probability to zero, while truncation conditions on retained values. Q: Is it entirely continuous? A: No, in the clipped-normal sense it has a point mass at zero. Q: Can the name mean something else in research? A: Yes; some nonnegative energy-based Gaussian models use the same label and require separate definitions.
Check your understanding
Why does the standard-normal example have half its probability at zero?
Answer: Half of the Gaussian inputs are nonpositive and all are mapped to zero.
Why is the positive density not doubled?
Answer: Its integral should remain one-half, with the other half represented by the separate point mass.
Related terms
- Gaussian distributions provide the latent normal input
- x)
Learn next
Simulate a standard Gaussian and compare clipping, truncation and absolute-value transformations. Calculate their zero probabilities and means separately. Then vary the Gaussian mean relative to its standard deviation and observe how the zero mass changes. When reading a paper, inspect its mathematical definition before importing results from another model with the same name.
More in Statistics and probability
- Rational subgroups - Rational subgroups are deliberately formed groups of observations used in statistical process control so that within-subgroup variation represents a...
- Realized variance - Realized variance is an ex-post measure of variation calculated by summing squared returns over subintervals within a specified period.
- Recall bias - Recall bias is systematic error arising when people inaccurately or incompletely remember and report past events or experiences.
- Recurrent event analysis - Recurrent event analysis studies events that can occur more than once for the same observational unit over time.
- Recursive partitioning - Recursive partitioning builds a model by repeatedly dividing a dataset or predictor space into smaller regions.
- Reference class problem - The reference class problem is the difficulty of deciding which group of comparable cases should inform a probability judgment about a particular case.
Sources
Sources: Beauchamp, On numerical computation for the distribution of the convolution of N independent rectified Gaussian variables: https://numdam.org/item/JSFS_2018__159_1_88_0.pdf ; Socci, Lee and Seung, The Rectified Gaussian Distribution: https://proceedings.neurips.cc/paper_files/paper/1997/file/28fc2782ea7ef51c1104ccf7b9bea13d-Paper.pdf . The first supports the clipped-normal sense; the second explicitly discusses a distinct nonnegative energy-based formulation. The calculations and fictional sensor case are original.
Previous: Rectification Next: Rectified Linear Unit