AI Glossary / Probabilistic machine learning / Intermediate

Gaussian discriminant analysis

Definition

Gaussian discriminant analysis (GDA) is a classification approach that models the feature distribution within each class as Gaussian and combines those class-conditional densities with class prior probabilities using Bayes' rule. The resulting posterior probabilities support class predictions. A common version shares one covariance matrix across classes, producing linear decision boundaries.

What it means

Rather than directly fitting a boundary or a conditional class probability, GDA describes what feature measurements tend to look like within each labeled class. A class mean locates its center, and a covariance matrix describes spread and relationships among features. Prior probabilities describe how common the classes are before seeing the features.

The Gaussian assumption applies within each class, not necessarily to the overall population. Combining several class-specific Gaussian distributions can produce a multimodal overall distribution. GDA is called generative because it models class priors and feature likelihoods; this classification use does not require a modern content-generation system.

How it works

From labeled data, estimate each class's mean vector and prior probability. In the shared-covariance version, estimate a pooled within-class covariance matrix. For a new feature vector, calculate each class's Gaussian density and multiply by that class's prior. Normalize these products to obtain posterior class probabilities.

Under equal error costs, selecting the largest posterior gives the classification decision. Different costs can justify a different decision rule. In practice, calculations commonly use log densities to improve numerical stability.

Shared covariance causes quadratic feature terms to cancel in pairwise log-posterior comparisons, yielding linear boundaries. Allowing a separate covariance for each class gives the related quadratic discriminant analysis formulation. Some sources use GDA broadly for both; the shared-covariance version is the main example here. Neither version assumes features are independent unless extra covariance restrictions are imposed.

Three practical examples

1. Quality inspection: classify parts into labeled condition categories using a small set of continuous measurements. Check whether each category is reasonably represented by the assumed shape and whether error costs differ.

2. Sensor classification: combine multiple measurements whose correlations help distinguish operating modes. A covariance model can capture relationships that a feature-independent model would omit.

3. Analytics baselines: compare a simple probabilistic classifier with more flexible alternatives. GDA can provide an understandable reference model when class-conditional structure and the available sample size make it plausible.

Think of it this way

Think of comparing a new measurement with several clouds of labeled examples, considering each cloud's center, shape and prevalence. A point near a rare cloud may still be less likely to belong to it than to a common one. The analogy is geometric; a Gaussian density is not a count of nearby points, and posterior probabilities depend on the complete model.

Worked example

Consider one continuous feature and two classes. Class 0 has mean 0, class 1 has mean 2, and both have variance 1. Their priors are 0.75 and 0.25. For a new point x = 1.5, the Gaussian density ratio p(x | class 1)/p(x | class 0) is exp(2x − 2) = exp(1), approximately 2.7183.

The prior odds for class 1 are 0.25/0.75 = 1/3. Posterior odds are therefore 2.7183/3, approximately 0.9061. Converting odds to probability gives 0.9061/(1 + 0.9061), approximately 0.4754. Under equal misclassification costs, the model selects class 0 even though x is closer to class 1's mean.

The equal-posterior boundary solves exp(2x − 2)/3 = 1, giving x = (2 + log 3)/2, approximately 1.5493. With equal priors, the boundary would instead be the midpoint 1. The example uses stipulated distributions and natural logarithms; it is a classification calculation, not an estimate from collected data.

Illustrative case study

Fictional case: Cedar Components tests a GDA baseline for classifying inspection measurements. The engineer notices that a rare defect category has a small estimated prior, so some borderline measurements are assigned to the common category despite being closer to the defect mean.

The team reviews the cost of missed defects, validates predicted probabilities and compares decisions under an appropriate operating threshold. It also checks whether the rare category contains multiple subtypes that a single Gaussian misses. This fictional case illustrates model and decision review, not a claim of proven inspection reliability.

Common misconceptions and mistakes

Closeness to a class mean alone does not determine the result. Covariance and class priors also matter. A density value is not a point probability for continuous features, and it should not be interpreted as a percentage without the Bayes normalization.

Gaussian features within classes need not be independent. A shared covariance does not mean classes have the same mean. Conversely, allowing separate covariance matrices adds flexibility but also increases estimation demands. With many features and few observations, covariance estimates can become unstable or singular, requiring appropriate regularization or dimensionality reduction.

Business use

Evaluate class-specific errors, probability calibration and performance on representative future data. A high overall accuracy can hide poor behavior for a rare class. Keep the training sample's class proportions distinct from the intended deployment population when sampling or rebalancing has changed them.

Document the covariance assumption and any regularization. Review outliers and multimodal classes rather than relying only on a fitted Gaussian contour. If the decision carries unequal costs, communicate the threshold choice separately from the probability model. A simple model is useful when its assumptions and limitations are understood, not merely because it is easy to fit.

Check your understanding

Why does the worked example classify x = 1.5 as class 0 despite its closeness to class 1's mean?

Answer: The feature likelihood favors class 1, but its smaller prior probability offsets that advantage. The posterior probability for class 1 remains below one-half.

Related terms

Linear discriminant analysis
Multivariate Gaussian
Covariance matrix
Logistic regression
Generative classifier
quadratic discriminant analysis
Bayes' rule
class prior

Learn next

Vary the class priors in the one-dimensional example and observe the boundary movement. Then compare shared and separate covariance matrices in two dimensions, including how sample size affects the reliability of covariance estimation.

Sample size

Sources

Stanford CS229, Generative Learning Algorithms, Gaussian Discriminant Analysis: https://cs229.stanford.edu/notes_archive/cs229-notes2.pdf

Independent review consulted Stanford notes sections 1.2–1.3 on shared covariance, estimation and logistic-form posterior probabilities. Original odds, posterior and boundary calculations verified; case fictional.

Previous: Gaussian blur Next: Gaussian error model