Yes. It uses known class labels to fit its class statistics or discriminant directions.

Does it assume features are independent?

No. A shared covariance matrix can contain nonzero correlations.

How many discriminant coordinates can it produce?

At most min(d,K−1) for d features and K classes, with fewer possible when the relevant structure has lower rank.

AI Glossary / Mathematics and optimization / Intermediate

Linear discriminant analysis

Definition

Linear discriminant analysis, or LDA, is a supervised method used for classification and for finding linear projections that separate known classes. Its common probabilistic classifier models each class with a Gaussian distribution and assumes a shared covariance matrix. This shared covariance produces linear boundaries between the class scores.

What it means

LDA uses class labels during fitting. It examines how class centers differ relative to variation within classes, rather than simply seeking directions with large overall variation. This distinguishes its dimensionality-reduction role from ordinary principal component analysis.
The name covers related classification and projection viewpoints. Fisher's discriminant criterion seeks directions with strong between-class separation relative to within-class spread. The Gaussian classifier supplies a probabilistic interpretation and incorporates class prior probabilities.
Here LDA means linear discriminant analysis, not latent Dirichlet allocation, the topic-modeling method that shares the abbreviation. It is also not a guarantee that the classes truly follow Gaussian distributions or that their covariances are identical in the data.

How it works

Estimate each class mean, the class priors, and a pooled within-class covariance. With an invertible shared covariance Σ, a class score can be written δₖ(x)=xᵀΣ⁻¹μₖ−½μₖᵀΣ⁻¹μₖ+log πₖ. Choose the class with the greatest score under the usual equal-error-cost decision rule.
The quadratic term involving x alone cancels when classes are compared because their covariance is shared. The remaining score differences are affine in x. Different covariance matrices generally lead instead to quadratic discriminant analysis.
For supervised projection, the available discriminant subspace has dimension at most the smaller of the feature dimension and K−1, where K is the number of classes. Estimating covariance can become difficult with few observations or redundant features; regularization or suitable numerical methods may be needed.

Three practical examples

A labeled measurement dataset can use LDA as a classification baseline with a compact decision rule.
Analysts can project labeled observations into one or two discriminant coordinates to inspect how known classes separate.
A high-dimensional classification workflow can examine covariance regularization when limited training data makes an unregularized estimate unreliable.

Think of it this way

Think of comparing how far apart two groups are while accounting for how much each measurement normally varies within the groups. A difference along a noisy direction should not automatically dominate a smaller but more consistent difference. The analogy is limited because LDA uses a specific linear and covariance-based construction, not an unrestricted judgment about group similarity.

Worked example

Consider an original two-class model with means μ_A=(0,0) and μ_B=(2,2). Let both classes have covariance Σ=diag(4,1) and equal priors. Its inverse is diag(1/4,1), so Σ⁻¹(μ_B−μ_A)=(0.5,2).
The class-score difference is δ_B(x)−δ_A(x)=0.5x₁+2x₂−2.5. The boundary is therefore 0.5x₁+2x₂=2.5. The second feature receives greater weight because a two-unit mean difference is more informative relative to its smaller within-class variance in this model.
For x=(4,0), the score difference is −0.5, so the classifier selects A. Ordinary Euclidean distance would favor B: the distances to A and B are 4 and √8. LDA instead accounts for covariance. The squared Mahalanobis distances are 4 to A and 5 to B, so A is the closer class under the shared covariance metric.
For x=(0,2), the score difference is 1.5, so B is selected. Now change the priors to π_A=0.8 and π_B=0.2 while leaving means and covariance fixed. Add log(0.2/0.8)=−log4, approximately −1.386, to the score difference. At x=(2,1), the equal-prior difference was 0.5; it becomes approximately −0.886, changing the decision from B to A. Priors therefore influence the boundary, not only a label displayed after classification.

Illustrative case study

In a fictional materials-testing project, an analyst compares LDA with other classifiers using labeled measurements from known product types. A projection separates some types clearly but leaves others overlapping. The team examines whether the overlap reflects measurement noise, meaningful similarities, or inadequate features.
It fits preprocessing and covariance choices using training data and evaluates the resulting classifier on held-out observations. The plot is treated as a view of a supervised model, not independent proof that the product types form naturally separated clusters. This hypothetical scenario does not claim a measured production outcome.

Common misconceptions and mistakes

Do not confuse linear discriminant analysis with latent Dirichlet allocation or with unsupervised PCA. Do not describe a squared Mahalanobis distance as the unsquared distance when reporting numerical values.
Shared covariance does not require independent features: off-diagonal covariance terms may represent within-class correlations. Also, an apparent separation in a projection fitted using labels is not an unbiased evaluation of classification accuracy. Fitting the projection on the entire dataset before splitting can leak information into evaluation. A covariance estimator with better statistical properties does not automatically guarantee better classification in every application.

Business use

LDA can provide a useful reference model when numerical features and labeled examples are available. Review whether class distributions and covariance structure make its assumptions plausible enough for the intended purpose, and compare actual predictive performance rather than relying on a visually attractive projection.
Record prior probabilities and any class-cost policy, especially when the training sample's class proportions differ from deployment. Evaluate probability estimates as well as class decisions if users will interpret them as confidence. When features greatly outnumber examples, pay particular attention to covariance estimation and regularization.

Check your understanding

With five features and three classes, what is the maximum number of LDA discriminant coordinates?

Answer: min(5,3−1)=2. This is a structural upper bound, not a promise that two useful directions exist.

Related terms

Linear classifier
Covariance matrix
Mahalanobis distance
Principal component analysis
Fisher discriminant
class prior
quadratic discriminant analysis
supervised dimensionality reduction

Learn next

Next, compare Euclidean and covariance-adjusted distances for the same two class means. Then contrast an LDA projection with PCA, checking which method uses the class labels and what each objective seeks to preserve.

Sources

Primary implementation reference: scikit-learn, Linear and Quadratic Discriminant Analysis, official mathematical formulation and dimensionality-reduction documentation: https://scikit-learn.org/stable/modules/lda_qda.html . The two-class covariance and prior calculations are original illustrations.

Previous: Linear constraints Next: Linear Dynamical System