Does DLDA remove correlated features?

No. It ignores their covariance in its model unless a separate preprocessing step removes them.

Can it work with more features than examples?

It avoids inverting a full sample covariance matrix, but meaningful positive variance estimates and appropriate validation are still needed.

How does it differ from diagonal QDA?

DLDA shares feature variances across classes; diagonal QDA permits them to differ by class.

AI Glossary / Probabilistic machine learning / Intermediate

Diagonal linear discriminant analysis

Definition

Diagonal linear discriminant analysis, or DLDA, is a classification method that uses class-specific means and a shared diagonal covariance matrix. It estimates each feature's within-class variance but ignores off-diagonal covariances. Under the Gaussian class model, this produces linear decision boundaries while reducing the number of covariance quantities that must be estimated.

What it means

Ordinary linear discriminant analysis allows features to vary together within each class. DLDA simplifies that model by retaining only the individual feature variances. This can be useful when there are many features but relatively few labeled examples.
The simplification is an assumption or approximation, not a discovery that the real features are independent. Within a Gaussian model, diagonal covariance implies conditional independence given the class. The method can still be used when this is imperfect, but its accuracy and probability estimates need evaluation. Simplicity can reduce estimation problems while sacrificing important information about correlated features.

How it works

Estimate each class mean and a pooled within-class variance for each feature. Choose class priors, commonly estimated from representative training frequencies or specified for the intended population. Avoid zero or unstable variance estimates through an appropriate documented treatment.
For each class, add squared differences between the input and the class mean, dividing each by the shared variance of that feature. Include the prior adjustment and select the class with the smallest adjusted distance. Equivalently, compare linear discriminant scores.
The squared input terms cancel when comparing classes because the variances are shared. This explains why the boundaries are linear even though the distance expression contains squares. Allowing a different diagonal variance for each class gives diagonal quadratic discriminant analysis instead.

Three practical examples

High-dimensional measurement classification: establish a simple baseline when estimating a full covariance matrix would be unstable.
Spectral-data analysis: compare class means using feature-wise variability while testing whether ignored correlations materially affect classification.
Model review: contrast DLDA with ordinary LDA and diagonal QDA to understand the effects of shared variances and covariance simplification.

Think of it this way

Think of comparing an item with category prototypes using several rulers, each scaled to the typical variation of its measurement. DLDA uses the rulers separately and does not account for their tendency to move together. The analogy is limited: its statistical distances are part of a fitted model, and a small distance does not establish a true category without appropriate labels and validation.

Worked example

Consider an original two-feature example with class means A = (0, 0) and B = (2, 4). Shared within-class variances are (1, 4), and the class priors are equal. For a new item x = (1.5, 1), calculate the standardized squared distances.
For A, the distance is (1.5 − 0)² / 1 + (1 − 0)² / 4 = 2.25 + 0.25 = 2.5. For B, it is (1.5 − 2)² / 1 + (1 − 4)² / 4 = 0.25 + 2.25 = 2.5. The item lies on the decision boundary.
The equivalent difference between B's and A's linear scores is 2x_1 + x_2 − 4. At (1.5, 1), this equals zero. At (1.5, 2), it equals 1, favoring B.
If the priors become 0.8 for A and 0.2 for B, add log(0.2 / 0.8), approximately −1.386, to that score difference. At (1.5, 2), the difference becomes approximately −0.386, favoring A. These assumed parameters illustrate the rule; they are not estimates from a real dataset.

Illustrative case study

Fictional case: Juniper Inspection compares classifiers for labeled product measurements. DLDA is easy to fit, but two highly correlated measurements appear to supply duplicated evidence. The team compares its predictions and calibration with a regularized full-covariance alternative.
Feature selection is performed inside each training fold, rather than on the complete dataset before validation. Analysts also check how changing production class frequencies would affect priors. The team retains DLDA as a transparent benchmark while choosing a model on held-out evidence. This case claims no measured operational result.

Common misconceptions and mistakes

Diagonal does not mean all feature variances are equal. Each feature can have its own variance, shared across classes.
Linear does not mean the raw distance calculation contains no squares. The common quadratic terms cancel between classes.
Do not equate ordinary Gaussian naive Bayes with DLDA without checking its variance assumptions. Gaussian naive Bayes often permits class-specific feature variances, yielding a different boundary.
Ignoring covariance also does not eliminate all estimation difficulties. Very small variances, irrelevant features, distribution shifts, and leakage from feature selection can still produce unreliable predictions.

Business use

DLDA can be a useful baseline where labeled data are limited relative to the number of continuous measurements. Its small covariance representation makes the assumptions easier to inspect and can reduce fitting cost.
Evaluate classification errors by class and examine whether fitted probabilities are appropriate for the intended decisions. Compare with regularized alternatives, and keep preprocessing and feature selection inside the training process. If training class frequencies differ from operational frequencies, review prior handling explicitly instead of treating the fitted proportions as universally correct.

Check your understanding

A DLDA model uses feature variances 1 and 9. Which feature contributes more for the same absolute deviation of 3 from a class mean?

Answer: The first contributes 3² / 1 = 9, while the second contributes 3² / 9 = 1. A deviation is less unusual when that feature has greater within-class variability.

Related terms

Linear discriminant analysis
Pooled variance
Gaussian Naive Bayes
Covariance
Regularization
Feature Selection
diagonal quadratic discriminant analysis
class prior

Learn next

Review Gaussian classification and standardized squared distance. Derive why shared covariance makes pairwise score differences linear. Then compare full, diagonal, and regularized covariance estimates, using validation that repeats all feature-selection steps within each training fold.

Sources

Pang, Tong, and Zhao, Shrinkage-based Diagonal Discriminant Analysis and Its Applications in High-Dimensional Data, original research paper and diagonal-rule formulation: https://tiejuntong.github.io/Biometrics2009.pdf Independent source review: Pang, Tong and Zhao section 2.1, equations 2–4 and shared diagonal covariance definition were read.

Previous: Diagnostic checking Next: Dialogue act