Must the full dataset be Gaussian?

No. The assumption is feature-by-feature within each class.

Can correlated features still produce useful classifications?

Sometimes, but performance and calibration must be checked rather than assumed.

Are the outputs guaranteed to be reliable probabilities?

No. They are model-based posterior estimates whose quality depends on assumptions, estimation and the deployment data.

AI Glossary / Statistics and probability / Intermediate

Gaussian Naive Bayes

Definition

Gaussian Naive Bayes is a classification method that combines Bayes' rule with a Gaussian model for each numerical feature within each class and a conditional-independence assumption across features. It estimates class-specific means and variances, combines their densities with class prior probabilities and predicts a class from the resulting posterior scores. Naive refers to the simplifying independence assumption, not to an absence of probabilistic reasoning. The method is computationally simple, but its probability estimates can be poorly calibrated when its assumptions are inaccurate.

What it means

The model asks how compatible an observed feature vector is with each class. It treats the features as independent after conditioning on the class, which is different from requiring unconditional independence in the overall dataset. Each feature can have a different mean and variance for each class. A class prior also matters, so similar likelihoods need not produce equal posterior probabilities. The entire dataset need not look Gaussian; the assumption concerns each individual feature's class-conditional distribution. Multimodal or highly skewed within-class features can challenge that approximation.

How it works

From labeled training data, estimate class priors and a mean and variance for every feature-class combination. For a new vector x, compute a score proportional to P(class c) times the product over features of their normal densities under c. Use log scores to avoid numerical underflow, then normalize if posterior probabilities are needed. Variance smoothing can prevent singular calculations for nearly constant features. Select a class using a decision rule appropriate to the task, and evaluate on data excluded from training and preprocessing. Missing values, dependence, class imbalance and changes in the data distribution need explicit handling.

Three practical examples

Baseline modeling: establish a simple numerical-feature classifier before testing more complex methods.
Sensor classification: distinguish labeled operating conditions using class-specific feature summaries when those assumptions are useful approximations.
Incremental analysis: update sufficient statistics as labeled observations arrive, using an implementation designed for that workflow and checking for distribution changes.

Think of it this way

Think of collecting several pieces of evidence and multiplying their class-specific compatibility scores. The shortcut assumes the pieces do not provide overlapping information once the class is known. If two pieces are essentially duplicates, multiplying them can count the same evidence twice.

Worked example

Suppose two hypothetical classes A and B have equal priors. Both features have variance one within each class. A's mean vector is [0,0], B's is [2,2], and a new observation is [1,0]. Its summed squared deviations are one under A and five under B. The shared Gaussian normalization factors cancel, so the likelihood ratio A to B is exp[(5 - 1)/2] = exp(2), approximately 7.389. The posterior probability assigned to A is 1/[1 + exp(-2)], approximately 0.880797. This result follows from the stipulated conditional-independence and Gaussian assumptions. To illustrate dependence risk separately, one feature yielding likelihood ratio exp(2) would give the same 0.880797 posterior with equal priors. Duplicating that identical feature and incorrectly treating the copy as independent would square the ratio, producing approximately 0.982014 without adding genuine information.

Illustrative case study

Fictional case: Reed Monitoring Lab uses Gaussian Naive Bayes as a baseline for labeled machine states. Analysts discover that two input columns are different exports of the same underlying measurement. They remove the duplication and reevaluate both classification performance and probability calibration on later runs. They compare against alternative models where class-conditional shapes differ substantially from normal. This invented case illustrates careful evaluation without claiming that the simplest method or any particular replacement will necessarily be best.

Common misconceptions and mistakes

Do not interpret conditional independence as a claim that the raw features are uncorrelated across all observations. Do not assume a good classification accuracy score proves calibrated probabilities. Correlated or duplicated features can lead to exaggerated evidence. A normal density evaluated at a point is not the probability of that exact continuous value, although densities can be used consistently in Bayes' rule. Zero variance requires numerical care. Gaussian Naive Bayes differs from multinomial or Bernoulli variants, which use different feature likelihoods. It also differs from a general Gaussian classifier with full within-class covariance matrices.

Business use

Businesses can use Gaussian Naive Bayes as a transparent baseline when numerical features and labeled examples are available. Check data definitions, leakage and repeated observations before training. Report performance on the decision-relevant classes and assess calibration if probabilities influence actions. Preserve class priors and preprocessing choices, and monitor whether class frequencies or feature distributions change after deployment. The small parameter count can be useful when data are limited, but simplicity does not remove the need for validation. If misclassification costs differ, use an appropriate decision threshold rather than assuming the most probable class is always the best action.

Check your understanding

Two identical columns are included as separate features. Why can that be problematic?

Answer: The conditional-independence product treats them as separate evidence, potentially overstating confidence even though the second column adds no new information.

Related terms

Conditional independence
Probability calibration
Classification
Naive Bayes
Bayes' rule
class prior
Gaussian likelihood
variance smoothing
generative classifier

Learn next

Learn how priors and likelihoods combine in Bayes' rule, then compare Gaussian, Bernoulli and multinomial feature models. Study calibration and dependence diagnostics before using classifier probabilities in operational decisions.

Sources

Scikit-learn, Naive Bayes and Gaussian Naive Bayes: https://scikit-learn.org/stable/modules/naive_bayes.html Scikit-learn, GaussianNB implementation reference: https://scikit-learn.org/stable/modules/generated/sklearn.naive_bayes.GaussianNB.html

Previous: Gaussian mixture model Next: Gaussian process