Does naive Bayes require independent training rows?

Its feature assumption concerns columns within a class. Dependence between training rows is a separate sampling and evaluation issue.

Can it classify more than two categories?

Yes. Compute a score for every class and normalize across all classes when probabilities are required.

AI Glossary / Statistics and probability / Advanced

Naive Bayes classifier

Definition

A naive Bayes classifier predicts a class by combining its prior probability with feature evidence under a simplifying assumption: features are mutually independent conditional on that class. It compares class scores formed from a prior and a product of class-specific feature probabilities or densities.

What it means

The word naive describes the dependence assumption, not a careless implementation. A classifier can produce useful decisions even when that assumption is imperfect. However, a correct class choice and a reliable probability estimate are different achievements. A displayed score of 90% should not automatically be interpreted as a verified nine-in-ten success rate.

How it works

Training estimates class frequencies and feature distributions within each class. Prediction multiplies the corresponding factors, or adds their logarithms. Gaussian variants model continuous measurements; multinomial variants commonly handle token counts; Bernoulli variants handle binary indicators, including their absence. Smoothing prevents unseen training events from automatically eliminating a class. The representation determines which likelihood calculation is appropriate.

Three practical examples

A help-desk prototype routes tickets to billing or technical support using indicators for selected words. Its evaluation uses later tickets, because duplicate messages distributed across training and testing would exaggerate performance.
A document archive compares a count-based classifier with a presence-based classifier. Repeating a term changes the count representation but leaves its binary presence unchanged, so the two systems answer different statistical questions.
A sensor team uses a Gaussian baseline to classify operating states. Two nearly duplicate measurements trigger an investigation: counting both as independent evidence might make its predictions more confident without adding useful information.

Think of it this way

Imagine a sorter receiving several clue cards about a parcel. It combines the clues as though each provides fresh information after the parcel type is known. If two cards are copies of the same observation, that bookkeeping can exaggerate the evidence. This analogy describes evidence combination, not how a person should reason about every classification problem.

Worked example

Consider an original two-feature routing example. A training set has ten urgent tickets and ten routine tickets. Feature A indicates an outage word; feature B indicates an escalation marker. Eight urgent tickets contain A and six contain B. Two routine tickets contain A and three contain B. Class priors are therefore one-half each.

Use Bernoulli naive Bayes with add-one smoothing for each binary feature. Within a class containing ten tickets, the denominator is twelve: ten observations plus one added count for presence and one for absence. The estimated presence probabilities are A: 9/12 urgent versus 3/12 routine, and B: 7/12 urgent versus 4/12 routine.

For a new ticket containing both features, the urgent score is (1/2)(9/12)(7/12) = 63/288. The routine score is (1/2)(3/12)(4/12) = 12/288. Normalizing gives an urgent model probability of 63/75 = 84%. The equal-cost decision chooses urgent.

Now suppose A is present but B is absent. Absence probabilities for B are 5/12 urgent and 8/12 routine. The scores become 45/288 and 24/288, yielding an urgent probability of 45/69, approximately 65.2%. Ignoring the absent marker would instead give 9/(9+3) = 75%. This demonstrates why a binary model must account for absence as well as presence.

Next create a third feature that is an exact copy of A. For the ticket containing A and B, an incorrectly independence-based calculation multiplies the urgent score by another 9/12 and the routine score by another 3/12. The normalized result is 567/(567+36), approximately 94.0%. No new observation was collected; the confidence increase comes entirely from duplicated evidence.

Finally, suppose urgent tickets make up only one-fifth of a deployment population, while the class-conditional feature probabilities remain unchanged. For A and B present, updated-prior scores are (1/5)(63/144) and (4/5)(12/144). The resulting urgent probability is 63/111, approximately 56.8%. Updating priors would be justified only if that unchanged-conditional assumption were credible. These calculations are demonstrations of a specified model, not empirical calibration results.

Illustrative case study

This fictional case concerns Alder Routing, an imaginary support operation. The team builds the preceding ticket model and initially celebrates its 94% prediction after adding the duplicate outage indicator. A reviewer traces both columns to the same text rule and removes the redundant feature.

The team then freezes its vocabulary and tests on a later period. It records missed urgent tickets separately from unnecessary urgent routing, because the two errors consume different resources. A small group of tickets with unfamiliar language is sent for human review. Management approves a limited pilot only after reviewing those error counts and the estimated workload. The case illustrates an evaluation process; it reports no measured commercial outcome.

Common misconceptions and mistakes

Unconditional independence is not the model's assumption. Two clues can be associated overall because both relate to class membership. Conversely, checking only pairs of features does not establish mutual conditional independence for a larger feature set.

Smoothing does not repair duplicated evidence or make a tiny dataset representative. An unseen word and a zero estimated probability are also different issues: vocabulary handling determines whether that word has a modeled feature at all.

Business use

For the example, suppose routing a routine ticket as urgent costs one workload unit and missing an urgent ticket costs four. Given a calibrated urgent probability p, urgent routing has expected cost 1-p, while routine routing has expected cost 4p. Urgent routing is preferred when p exceeds 0.2. This threshold follows from the stated costs; applying it to an uncalibrated model score requires additional validation.

Check your understanding

Why did duplicating A raise 84% to approximately 94%?

Answer: The model counted the same clue twice.

Why did replacing the prior lower the result to approximately 56.8%?

Answer: Urgent tickets became less common under the revised population assumption.

Related terms

Learn next

Reproduce the two presence-and-absence calculations, then compare held-out error counts and calibration before changing the routing threshold.

Sources

scikit-learn contributors, Naive Bayes, official user guide: https://scikit-learn.org/stable/modules/naive_bayes.html . Supports the classifier framework, variants, smoothing, and probability-estimation caution. All ticket data, calculations, cost assumptions, and the Alder Routing case are original illustrations.

Previous: Naimi - Trehel algorithm Next: Naive semantics