Can the method use continuous measurements directly?

The categorical model expects discrete categories. Discretization is possible but changes the representation and can lose information.

What does Laplace smoothing do?

It adds one to each category count under the usual convention, producing nonzero probabilities for categories represented in the defined category set.

Are category numbers ordered?

They act as identifiers in this model, so their numerical distances are not used as measurement distances.

AI Glossary / Statistics and probability / Intermediate

Categorical Naive Bayes

Definition

Categorical Naive Bayes is a classification method that models each input feature as a categorical variable and combines its class-conditional probabilities using Bayes' rule. Its simplifying assumption is that the features are conditionally independent given the class. It is a useful baseline for discrete attributes, but its probability estimates and independence assumptions require validation.

What it means

Each feature takes one value from a set of categories, such as contact channel or reported issue type. The model learns how often each category occurs within each class and combines that evidence with the class prior. Category codes identify labels rather than numerical distances: code 4 is not inherently twice code 2. The method differs from multinomial Naive Bayes, which commonly models counts across features, and from Gaussian Naive Bayes, which models continuous feature values using normal distributions.

How it works

For each class, estimate its prior probability and count category occurrences separately for every feature. With additive smoothing alpha, a category probability is its class-specific count plus alpha, divided by the class count plus alpha times the number of categories for that feature. For a new observation, multiply the class prior by the relevant feature probabilities, or add their logarithms for numerical stability. Compare class scores and normalize them when model probabilities are required. The categorical encoding and allowed category sets must be consistent between training and prediction. Plan how unknown or missing categories will be represented rather than assuming smoothing creates an unlimited new-category mechanism.

Three practical examples

Support routing: classify tickets from discrete attributes such as selected issue type, contact channel and product family.
Quality screening: build a baseline classifier using categorical inspection findings, comparing its errors with more flexible models.
Educational modeling: demonstrate how category counts, smoothing and priors influence a prediction without requiring a complex neural architecture.

Think of it this way

Think of combining several small frequency tables, one for each clue, to decide which category of outcome fits best. The shortcut is to treat those clues as independent once the outcome is known. Repeated versions of the same clue can then make the evidence appear stronger than it really is.

Worked example

Suppose a training set has ten priority tickets and ten routine tickets. A severity feature has three categories, and a channel feature has two. Among priority tickets, six have severity “high” and seven arrive by phone. Among routine tickets, two have severity “high” and three arrive by phone. Use equal class priors and Laplace smoothing, alpha = 1. For a high-severity phone ticket, the priority score is 0.5 × (6 + 1)/(10 + 3) × (7 + 1)/(10 + 2) = 28/156, approximately 0.17949. The routine score is 0.5 × 3/13 × 4/12 = 6/156, approximately 0.03846. Normalizing gives priority probability 28/(28 + 6) = 14/17, about 82.35%, within this fitted model. The severity denominator uses three categories, while the channel denominator uses two. These scores are not observed joint frequencies: the product follows the conditional-independence assumption. An 82.35% model output is not evidence that future tickets with this score will be priority cases exactly that often.

Illustrative case study

Fictional case: Pine Desk tests Categorical Naive Bayes for routing incoming service requests. Analysts discover that selected issue category and a second category field encode almost the same information, making probability outputs overly confident. They compare feature variants, check calibration on later tickets and evaluate misrouting costs separately from overall accuracy. Unknown product codes receive a documented handling rule. Human staff review ambiguous requests during the pilot. This fictional case describes model evaluation and does not claim a measured reduction in response time.

Common misconceptions and mistakes

The naive assumption is conditional independence given the class, not unconditional independence in the overall dataset. Integer encoding does not convert a nominal feature into a measured quantity. One-hot encoding followed by an arbitrary Naive Bayes variant need not reproduce the categorical model. Smoothing prevents zero estimates for represented categories but does not correct mislabeled data or dependency between features. Good classification accuracy does not guarantee calibrated probabilities. High-cardinality identifiers can create sparse, misleading patterns, and fitting encoders or selecting features using test labels introduces leakage.

Business use

For businesses, the model can provide a transparent and computationally economical reference point for categorical classification tasks. Define the decision, class labels and cost of different errors before choosing a threshold. Preserve category mappings and check whether deployment class frequencies differ from training frequencies. Evaluate precision, recall or cost-sensitive measures as appropriate, and assess calibration when probabilities drive decisions. A more elaborate model should justify its complexity through relevant validation; a simple baseline still needs data-quality checks and ongoing monitoring.

Check your understanding

Q: Why does the worked example use 13 in one denominator and 12 in the other?

Answer: A: Each class contains ten observations, but severity has three categories and channel has two. With alpha equal to one, smoothing adds three and two total pseudo-counts respectively.

Related terms

Bayes theorem
Categorical variable
Conditional independence
Laplace smoothing
Multinomial Naive Bayes
class prior
Bernoulli Naive Bayes and probability calibration
assumptions and output interpretation

Learn next

reproduce the two-class calculation with explicit frequency tables. Change the class priors and observe when the preferred class changes. Then duplicate a feature and inspect how the predicted confidence changes, illustrating why redundant evidence can distort probability estimates.

Sources

Sources: scikit-learn, categorical Naive Bayes formulation and comparison with other variants: https://scikit-learn.org/stable/modules/naive_bayes.html ; scikit-learn, CategoricalNB estimator, smoothing and category handling: https://scikit-learn.org/stable/modules/generated/sklearn.naive_bayes.CategoricalNB.html ; Poole and Mackworth, Bayesian learning and naive Bayes assumptions: https://www.cs.ubc.ca/~poole/aibook/3e/html/ArtInt3e.Ch10.S2.html .

Previous: Categorical data Next: Categorical variable