Is CNB only a binary classifier?

No. Each class can be compared with the aggregate of all other classes.

Does it guarantee better rare-class performance?

No. The result depends on data and configuration.

Can ordinary standardized features be used directly?

Not if standardization creates negative values incompatible with the model's nonnegative feature requirements.

AI Glossary / Statistics and probability / Intermediate

Complement Naive Bayes

Definition

Complement Naive Bayes, or CNB, is a classification method derived from multinomial naive Bayes that estimates each class's weights using examples outside that class. A new observation is assigned according to how poorly it matches the competing class complements. The approach is especially associated with text classification and imbalanced datasets. It can improve on ordinary multinomial naive Bayes, but better performance is an empirical question rather than a guarantee.

What it means

A small class may have too few examples to estimate stable word frequencies directly. Its complement combines the examples belonging to all other classes, often providing more information about what the class is not.
The classifier uses that complementary evidence to construct a decision score. This is not the same as taking one minus an ordinary class probability. In a multiclass problem, each complement contains a different mixture of classes, and the scoring and normalization conventions must be understood before interpreting outputs.

How it works

Represent each example with nonnegative features, commonly word counts or nonnegative text weights. Fit the vocabulary and preprocessing using training data so information from evaluation data does not leak into the model.
For each class c, aggregate feature values from training examples whose labels are not c. Add smoothing and normalize by the total smoothed feature mass in that complement. Taking logarithms produces weights. Some forms then normalize the weight vector again; implementations differ on whether that second normalization is enabled.
Score a new feature vector against each complement. With logarithmic complement weights and the corresponding minimum-score convention, select the class whose complement gives the poorest match. Equivalent implementations may reverse weight signs and maximize instead, so formulas must be read consistently.
Tune smoothing and other supported choices using appropriate validation. Inspect per-class precision and recall, particularly for rare classes. A normalized output returned as a probability should not automatically be treated as calibrated confidence. Test the complete text pipeline under the expected distribution of future documents.

Three practical examples

A support-ticket classifier evaluates CNB when some issue categories have many fewer examples than others.
A document-routing system uses sparse nonnegative text features and compares CNB with multinomial naive Bayes on held-out material.
A research team studies whether complement-based weights improve rare-class recall without creating unacceptable false positives.

Think of it this way

Think of identifying a category partly by learning the vocabulary of everything outside it. A message that fits those alternatives poorly may be a good candidate for the category. The method still needs representative examples and a precise scoring rule; being unlike one alternative does not automatically prove membership in another.

Worked example

Consider invented counts for two terms, lens and bread, across three classes. Class A contributes (8, 1), class B contributes (1, 8), and class C contributes (1, 1). Use additive smoothing of one per term and omit the optional second weight normalization for this example.
For A, the complement contains B and C, with counts (2, 9). After smoothing, its term probabilities are (3/13, 10/13). For B, the complement gives (10/13, 3/13). For C, A and B contribute (9, 9), giving smoothed probabilities (10/20, 10/20) = (0.5, 0.5).
A new document contains two occurrences of lens and none of bread, so its vector is (2, 0). The logarithmic complement scores are 2 ln(3/13) ≈ −2.933 for A, 2 ln(10/13) ≈ −0.525 for B, and 2 ln(0.5) ≈ −1.386 for C.
Under the minimum-score convention, A is selected because its complement matches the document least well. These scores are not class probabilities. The example demonstrates complement aggregation, smoothing, and scoring; it does not establish that the invented classes are imbalanced or that CNB outperforms another model.

Illustrative case study

In this fictional case, Alder Support tests a ticket classifier with several rare categories. Overall accuracy initially looks strong for both CNB and a baseline, but the team discovers that the largest category dominates that summary.
It compares per-class results, examines false routes, and tunes preprocessing within the training procedure. It also checks whether new product terminology changes performance after deployment. A confidence threshold is evaluated on separate data rather than assumed reliable from the classifier's raw outputs.
The final selection considers routing quality and operational consequences. CNB is retained only if the measured improvement justifies it for this particular dataset and workflow.

Common misconceptions and mistakes

Complement means examples outside a class, not the numerical complement of its predicted probability. CNB also does not remove every limitation associated with bag-of-words representations or correlated features.
Another mistake is copying a formula that minimizes log weights into software that uses negated weights and maximizes. The conventions can be equivalent, but mixing them reverses decisions.
Implementation options matter. The scikit-learn reference documents the second normalization separately and does not use class_prior in the ordinary multiclass calculation. Do not assume every parameter behaves like its multinomial-naive-Bayes counterpart. Negative feature values also do not fit the count-based complement construction.

Business use

CNB can provide a computationally economical baseline for document classification, particularly when category sizes differ. Sparse features can make training and prediction practical for large vocabularies.
Evaluate it against suitable alternatives using the same data split and preprocessing rules. Report class-specific performance and the costs of routing errors. Monitor changes in labels and vocabulary, and assess calibration separately when downstream decisions depend on probability-like outputs.

Check your understanding

Which examples form A's complement?

Answer: Those belonging to B and C.

Why is A selected in the worked example?

Answer: It has the smallest logarithmic complement score.

Why are the scores not probabilities?

Answer: They are weighted sums of log complement frequencies used for classification.

Related terms

Learn next

Learn next how multinomial and complement naive Bayes construct their weights. Compare normalization options and per-class evaluation, then study calibration before using classifier outputs as decision probabilities.

Sources

Scikit-learn, Complement Naive Bayes user guide: https://scikit-learn.org/stable/modules/naive_bayes.html#complement-naive-bayes Scikit-learn, ComplementNB API reference: https://scikit-learn.org/stable/modules/generated/sklearn.naive_bayes.ComplementNB.html Rennie and colleagues, Tackling the Poor Assumptions of Naive Bayes Text Classifiers: https://people.csail.mit.edu/jrennie/papers/icml03-nb.pdf

Previous: Compiled Query Next: Complement Of An Event