Which class is the positive one?

Whichever you are trying to catch: fraud, churn, failure. It is a convention, not a compliment.

What if I have three or more categories?

That is multi-class classification, the same idea with more classes.

How much data do I need?

Enough labeled examples of both classes that the model can learn the rarer one; hundreds is a start, not a rule.

Can the two classes overlap in practice?

The labels must be exclusive in training; a genuinely borderline real-world case simply gets a probability near 0.5.

AI Glossary / AI and machine learning / Beginner

Binary classification

Definition

Binary classification is the task of sorting items into exactly two categories, such as spam or not spam, churn or stay, approve or decline.

What it means

Most business questions start as yes-or-no: will this customer leave, is this invoice wrong, will this machine fail this month. Binary classification is the machine version of that decision.

The model usually produces a probability, say a 0.82 chance of churn, and a threshold turns the probability into a decision. The default threshold is 0.5, but it is a choice, not a law.

Every decision can be wrong in two ways. A false positive cries wolf: a loyal customer flagged as leaving. A false negative misses the wolf: a leaver flagged as safe. The two errors rarely cost the same.

Moving the threshold trades one error for the other. Lower it and you catch more real cases but raise more false alarms. Raise it and the reverse. This trade-off is a business decision, not a technical one.

Accuracy alone misleads. If only 2 percent of transactions are fraud, a model that says not fraud every time is 98 percent accurate and completely useless.

Class balance shapes everything. When one class is rare, the model sees few examples of it, plain accuracy stops meaning anything, and the choice of metric and threshold matters more than the choice of model.

How it works

Labeled examples train the model: past customers marked churned or stayed, past transactions marked fraud or clean. The model learns the patterns that separate the two classes.

At prediction time the model scores each new item, and the threshold converts the score into a class.

Results are summarized in a confusion matrix: true positives, true negatives, false positives and false negatives. Every useful metric comes from those four counts.

Precision asks, of the items we flagged, how many were real. Recall asks, of all the real cases, how many we caught. You usually cannot maximize both at once.

Choose the metric that matches the cost. Screening for a serious illness prizes recall; a fraud team with three reviewers prizes precision, because every false alarm costs review time.

Imbalance can be addressed during training by weighting the rare class more heavily or sampling it more often, both of which tell the model that mistakes on the rare class cost more.

Three practical examples

Email filtering: spam or not spam, with the junk folder as the threshold's landing zone.
Lending: default or repay, where a false negative costs money and a false positive costs a customer.
Maintenance: fail within 30 days or keep running, driving which machines get serviced first.

Think of it this way

Think of a door supervisor deciding who needs a bag check. Check nobody and things get stolen; check everybody and the queue walks away. The threshold is how strict the door policy is.

The right strictness depends on what a missed thief costs against what a lost customer costs, and only the shop owner can answer that.

Worked example

setting the threshold for a churn flag

Suppose a telecom firm trains a churn model on two years of customer history and tests it on held-out customers.

At a 0.5 threshold it flags 300 customers a month, of whom 40 percent really leave. The retention team can only call 150 a month, so half the flags are wasted.

Raising the threshold to 0.7 flags 120 customers, of whom 65 percent really leave. Fewer calls, more saves per call. The figures are invented; the trade-off is real.

The firm reviews the threshold quarterly, because both customer behavior and team capacity change.

Illustrative case study

A fictional clinic builds a tool to flag scans needing urgent review. Missing an urgent case is far worse than a wasted review, so it sets a low threshold for high recall.

Every flagged scan is then read by a specialist, so false positives cost minutes, not patients. Accuracy is modest on paper, but the design matches the cost of each error.

The invented lesson: pick the error you can afford, then set the threshold to produce it, and keep a human where the stakes are high.

Common misconceptions and mistakes

High accuracy means a good model. On lopsided data, accuracy can look excellent while the model misses every case that matters.
The threshold is part of the model. It is a separate dial you set for your own costs.
Binary means the model is sure. It only means two categories; confidence can still be low.

Business use

When a vendor quotes accuracy, ask for precision and recall at the threshold they recommend, and what moving the threshold does.

Before building, write down what a false positive and a false negative each cost you. That note will choose your threshold and your metric.

Ask whoever builds the model which class they treated as positive and why; it decides which errors the system tries hardest to avoid.

Check your understanding

A fraud model flags nothing and is 98 percent accurate on a stream with 2 percent fraud. Why is it useless?

Answer: Accuracy counts the easy majority class; the model has zero recall on the fraud cases, which are the only ones that matter.

Related terms

Learn next

AI for Business Owners covers how to frame prediction problems and judge model claims with a business lens: https://aiforbusinessandlife.com/academy/ai-for-business-owners. [C2]

Sources

[S12] Aalto University, Dictionary of Machine Learning; [S2] Google, Machine Learning Glossary; [S14] KNIME, Data Science Glossary; [S19] scikit-learn documentation, Metrics and scoring; [S22] Berkeley BOP, Artificial Intelligence Ontology (aio.owl). Full URLs are in the source ledger. Verified 11 October 2026.

Previous: Binary branching Next: Binary condition