Is class imbalance the same as unequal cost?

No. A rare class can have high, low or ordinary error cost.

Must the model be retrained?

Not always. A suitable probability model may support a different decision threshold.

Can costs vary by case?

Yes. Some applications require instance-specific consequences rather than one fixed matrix.

AI Glossary / AI and machine learning / Intermediate

Cost-sensitive machine learning

Definition

Cost-sensitive machine learning accounts for the fact that different prediction errors or actions can have different consequences. It aims to support decisions with lower expected cost, rather than treating every mistake as equally important.

What it means

Missing an urgent service request may be more costly than sending a routine request for unnecessary review. A model with fewer total mistakes can still be worse for the business if its remaining mistakes are especially expensive.

Costs may represent money, time, inconvenience or another clearly defined consequence. They can differ by case and by action. A cost-sensitive design makes those trade-offs explicit instead of hiding them behind one accuracy percentage.

The numbers are assumptions that need scrutiny. Assigning a low numeric cost to a serious consequence does not make it unimportant. Some requirements should be treated as constraints or review rules rather than reduced to an arbitrary weight.

How it works

A team defines outcomes, available actions and their costs. It can incorporate those costs into training, change the decision threshold of a probability model, or optimize a downstream decision rule.

Expected-cost decisions combine estimated probabilities with consequences. This makes probability calibration important: a score that ranks cases well may still be unsuitable as a literal probability.

Evaluation uses realistic case frequencies and the full cost model. Changing class weights during training is not automatically equivalent to choosing an economically sensible action threshold. The complete decision process needs testing.

Three practical examples

A support system balances missed urgent cases against the workload of unnecessary escalation.
A quality-inspection team compares the cost of missed defects with the cost of checking acceptable items again.
A document-routing project gives different penalties to sending a case to the wrong specialist and delaying a low-priority case.

Think of it this way

Think of packing for a trip: forgetting a spare pen and forgetting a passport are both omissions, but their consequences differ sharply.

The analogy has limits because operational costs may be uncertain, shared across people or constrained by policy. A single number can simplify a decision without capturing every value that matters.

Worked example

Choosing between review and no review

Suppose a fictional inspection system estimates a 10% probability that an item has a defect. For this simplified exercise, sending any item for review costs 8 units and review is assumed to catch the defect. Letting a defective item pass costs 100 units, while letting an acceptable item pass costs zero.

The expected cost of no review is 0.10 times 100, or 10 units. The cost of review is 8 units. Under these assumptions, review has lower expected cost.

The threshold is 8 divided by 100, or 8%: above that estimated defect probability, review is preferred. Real workflows may have imperfect review, capacity limits and additional costs, which change the calculation.

The example shows why a default 50% classification threshold can be inappropriate. It also shows why the probability estimate and cost assumptions must be credible before the decision rule is trusted.

Illustrative case study

A fictional support center selects a model using overall accuracy, then discovers that it misses too many urgent cases. Analysts define the review workload and consequences of missed urgency with operations staff.

They compare several thresholds on an untouched evaluation set and estimate how many cases each would send to specialists. A lower threshold catches more urgent cases but exceeds the team's capacity unless staffing or prioritization changes.

The final proposal therefore combines a model threshold with a realistic review process. The case illustrates that cost-sensitive decisions are operational designs, not just a new training parameter.

Common misconceptions and mistakes

Assuming highest accuracy means lowest cost: different errors can have different consequences.
Choosing cost weights without business input: arbitrary numbers can produce arbitrary decisions.
Treating a model score as a calibrated probability automatically: calibration needs evaluation.
Ignoring review capacity: a theoretically attractive policy may be impossible to operate at the resulting volume.

Business use

Cost-sensitive methods are useful whenever false positives, false negatives and interventions have unequal consequences. They make model comparisons more closely reflect the actual workflow.

Document the cost assumptions, their owners and plausible ranges. Run sensitivity analysis to see whether modest changes alter the preferred policy. Report error counts alongside estimated cost so readers can understand the trade-off. Where consequences cannot reasonably be traded against one another, use explicit constraints or escalation requirements.

Check your understanding

A review costs 8 units and an undetected defect costs 100. Under the simplified assumptions in the example, should a case with a 5% defect probability be reviewed purely to minimize expected cost?

Answer: No. Its expected no-review cost is 5 units, below 8. Different constraints or imperfect assumptions could change the real decision.

Related terms

Learn next

Read probability calibration. Write the actions and consequences for one model-assisted workflow, then test how its preferred threshold changes when cost estimates vary.

Probability calibration

Sources

Elkan, The Foundations of Cost-Sensitive Learning: https://cseweb.ucsd.edu/~elkan/rescale.pdf. The abstract and section 1 were checked for expected-cost decisions, probability estimates and coherent cost baselines. All numerical costs and scenarios are invented teaching examples.

Previous: Cost Model Next: Costmap