Can accuracy be high for a useless classifier?

Yes. In a highly imbalanced dataset, always predicting the majority class can achieve a high score while missing the events of interest.

Can accuracy evaluate continuous numerical predictions directly?

Exact label matching is generally unsuitable for regression; use an appropriate error metric unless the task defines a meaningful tolerance rule.

What does weighted accuracy mean?

Each example's correctness contributes according to a specified weight, so it answers a different population or priority question from the unweighted metric.

AI Glossary / Causal inference and experimentation / Beginner

Accuracy

Definition

Accuracy is the proportion of evaluated predictions that match the correct labels under a stated scoring rule. For ordinary single-label classification, it equals the number of correct predictions divided by the number of evaluated examples. Accuracy is easy to communicate, but it treats each counted error equally unless weights are specified. It does not by itself describe which classes are missed, how confident a model is, or how costly its mistakes are.

What it means

If a classifier predicts one category for each example, accuracy asks a simple question: how often was that category right? The answer depends on the evaluation dataset, label quality, decision threshold or selection rule, and any weighting. A reported percentage is incomplete without those details. In broader conversation, “accuracy” is sometimes used loosely to mean quality, but technical evaluation should reserve it for a defined metric rather than use it as a substitute for precision, recall, factuality, or all-round usefulness.

How it works

Compare each predicted label with its reference label, assign one for a match and zero otherwise, then average. For binary classification, the formula is (true positives plus true negatives) divided by all cases. Weighted accuracy averages those correctness indicators using specified nonnegative example weights. Multiclass accuracy uses the same match principle. In multilabel tasks, some software uses subset accuracy, which counts an example as correct only when its entire predicted label set matches the reference set; that differs from averaging correctness across individual labels.

Three practical examples

Document routing: measure the fraction of documents assigned to the correct department, while examining which department pairs are confused.
Image classification: compare predicted product categories with verified labels on representative held-out images.
Intent recognition: track correct intent selections over time, alongside class-specific performance and the consequences of misrouting a request.

Think of it this way

Think of marking a quiz by counting how many answers match the answer key. The analogy captures simple per-item correctness. Its limit is that real tasks can have unequal error costs, ambiguous labels, and changing question distributions, so the final percentage may hide exactly the failures that matter most.

Worked example

Suppose 1,000 inspection examples contain 950 acceptable items and 50 defective ones. A trivial classifier that always predicts “acceptable” gets 950 right, giving 95% accuracy, but detects none of the defects. A second classifier identifies 40 defective items, misses 10, correctly accepts 900 acceptable items, and wrongly flags 50 acceptable items. Its accuracy is (40 + 900)/1,000 = 94%, lower than the trivial baseline. Yet its defect recall is 40/50 = 80%, whereas the baseline's recall is zero.
This does not prove that the second classifier is the right operational choice: its 50 false alarms also matter. It shows that ranking systems by accuracy alone can conflict with the task's priorities. For a separate multilabel illustration, suppose the true tags are {red, fragile} and the predicted tags are {red}. Exact-set or subset accuracy marks the example incorrect, even though one tag was correctly recovered. A label-by-label metric gives a different view. Both calculations require the scoring convention to be named before comparing results.

Illustrative case study

A fictional support team celebrates 96% intent-classification accuracy. A breakdown reveals that the dominant “general query” category accounts for most examples, while a small but important “account access” category is frequently misrouted. The team reviews the confusion matrix and evaluates recall and precision for that category. It revises its reporting so the headline percentage remains visible but no longer obscures a recurring service problem.

Common misconceptions and mistakes

Accuracy is not the same as precision: precision asks how many predicted positives are truly positive. It is not recall, which asks how many actual positives were found. High accuracy can coexist with poor minority-class performance. Training accuracy does not establish performance on new examples, and threshold changes can alter accuracy even when the underlying scores are unchanged. A 95% accuracy estimate also does not mean every individual prediction has a 95% chance of being correct.

Business use

Use accuracy when its equal-per-example correctness interpretation aligns with the task and report complementary measures where needed. Include the sample size, class distribution, dataset period, and evaluation procedure. Compare against simple baselines such as predicting the most common class. Inspect meaningful subgroups and error categories, and consider uncertainty in the estimate. If consequences differ sharply across mistakes, evaluate those costs directly instead of assuming that maximizing the total number of correct labels maximizes business value.

Check your understanding

Which model has higher accuracy in the inspection example, and which detects defects?

Answer: The always-acceptable baseline has 95% accuracy and detects no defects. The second model has 94% accuracy and finds 40 of 50 defects. The comparison demonstrates why the intended error tradeoff must accompany the headline score.

Related terms

Classification
Confusion matrix
Precision
Recall
Error rate
Calibration
balanced accuracy
class imbalance
decision threshold
subset accuracy

Learn next

Build a confusion matrix from a small set of predictions and calculate accuracy, precision, and recall separately. Then change the class proportions without changing the class-specific error rates to see how overall accuracy moves. For multilabel data, compare exact-set accuracy with label-wise measures before choosing a reporting convention.

Sources

Scikit-learn, accuracy_score and multilabel subset accuracy: https://scikit-learn.org/stable/modules/generated/sklearn.metrics.accuracy_score.html Scikit-learn, classification metrics and scoring: https://scikit-learn.org/stable/modules/model_evaluation.html Scikit-learn, DummyClassifier baseline: https://scikit-learn.org/stable/modules/generated/sklearn.dummy.DummyClassifier.html

Previous: Accumulating eligibility trace Next: Accuracy barrier