Is MCC the same as F1?

No. Binary MCC includes true negatives, whereas the usual positive-class F1 does not.

Yes. In the binary nondegenerate case, negative values indicate that predictions and actual labels are negatively associated.

AI Glossary / Statistics and probability / Intermediate

Matthews correlation coefficient

Definition

The Matthews correlation coefficient, or MCC, measures agreement between predicted and observed class labels using the full confusion matrix. In nondegenerate binary classification, it equals the Pearson correlation between the two binary label variables. Values range from -1 to 1, with 1 indicating perfect agreement, 0 indicating no empirical binary association, and -1 indicating complete reversal.

What it means

MCC asks whether a classifier's decisions correspond to the actual labels after considering correct and incorrect predictions for both classes. It can reveal weaknesses hidden by accuracy when one class dominates the evaluation set.
It is still one summary of a particular dataset and decision rule. A useful MCC does not establish calibrated probabilities, acceptable error costs, or good performance for every subgroup. Those questions need additional evidence.

How it works

For binary outcomes, form the four confusion-matrix counts: true positives TP, true negatives TN, false positives FP, and false negatives FN. Define the positive class and ensure predicted labels and reference labels refer to the same cases.
The formula is MCC = (TP*TN - FP*FN) divided by the square root of (TP+FP)*(TP+FN)*(TN+FP)*(TN+FN). The denominator scales the numerator using the observed and predicted class totals.
If either label variable is constant, the denominator is zero and the ordinary correlation formula is undefined. Software may impose a reporting convention for these degenerate cases. State the convention rather than interpreting a supplied zero as an ordinary estimated correlation.
Unlike accuracy, MCC is sensitive to how correct decisions are distributed across the classes. A classifier that always predicts a dominant class can have high accuracy while providing no separation of the two outcomes. The confusion matrix makes that limitation visible.
MCC uses hard class decisions. For a model returning probabilities, choosing a decision threshold changes the predicted labels and therefore usually changes MCC. Select thresholds using suitable development data; evaluating repeatedly chosen thresholds on the final test set can produce optimistic results.
The binary measure is symmetric in the two classes: consistently swapping positive and negative labels in both truth and prediction leaves it unchanged. Swapping only the predicted binary labels reverses its sign when the ordinary formula is defined.
A multiclass generalization uses the full multiclass confusion matrix. It is not simply the average of separate one-versus-rest binary MCC values. Its attainable minimum depends on the class distribution, so interpret it using the stated multiclass definition.
Although useful with unequal class frequencies, MCC is not invariant to prevalence. Comparisons across datasets with different class mixtures require care, even if other aspects of classifier behavior are similar. Report the class counts alongside the score.
Estimate uncertainty using a method appropriate to the evaluation design. Repeated observations from the same customer or document should not automatically be treated as independent. Also examine class-specific errors and relevant subgroups; an aggregate score can conceal important differences.

Three practical examples

A document classifier is evaluated with MCC alongside its confusion matrix because one label is much more common than another.
A quality-inspection team compares classifiers that have similar overall accuracy but different detection patterns.
An AI evaluation team reports a multiclass MCC while separately reviewing which categories are being confused.

Think of it this way

Think of checking whether two sets of labels move together, rather than merely counting how often the most common label is correct. MCC summarizes their association. The underlying table still shows which mistakes produced that summary and whether those mistakes are acceptable.

Worked example

An original test set contains 100 cases: 10 positives and 90 negatives. A classifier produces TP = 5, FN = 5, FP = 5, and TN = 85.
Its accuracy is (5+85)/100 = 90%. The MCC numerator is 5*85 - 5*5 = 400. The denominator is sqrt(10*10*90*90) = 900. MCC is therefore 400/900 = 4/9, approximately 0.4444.
As an independent check, encode truth and predictions as zeros and ones. Both have mean 0.1. Their mean product is 0.05, so covariance using population-style divisors is 0.05 - 0.01 = 0.04. Each variance is 0.1*0.9 = 0.09, giving correlation 0.04/0.09 = 4/9.
An always-negative classifier also has 90% accuracy on these cases, but misses all 10 positives. Its prediction variance is zero, so the ordinary MCC expression is undefined. Any software-reported replacement value must be recognized as a convention.
Neither the accuracy tie nor the MCC comparison alone decides deployment: the consequences of five false positives and five false negatives still matter.

Illustrative case study

In a fictional case, Juniper Review evaluates an automated document-routing system. Most documents belong to a routine category, so an initial accuracy report looks reassuring.
The evaluation team adds MCC, class counts, and per-class error summaries. These reveal that the system's apparent success depends heavily on the majority category. Managers then compare error costs and test coverage before choosing a decision rule. The example illustrates why several views of performance are needed; it reports no real deployment result or numerical improvement.

Common misconceptions and mistakes

Do not treat MCC as a percentage of correct predictions or a probability that the model is right. Do not claim that a zero score proves the classifier was generated randomly.
Being useful with imbalanced data does not make MCC immune to changes in class prevalence. A higher MCC also does not guarantee lower business cost, better calibration, or fairness across groups.

Business use

MCC can provide a compact comparison when both classes matter and accuracy is easily dominated by their relative frequency. Keep the confusion matrix and operational error costs visible. For decisions driven by probability estimates or specific recall requirements, add metrics and threshold analyses that directly address those objectives.

Check your understanding

Why do two classifiers with 90% accuracy behave differently in the example?

Answer: One identifies five positives while making five false alarms; the other predicts every case negative. Accuracy hides that distribution of errors.

Related terms

Confusion matrix
Accuracy
Precision
Recall
Phi coefficient
F1 score
Class imbalance
Decision threshold

Learn next

Next, calculate MCC from several small confusion matrices. Then compare it with class-specific rates, calibration measures, and cost-based evaluation under the expected deployment class mixture.

Sources

Scikit-learn, Matthews correlation coefficient user guide: https://scikit-learn.org/stable/modules/model_evaluation.html#matthews-correlation-coefficient Scikit-learn, matthews_corrcoef API: https://scikit-learn.org/stable/modules/generated/sklearn.metrics.matthews_corrcoef.html Chicco and Jurman, The advantages of the Matthews correlation coefficient over F1 score and accuracy in binary classification evaluation: https://link.springer.com/article/10.1186/s12864-019-6413-7

Previous: Matrix variate Dirichlet distribution Next: Matting Laplacian