Does every method use group-specific thresholds?

No. Post-processing is a broad family; input requirements depend on the particular procedure.

Why might a method randomize?

Mixing rules can reach aggregate operating points unavailable from a single threshold, but its suitability must be assessed.

Can it replace better data?

Not generally. It can change decisions from existing outputs without supplying information the model never received.

AI Glossary / Governance, ethics and safety / Intermediate

Post-processing algorithm

Definition

A post-processing algorithm is a defined procedure that transforms a model's outputs after prediction. In AI fairness work, it often means a procedure that adjusts scores or decisions to pursue a specified fairness constraint while leaving the base predictor unchanged.

What it means

The algorithm sits between a prediction and its final use. It may choose thresholds, map scores or alter a ranking according to an explicit objective. This makes it more specific than the general activity called post-processing: the algorithm describes how the transformation is performed.

In fairness applications, the objective might be to reduce a particular difference in group error rates while retaining useful predictive performance. The selected fairness definition must be stated. Improving equality of true-positive rates, for example, does not automatically equalize false-positive rates or resolve all concerns about the decision process.

This entry focuses on fairness-related algorithms because of the source-list context. The wider term also includes output transformations unrelated to fairness. It is not a guarantee that an algorithm's results are appropriate, lawful or beneficial merely because a fairness constraint appears in its design.

How it works

A threshold-based method starts with model scores and an appropriate dataset containing outcome labels and relevant group information. It evaluates possible decision rules, estimates their performance and selects a rule or combination of rules under the specified objective and constraint.

Some methods use different rules for different groups. Others randomize between rules to obtain a desired statistical operating point. Such randomization means that the constraint is about expected or aggregate behavior under the fitted assumptions, not a promise that every finite collection of decisions will have exactly identical rates.

The fitted transformation must then be evaluated on suitable independent data and implemented with the required inputs. If group attributes are needed at decision time, their availability, quality and appropriateness need explicit review. Monitor distribution changes and the consequences of errors. A mathematically feasible transformation may be a poor operational choice, especially if it achieves parity by worsening outcomes without addressing the underlying problem.

Three practical examples

A research team compares threshold rules that seek similar true-positive rates across specified groups in a classification task.
An analyst evaluates a randomized combination of decision thresholds and reports both expected rates and variability in finite samples.
A product team compares a post-processing mitigation with improving data collection, checking whether the latter addresses missing information more effectively.

Think of it this way

Think of a routing desk that receives priority scores and applies a written dispatch rule. The scores remain the same, but the rule determines which jobs enter the next queue. Revising the rule changes the actual allocation.

The analogy does not establish which allocation is fair. That judgment depends on the purpose, affected people and consequences, not simply on having a consistent rule.

Worked example

Matching one rate while changing another

In an invented evaluation, Group A contains 50 actual positive and 50 actual negative cases. Group B has the same counts. The original rule identifies 40 positives in A and 30 in B, giving true-positive rates of 80% and 60%. It also flags 5 negatives in A and 5 in B, giving false-positive rates of 10% each.

Suppose a candidate rule for B identifies 40 positives but also flags 15 negatives. True-positive rates now match at 80%, while B's false-positive rate rises to 30%.

The candidate satisfies the stated true-positive-rate comparison in this example. It does not satisfy equalized odds, which also concerns false-positive rates, and it is not automatically the right policy. The extra incorrect flags and their consequences need evaluation.

Illustrative case study

Fictional case: Larch Services receives a proposal for an output-adjustment algorithm. Its headline says the algorithm removes a disparity, but the documentation reveals that it requires a group attribute the deployed service does not reliably collect.

The team reviews the proposed criterion, missing-data handling and the justification for using the attribute. It also evaluates alternatives that address weaknesses in the underlying dataset. The proposal is assessed as a complete operational design rather than a formula detached from its inputs.

The case illustrates why implementation conditions belong in a mitigation review. It does not imply that group-specific rules are always prohibited or always appropriate.

Common misconceptions and mistakes

A post-processing algorithm necessarily learns new base-model parameters. Many operate on a fixed predictor's outputs.
Exact optimization means exact parity in every future sample. Estimation error, randomization and population change can alter observed results.
A fairness label removes the need to examine harms. Each criterion measures a limited relationship and can involve consequential tradeoffs.

Business use

Businesses should request a description of the objective, constraint, fitting data, required deployment inputs and evaluation results. Ask whether the reported rates concern the fitting sample, an independent test or an expectation under randomization. These distinctions affect the strength of the claim.

Keep the algorithm's version and configuration linked to the base model version. A model update can change score distributions and invalidate a previously fitted transformation. Assess the final decisions with relevant stakeholders and expertise, including the practical consequences of false positives and false negatives.

Check your understanding

A candidate algorithm equalizes true-positive rates but triples one group's false-positive rate. Has it established equalized odds or overall fairness?

Answer: No. Equalized odds also concerns false-positive rates, and broader fairness requires evaluating the context and consequences beyond one metric.

Related terms

Learn next

Read equalized odds and false-positive rate. For one proposed mitigation, calculate both missed positive cases and incorrectly flagged negative cases before deciding whether the reported improvement addresses the intended harm.

Sources

Hardt, Price and Srebro, Equality of Opportunity in Supervised Learning: https://arxiv.org/abs/1610.02413. The primary abstract describes adjusting learned predictors under a specified criterion. Fairlearn, Postprocessing: https://fairlearn.org/main/user_guide/mitigation/postprocessing.html. Relevant documentation on thresholds, required attributes, randomisation and tradeoffs was reviewed. The confusion-count example and fictional case are original illustrations.

Previous: Post-processing Next: Post-sample evaluation