Does in-processing require changing the model architecture?

Not always. Some approaches wrap an existing learner or modify its objective and training costs.

Is a fairness penalty the same as a strict constraint?

No. A penalty weights a concern within an objective; a constraint specifies a condition the optimization seeks to satisfy.

Does successful mitigation prove fairness?

No. It provides evidence about a selected criterion under particular assumptions and evaluation conditions.

AI Glossary / Governance, ethics and safety / Intermediate

In-processing

Definition

In-processing is a family of fairness-mitigation approaches that change how a machine-learning model is trained. The learning procedure incorporates a selected fairness objective or constraint alongside predictive performance. It differs from preprocessing, which changes training data or representations before the main learning step, and postprocessing, which adjusts the outputs of an already trained model.

What it means

Ordinary training often seeks to reduce prediction error. An in-processing method asks the training procedure to consider an additional requirement, such as limiting a specified difference in outcomes or error rates between groups.

This is a location in the development pipeline, not one particular algorithm or a universal definition of fairness. Different methods use penalties, constraints, adversarial objectives, or repeated training with adjusted costs. They can produce different models and tradeoffs.

Choosing the requirement is a substantive decision. A model can satisfy a numerical criterion while the wider service remains unfair, inaccessible, or based on unsuitable labels. In-processing addresses a defined mathematical concern within a broader governance problem; it does not certify the whole system as fair.

How it works

First define the task, relevant groups, and criterion to investigate. Establish a baseline and select a training approach compatible with the model and available data.

One approach adds a penalty for the chosen disparity to the usual training objective. Another seeks low error subject to a constraint. Reduction-based methods can translate the constrained problem into a sequence of cost-sensitive learning problems; some return a randomized combination of predictors. Adversarial methods train a predictor while limiting an auxiliary model's ability to recover sensitive information from specified outputs or representations.

Evaluate candidate models on separate data using the selected criterion, predictive performance, and other relevant measures. Check subgroup sample sizes and uncertainty. A training result does not guarantee the same behavior after deployment, especially when the population or data changes.

Three practical examples

1. A team trains a classifier with a penalty on a selected group disparity, then compares the resulting candidates on an untouched evaluation set.

2. A developer uses a constrained-learning reduction to seek lower error while keeping a chosen fairness violation within an investigated tolerance.

3. A research team trains an adversarial fairness model and checks whether the intended constraint improves without overlooking other errors or loss of useful information.

Think of it this way

Think of practicing a musical passage while paying attention to both timing and volume balance. The practice objective shapes how the performance develops; it is different from editing a completed recording afterward.

The analogy is limited because social fairness cannot be reduced to musical balance or a single agreed score. The selected objective may omit something important. The useful lesson is that in-processing changes learning itself, while the choice of what to optimize remains a separate question requiring judgment.

Worked example

Consider an invented training objective J = E + lambda × D. Here E is an error measure, D is a selected disparity measure, and lambda controls its weight. Assume both measures have been defined and scaled for this illustration.

Candidate A has E = 0.10 and D = 0.20. Candidate B has E = 0.13 and D = 0.05. With lambda = 0.1, A scores 0.12 and B scores 0.135, so the objective favors A. With lambda = 0.5, A scores 0.20 and B scores 0.155, so it favors B.

The arithmetic shows how an objective can change the preferred training outcome. It does not show that either candidate is fair or that this formula is appropriate for a real application. The meaning and scale of D, the quality of labels, evaluation uncertainty, and consequences of errors still matter. A held-out assessment must also check whether the apparent improvement generalizes.

Illustrative case study

Fictional case: Westhaven Services studies a model that routes requests for additional review. The baseline shows a disparity under a criterion the team considers relevant. Developers compare several training penalties rather than treating the first numerical improvement as a final answer.

Operational reviewers examine the resulting errors, while the team checks whether small subgroups have enough evidence for reliable conclusions. One candidate reduces the target disparity but produces an undesirable increase in missed cases, prompting further investigation.

The team documents the tradeoff and keeps the deployment decision open. This fictional case illustrates evaluation after mitigation; it does not claim that a particular penalty or constraint guarantees an acceptable service.

Common misconceptions and mistakes

In-processing is not simply any data processing performed while a program runs. In this context, it refers to modifying the learning procedure for fairness mitigation.

Satisfying one constraint does not satisfy every fairness definition. Some criteria can conflict, and the right choice depends on the concern being addressed.

Another mistake is evaluating only training data. A model may meet a condition there without doing so on future cases. Also check whether a method needs sensitive attributes at training time, prediction time, or both; requirements differ. Do not infer the operational requirements from the category name alone.

Business use

Businesses considering in-processing need access to an appropriate training process and evidence that the selected criterion matches the actual concern. If only a fixed external prediction service is available, changing its training may not be possible.

Ask developers to show the baseline, candidate tradeoffs, independent evaluation, and remaining limitations. Include the cost of collecting and managing any required group information appropriately. The business decision should address the completed workflow and affected people, rather than equating an improved fairness metric with automatic approval for deployment.

Check your understanding

A training penalty lowers demographic disparity on the training set. What remains to be checked?

Answer: Performance on separate representative data, other relevant harms and error patterns, uncertainty, and whether the selected disparity criterion fits the intended use.

Related terms

Preprocessing
Fairness constraint
Regularization
Demographic parity
Equalized odds
postprocessing
adversarial debiasing
model evaluation

Learn next

Study fairness constraints and postprocessing next. Compare where each approach changes a pipeline, what information it requires, and which evidence would demonstrate improvement in the actual deployment context.

Sources

AI Fairness 360, Algorithms: https://aif360.readthedocs.io/en/stable/modules/algorithms.html

Fairlearn, Mitigations: https://fairlearn.org/main/user_guide/mitigation/index.html

Agarwal and colleagues, A Reductions Approach to Fair Classification: https://proceedings.mlr.press/v80/agarwal18a.html

Official method descriptions and the original paper's abstract were read. The objective comparison and fictional service are original teaching examples, not implementation guarantees or empirical results.

Previous: In-group bias Next: In-set condition