10-fold cross-validation - 10-fold cross-validation is a model-evaluation procedure that divides a dataset into ten parts, trains a fresh model on nine parts, and evaluates it on the...
A/B testing - A/B testing is a randomized experiment that compares two versions of a product, process, or experience by assigning eligible units to different conditions...
A-optimal design - A-optimal design is an experimental-design criterion that chooses observations to minimize the sum, or equivalently the average for a fixed number of...
Accuracy - Accuracy is the proportion of evaluated predictions that match the correct labels under a stated scoring rule.
Area under the PR curve - Area under the PR curve summarizes a model's precision-recall tradeoff as its decision threshold changes.
AUC (Area under the ROC curve) - AUC, when specified as area under the ROC curve, summarizes how well prediction scores separate positive from negative examples across decision thresholds.
Augmented inverse probability weighting - Augmented inverse probability weighting, or AIPW, is a causal estimation approach that combines outcome predictions with corrections weighted by treatment...
Automatic evaluation - Automatic evaluation uses software to assess a model, its outputs, or an AI-enabled process according to specified tests or scoring rules.
Autorater evaluation - Autorater evaluation uses an automated rating system, often a language model, to judge outputs against defined criteria or compare competing responses.
Average precision at k - Average precision at k, commonly written AP@k, evaluates how early relevant items appear within the first k positions of a ranked result list.
Average treatment effect - The average treatment effect, or ATE, is the average difference between the outcome a defined population would experience under one intervention and the...
Average treatment effect on the treated - The average treatment effect on the treated, or ATT, is the average causal effect of an intervention among the units that actually received it.
Balanced accuracy score - Balanced accuracy score measures classification performance by averaging the recall of each class with equal class weight.
BLEU (Bilingual Evaluation Understudy) - BLEU, short for Bilingual Evaluation Understudy, is an automatic metric originally designed to compare machine translations with human reference translations.
Box-Benkhen experiment - A Box - Behnken experiment uses a response-surface design to study how several controllable factors affect an outcome and to fit a quadratic model.
Brier score - The Brier score measures the quality of probability predictions by averaging squared differences between predicted probabilities and observed outcomes.
Calinski-Harabasz Index - The Calinski-Harabasz Index is an internal clustering score that compares separation between cluster centers with variation inside clusters, adjusted for...
Causal estimand - A causal estimand is the precisely defined causal quantity that an analysis aims to learn.
Causal graphs - Causal graphs are diagrams that represent assumptions about causal relationships among variables.
Causal inference - Causal inference is the process of drawing conclusions about how outcomes would change under specified interventions.
Causal language modeling - Causal language modeling is a training task in which a language model predicts the next token using only the preceding context.
Causal link - In partial-order AI planning, a causal link records that one action establishes a condition needed by another action.
Causal network - A causal network is a model that represents variables and their causal relationships, commonly using a directed graph together with probability...
Causal positivity - Causal positivity is the requirement that the treatment choices needed for a causal comparison have positive probability within the relevant covariate...
Causality - Causality concerns relationships in which changes in one part of a system can produce changes in another.
Causation - Causation is a cause-and-effect relationship in which one event, condition or intervention contributes to producing another.
Character N-gram F-score - Character N-gram F-score evaluates generated text by combining precision and recall of matching character sequences against reference text.
Citation precision - Citation precision measures how well the citations an AI system supplies support the statements to which they are attached, under a specified evaluation rule.
Citation recall - Citation recall measures how much of an answer's evaluated content is supported by the citations provided, under a defined claim or statement-level protocol.
Cohen's kappa - Cohen's kappa is a statistic measuring agreement between two raters who assign categorical labels to the same items, adjusted for agreement expected under a...
Collider - A collider is a variable on a causal path where two arrowheads meet, as in X→Z←Y.
Computer experiment - A computer experiment is a planned study that runs a computational model at selected input settings and analyses its outputs.
Computerized adaptive testing - Computerized adaptive testing (CAT) is assessment in which a computer selects later questions using information from a person's earlier responses.
Conditional average treatment effect - The conditional average treatment effect, or CATE, is the average causal effect of a treatment among units with specified characteristics.
Confounder - A confounder is a variable that, in the relevant causal structure, helps create a noncausal association between an exposure or intervention and an outcome.
Cooperative inverse reinforcement learning - Cooperative inverse reinforcement learning (CIRL) is a framework in which a human and an artificial agent share an objective, but the agent is uncertain...
Counterfactual explanation - A counterfactual explanation describes how an input could be changed so that a model would produce a different specified output.
Counterfactual fairness - Counterfactual fairness is a causal-model-based fairness criterion asking whether a prediction for an individual would remain the same in distribution under...
Cross-validation - Cross-validation evaluates a learning procedure by repeatedly fitting it on one part of a dataset and assessing it on a separate part.
D-optimal design - D-optimal design selects an experimental allocation that maximizes the determinant of its information matrix.
Davies-Bouldin Index - The Davies-Bouldin Index is an internal clustering measure that compares within-cluster spread with separation between cluster centers.
Detection error tradeoff - Detection error tradeoff describes the relationship between false alarms and missed detections as a binary detector's decision threshold changes.
Disparate treatment - In machine-learning fairness discussions, disparate treatment refers to treating people differently through a decision process that uses sensitive group...
Dueling bandit - A dueling bandit is a sequential learning problem in which the learner chooses pairs of options and receives relative feedback, such as which option was...
E-optimal design - E-optimal design chooses an experimental allocation that maximizes the smallest eigenvalue of its information matrix.
Error rate - Error rate is the proportion of evaluated units counted as incorrect under a defined rule.
Estimand - An estimand is the precise quantity that a study aims to learn about.
Evaluation - Evaluation is the systematic assessment of an AI model, output, or complete system against defined purposes and criteria.