Evaluation function - An evaluation function is a rule or program that assigns a value to something being assessed.
Evaluation metric - An evaluation metric is a defined measure used to summarize an aspect of a model's or system's performance.
Exact match - Exact match is an evaluation rule that gives credit only when a prediction equals an accepted reference answer under the specified comparison procedure.
Experiment - An experiment is a planned investigation in which a researcher deliberately changes one or more conditions and observes the resulting outcomes.
Extrinsic evaluation - Extrinsic evaluation assesses an AI component or output by measuring its effect on a downstream task or wider application.
F 1 - F1, sometimes written F 1 or F1 score, combines precision and recall through their harmonic mean.
False negative rate - False negative rate is the proportion of actual positive cases that a binary classifier incorrectly labels negative.
False positive rate - False positive rate is the proportion of actual negative cases that a binary classifier incorrectly labels positive.
Fowlkes-Mallows scores - Fowlkes-Mallows scores compare two partitions of the same observations by examining which pairs are grouped together.
Gini impurity - Gini impurity measures how mixed the class labels are within a group of observations, commonly a node in a classification tree.
Granger causality - Granger causality is a predictive relationship between time series: the past of one series helps predict another beyond the information already included in...
Hamming - Hamming is a name attached to several distinct concepts, including Hamming distance, Hamming loss, Hamming codes, and the Hamming window.
Hamming distance - Hamming distance counts the positions at which two aligned sequences of equal length contain different symbols.
Hamming loss - Hamming loss measures the fraction of individual label decisions that are incorrect.
Heterogeneous treatment effect - A heterogeneous treatment effect is a causal effect that varies across individuals, groups or contexts.
Highly adaptive lasso - Highly adaptive lasso, or HAL, is a flexible regression method that represents complex relationships using a large collection of basis functions and...
Holdout cross-validation - Holdout cross-validation is an imprecise phrase commonly used for evaluating a model on a held-out portion of data, sometimes repeated across several random...
Human evaluation - Human evaluation is the assessment of an AI system's outputs or behavior by people using a defined task and judgment procedure.
Human-operator-intervention - Human-operator intervention is a person's action or input to help an otherwise autonomous system handle an exceptional situation or complete a task.
Inter-rater agreement - Inter-rater agreement measures how consistently different people assign judgments to the same items under a specified procedure.
Intermediate phrase - An intermediate phrase is a unit of spoken prosodic structure smaller than a full intonational phrase in frameworks that make this distinction.
Interpolated precision - Interpolated precision is a summary of a precision - recall curve that assigns each recall level the highest observed precision obtainable at that recall or...
Intrinsic evaluation - Intrinsic evaluation assesses a model or component directly against criteria for its own task, rather than measuring its contribution to a larger application.
Inverse odds of sampling weighting - Inverse odds of sampling weighting is a method for reweighting study participants so that their measured characteristics better represent a specified target...
Inverse probability of censoring weighting - Inverse probability of censoring weighting, or IPCW, adjusts an analysis for incomplete follow-up by giving observed contributions weight according to the...
Inverse probability of treatment weighting - Inverse probability of treatment weighting, or IPTW, weights observations by the inverse probability of receiving their observed treatment, conditional on...
Item response theory - Item response theory (IRT) is a family of statistical models that relates responses to individual questions or tasks to an unobserved ability or trait and...
Jaccard similarity coefficient score - The Jaccard similarity coefficient score measures overlap between two sets by dividing the size of their intersection by the size of their union.
k-fold cross validation - K-fold cross validation is an evaluation procedure that divides data into k non-overlapping parts, or folds, and repeatedly trains a model on all but one...
L 1 loss - L1 loss measures prediction error using the absolute difference between a predicted value and its observed target. For one example it is |prediction − target|.
L 2 loss - L2 loss commonly means squared prediction error: the square of the difference between a prediction and its observed target.
Label ranking average precision - Label ranking average precision, or LRAP, evaluates how well a model ranks the relevant labels for each example above irrelevant labels.
Labeled precision - Labeled precision is the proportion of predicted structures that match reference structures in both their structural identity and their label.
Labeled recall - Labeled recall is the proportion of reference structures that a system recovers with the correct structural identity and label.
Leave-one-out cross-validation - Leave-one-out cross-validation evaluates a learning procedure by repeatedly training on all observations except one and testing on the single omitted...
Log likelihood ratio - A log likelihood ratio compares how well two statistical models or hypotheses account for the same observed data by taking the logarithm of their likelihood...
Longitudinal modified treatment policy - A longitudinal modified treatment policy, or LMTP, defines interventions over time that can modify the treatment value that would naturally arise at each...
Longitudinal targeted maximum likelihood estimation - Longitudinal targeted maximum likelihood estimation, or LTMLE, is a framework for estimating specified causal quantities from data containing treatment...
Max error - Max error is the largest absolute difference between a prediction and its observed target in an evaluated dataset.
METEOR - METEOR is an automatic text-evaluation metric developed for machine translation.
Metric - A metric is a defined quantitative measure used to describe or compare some property of an AI system, dataset, process, or outcome.
Metric space - A metric space is a set of objects together with a distance function that satisfies four rules: nonnegativity, zero distance exactly for identical objects...
Minimax loss - Minimax loss, in the supplied glossary's generative-adversarial-network context, is the original GAN objective that a generator tries to minimize while a...
Mixed Precision - Mixed precision is the use of more than one numerical precision within a computation.
Mixture experiment - A mixture experiment studies how the proportions of components in a blend affect a measured response.
Model validation - Model validation is the process of examining evidence that a model is suitable for its intended purpose under specified conditions.
Negative class - The negative class is one of the two classes in a binary classification task, defined as the counterpart of the positive class.
Nested cross-validation - Nested cross-validation is an evaluation procedure that places a model-selection process inside an outer cross-validation process.