Grouped Query Attention - Grouped Query Attention, usually written grouped-query attention or GQA, is an attention design in which several query heads share a smaller set of key and...
Head - A head is a model component that performs a particular computation or produces a particular kind of output.
Hierarchical Language Model - A Hierarchical Language Model organizes some aspect of language modeling into multiple levels, such as word classes and words, characters and words, or...
Hilbert space - A Hilbert space is a vector space equipped with an inner product and complete in the distance induced by that inner product.
Implicit Language Model - An Implicit Language Model is language-pattern knowledge learned within a broader system without being supplied as a separate, explicitly integrated...
Importance sampling - Importance sampling is a Monte Carlo technique that estimates quantities under one probability distribution by drawing samples from another and correcting...
Inpainting - Inpainting is the process of filling, repairing, or replacing selected regions within an image using surrounding content and, in some systems, additional...
Language model - A language model is a model of patterns in language, commonly expressed through probabilities over tokens or token sequences.
Latent diffusion - Latent diffusion is a generative modeling approach that performs a diffusion process in a learned latent representation instead of directly in the original...
Latin hypercube sampling - Latin hypercube sampling is a method for selecting combinations of input values so that each input's range is represented evenly through equal-probability...
LLM-as-a-judge - LLM-as-a-judge is an evaluation method in which a large language model assesses an answer, conversation, or other output using specified criteria.
LLM evaluations (evals) - LLM evaluations, often shortened to evals, are structured tests used to measure how a large language model or a system built around it performs.
LLM Temperature - LLM temperature is a generation setting that rescales a language model's next-token scores before sampling.
Machine translation - Machine translation is the automatic conversion of text or speech content from one human language into another by a computer system.
Masked language model - A masked language model is a language model trained to predict selected hidden or corrupted tokens using the surrounding text.
Masked language modeling - Masked language modeling is a self-supervised training task in which selected tokens in a text are hidden or altered and a model learns to predict their...
Mixture of experts - A mixture of experts, or MoE, is a model design that combines multiple component networks called experts through a gating or routing mechanism.
Model Quantization - Model quantization represents a model's numerical values using a more limited set of values, often with fewer bits than the original representation.
Modular Language Model - A modular language model is a language-model design organized around distinct components that can be selected, combined, adapted, or replaced through a...
Monte Carlo sampling - Monte Carlo sampling uses random draws from specified probability distributions to explore a system or estimate a numerical quantity.
Multi-head attention - Multi-head attention is an attention mechanism that performs several learned attention operations in parallel and combines their outputs.
Multi-head self-attention - Multi-head self-attention is an attention operation in which several heads build contextual representations by comparing positions within the same input...
Multimodal Transformer - A multimodal Transformer is a model that uses Transformer-based processing to work with information from more than one modality, such as text and images, or...
Natural language generation - Natural language generation, or NLG, is the production of human-language text by a computer system.
Negative sampling - Negative sampling is a training technique that contrasts observed positive examples with a selected set of negative examples instead of comparing against...
No one right answer - No one right answer, often abbreviated NORA in generative-AI evaluation discussions, describes a prompt that can have multiple acceptable responses.
One right answer - One right answer, often abbreviated ORA in generative-AI evaluation discussions, describes a prompt with a single correct answer under its stated assumptions.
Oversampling - Oversampling increases the representation of selected examples or classes during training, commonly to address a class-imbalanced dataset.
Parameter-efficient fine-tuning - Parameter-efficient fine-tuning, or PEFT, adapts a pretrained model while training substantially fewer parameters than full fine-tuning.
Personal access token - A personal access token, or PAT, is a secret credential that software can present to a service when making authenticated requests.
PLM - PLM stands for pretrained language model: a language model whose parameters have already been learned through an earlier training stage before it is used or...
Pre-training - Pre-training is an earlier stage of model training that learns broadly useful patterns before the model is used directly or adapted to more specific tasks.
Pretokenization - Pretokenization is an initial division of text into pieces before a tokenizer applies its vocabulary or subword segmentation method.
Probabilistic language model - A probabilistic language model assigns probabilities to sequences of language units, usually words or tokens, or to the next unit given its context.
Recursive Language Model - In the supplied ontology's sense, a Recursive Language Model uses recursive neural composition over structured language, such as a syntactic tree, to...
Reference text - Reference text is a prepared example of an appropriate response used when evaluating generated text.
Reflection - Reflection is a workflow strategy in which an AI system examines an earlier output or action result before deciding what to do next.
Rejection sampling - Rejection sampling is a method for obtaining samples from a target probability distribution by generating candidates from a convenient proposal distribution...
Representation learning - Representation learning is the process of learning useful ways to encode data from examples, rather than relying entirely on manually designed features.
Reproducing kernel Hilbert space - A reproducing kernel Hilbert space, usually abbreviated RKHS, is a space of functions with an inner product in which evaluating a function at a point is a...
Resampling - Resampling is the repeated construction of samples or data partitions from available data to investigate statistical variability, evaluate models, or...
Response - A response is the output a generative AI model or application returns in reaction to its input.
Response set - A response set is the collection of outputs produced by a language model for a corresponding collection of prompts.
Sampling - Sampling means drawing an outcome according to a probability distribution.
Sampling rate - Sampling rate is the number of signal measurements taken or represented per second, usually expressed in hertz.
Sampling with replacement - Sampling with replacement is a selection procedure in which an item remains eligible to be selected again after it has been drawn.
Self-attention - Self-attention is a neural-network mechanism that updates representations of positions in a sequence by combining information from positions in that same...
Sequence-to-sequence (seq2seq) - Sequence-to-sequence, often shortened to seq2seq, describes a model or task that transforms an input sequence into an output sequence.