Differentiable neural computer - A differentiable neural computer, or DNC, is a memory-augmented neural architecture that combines a neural controller with an external, addressable memory.
Dropout - Dropout is a neural-network regularization technique that randomly suppresses selected activations during training.
Echo State Network - An echo state network, or ESN, is a recurrent neural-network approach in which a dynamic reservoir transforms input sequences into internal states and a...
Efficiently updatable neural network - An efficiently updatable neural network, commonly called NNUE, is an architecture designed to reuse computations when successive inputs change only slightly.
ELU Function - The ELU function, short for exponential linear unit, is a neural-network activation function that is linear for positive inputs and follows a saturating...
Embedding layer - An embedding layer maps discrete identifiers, such as token or category IDs, to dense numerical vectors.
Embedding lookup layer - An embedding lookup layer retrieves numerical vectors from an embedding table using discrete input identifiers.
Exponential Function - An exponential function places the variable in the exponent of a fixed positive base.
Exponential Smoothing State Space Model - An exponential smoothing state space model is a statistical time-series model that expresses exponential smoothing through equations for observations and...
Extension neural network - An extension neural network, or ENN, is a specialized pattern-recognition approach that combines neural-network ideas with extension theory, also called...
Feed forward (control) - Feedforward control determines a control action from a reference signal, a measured disturbance, or other predictive information, without relying on the...
Feedback Network - In neural-network terminology, a feedback network contains connections through which information can return to an earlier part of the computation, creating...
Feedforward neural network - A feedforward neural network is a neural network whose prediction computation follows directed connections without cycles.
Fixed Effects Model - A fixed effects model is a statistical model that represents specified groups or factor levels with their own parameters.
Frequency principle/spectral bias - The frequency principle, also called spectral bias in closely related research, describes a tendency of many neural networks and training setups to learn...
Fully connected layer - A fully connected layer is a neural-network layer in which each output unit has a trainable weight for every input feature in the vector it receives.
Gating mechanism - A gating mechanism is a component that regulates how information contributes to a neural-network calculation, commonly by multiplying a signal by an...
GELU Function - The GELU function, short for Gaussian error linear unit, is a smooth neural-network activation defined by GELU(x) = x times Phi(x), where Phi is the...
Generalized Hebbian algorithm - The generalized Hebbian algorithm, or GHA, is an unsupervised learning rule for a linear neural network that learns leading directions of variation in its...
Generative topographic map - A generative topographic map, usually discussed as generative topographic mapping or GTM, is a probabilistic nonlinear latent-variable model for...
Gradient clipping - Gradient clipping is a training technique that limits the magnitude of gradients before an optimizer uses them to update model parameters.
Graph neural network - A graph neural network, or GNN, is a neural model designed to learn from data represented as a graph of nodes and edges.
Grokking - Grokking is a machine-learning phenomenon in which a model achieves strong performance on its training examples, remains poor on unseen examples for an...
Grossberg network - Grossberg network is a broad label associated with neural models developed by Stephen Grossberg, rather than one uniquely specified modern architecture.
Group method of data handling - The group method of data handling, or GMDH, is a family of inductive modeling methods that generate candidate models, estimate their parameters from data...
Hard Sigmoid Function - The hard sigmoid function is a piecewise-linear activation that approximates the S-shaped logistic sigmoid using a straight middle segment and flat lower...
Helmholtz machine - A Helmholtz machine is a probabilistic neural model that pairs a top-down generative network with a bottom-up recognition network.
Hidden layer - A hidden layer is an intermediate layer of a neural network between the supplied input and the final output.
Hierarchical temporal memory - Hierarchical temporal memory, or HTM, is a family of machine-intelligence models inspired by proposed principles of neocortical computation.
Hopfield Network - A Hopfield network is a recurrent neural model that stores patterns through its connection weights and retrieves them by allowing an initial state to evolve...
Hybrid Kohonen self-organizing map - A hybrid Kohonen self-organizing map is a model that combines a Kohonen self-organizing map, or SOM, with another learning, prediction, optimization, or...
Hyperdimensional computing - Hyperdimensional computing is a family of methods that represent information using very high-dimensional distributed vectors and manipulate those vectors...
Infomax - Infomax is an information-theoretic learning principle that seeks representations retaining as much information about their inputs as possible under...
Input layer - An input layer is the entry interface through which a neural network receives its data representation.
Instantaneously trained neural networks - Instantaneously trained neural networks are a family of feedforward models whose parameters can be assigned directly from training examples rather than...
Interactive activation and competition networks - Interactive activation and competition networks, or IAC networks, are recurrent connectionist models in which mutually supportive representations excite one...
Kohonen Network - A Kohonen network commonly refers to a self-organizing map, or SOM: an unsupervised neural model that learns representative vectors arranged on a predefined...
Large memory storage and retrieval neural network - A large memory storage and retrieval neural network usually refers to LAMSTAR, a historical neural architecture that combines modules based on...
Large width limits of neural networks - Large width limits of neural networks are mathematical descriptions of what happens when the number of units or channels in selected layers grows without...
Layer - A layer is a computational component of a neural network that transforms one or more inputs into outputs.
Layer norm - Layer norm, short for layer normalization, is a neural-network operation that normalizes activation values across specified feature dimensions within an...
Learning rule - A learning rule is a procedure that specifies how a model changes its adjustable parameters or internal state in response to data or feedback.
Least-squares Analysis - Least-squares analysis estimates model parameters by minimizing the sum of squared residuals: the differences between observed values and the model's fitted...
Lernmatrix - The Lernmatrix, German for learning matrix, is an early associative-learning architecture developed by Karl Steinbuch around 1960.
Liquid State Machine Network - A liquid state machine network is a reservoir-computing architecture that transforms a time-varying input into changing internal activity, commonly in a...
Manifold Learning - Manifold learning is a family of methods that seek a lower-dimensional representation of data whose structure may be curved or nonlinear within a...