Sequential model-based optimization - Sequential model-based optimization, or SMBO, is an approach to optimizing an expensive objective by repeatedly fitting a surrogate model, using it to...
Shampoo optimizer - Shampoo is an adaptive gradient optimizer that uses the matrix or tensor structure of model parameters to build manageable preconditioners.
Slice sampling - Slice sampling is a Markov chain Monte Carlo method that samples a distribution by introducing an auxiliary height beneath its unnormalized density curve.
Sliced score matching - Sliced score matching is a method for learning a probability distribution's score by comparing score vectors along random directions.
Smoothing inference with twisted objectives - Smoothing inference with twisted objectives, or SIXO, is a method for learning probabilistic sequence models using sequential Monte Carlo targets that...
Sparse variational Gaussian process - A sparse variational Gaussian process is an approximate Gaussian process model that represents posterior information using a smaller set of inducing...
Spectral mixture kernel - A spectral mixture kernel is a covariance function built by modeling a stationary process's frequency spectrum with a mixture of Gaussian components.
Spike-and-slab distribution - A spike-and-slab distribution is a mixture with one component concentrated at or near zero and another spread over a broader range.
Stationary kernel - A stationary kernel is a covariance function whose value depends on the displacement between two inputs, rather than their absolute locations.
Statistical linear regression - Statistical linear regression, in nonlinear Bayesian estimation, approximates a relationship between uncertain variables by an affine transformation plus an...
Stein discrepancy - A Stein discrepancy measures how strongly a candidate distribution violates identities that should hold under a target probability distribution.
Stochastic gradient Hamiltonian Monte Carlo - Stochastic gradient Hamiltonian Monte Carlo, or SGHMC, is a posterior sampling method that combines momentum-based dynamics with gradients estimated from...
Stochastic neighbor embedding - Stochastic neighbor embedding, or SNE, is a nonlinear dimensionality-reduction method that represents objects in a lower-dimensional space by trying to...
Stochastic variational inference - Stochastic variational inference, or SVI, uses stochastic optimization to fit a tractable approximation to a Bayesian posterior distribution.
Stochastic weight averaging - Stochastic weight averaging, or SWA, forms a single neural-network model by taking an equal average of selected parameter vectors along a training trajectory.
Stochastic weight averaging Gaussian approximation - Stochastic weight averaging Gaussian approximation, usually called SWA-Gaussian or SWAG, fits a Gaussian distribution over neural-network parameters using...
Straight-through estimator - A straight-through estimator is a training rule that substitutes a usable derivative for the derivative of a hard or discrete operation during backpropagation.
Stratified resampling - Stratified resampling is a particle-resampling method that divides the unit interval into equally sized strata, draws one independent uniform value within...
Structural time series model - A structural time series model represents an observed series through interpretable latent components, such as a changing level, trend, seasonality, cycle...
Structured kernel interpolation - Structured kernel interpolation, or SKI, approximates a covariance kernel using interpolation from a set of inducing or interpolation points.
Structured mean field variational inference - Structured mean field variational inference approximates a difficult posterior using a tractable distribution that preserves selected dependencies among...
Submodular optimization - Submodular optimization seeks to minimize or maximize a set-valued objective with a diminishing-returns property.
Subset of regressors Gaussian process approximation - The subset of regressors Gaussian process approximation, or SoR, replaces a full Gaussian process covariance with a finite-rank covariance determined by a...
Subspace identification - Subspace identification is a family of methods for estimating dynamical state-space models from observed time-series data using matrix projections and...
Supervised principal component analysis - Supervised principal component analysis uses outcome information to guide a lower-dimensional representation of predictors.
Surrogate loss - A surrogate loss is an alternative training objective used in place of a task loss that is difficult to optimize directly.
Symmetric stochastic neighbor embedding - Symmetric stochastic neighbor embedding is a method for arranging high-dimensional observations in a low-dimensional map by matching symmetric probabilities...
Systematic resampling - Systematic resampling is a particle-filtering procedure that replaces weighted particles with equally weighted copies by placing evenly spaced selection...
TD(lambda) - TD(lambda), written TD(λ), is a family of temporal-difference learning methods that use eligibility traces to spread prediction errors across recently...
Test-time adaptation - Test-time adaptation is the adjustment of a trained model or its inference statistics using data encountered during deployment or evaluation.
Tied covariance in Gaussian discriminant analysis - Tied covariance in Gaussian discriminant analysis means that all modeled classes share one covariance matrix, while retaining their own mean vectors and...
Translation-invariant prior - A translation-invariant prior, in the location-parameter sense, assigns the same prior measure to equal-width parameter intervals regardless of where they lie.
Truncated singular value decomposition - Truncated singular value decomposition is a low-rank representation of a matrix obtained by retaining only its largest singular values and associated left...
Two-filter smoothing - Two-filter smoothing is a way to infer past hidden states by combining a forward filtering calculation with information propagated backward from later...
Underdamped Langevin dynamics - Underdamped Langevin dynamics is a stochastic process that evolves both position and velocity under a potential force, friction and random noise.
Unscented Kalman filter - The unscented Kalman filter, or UKF, is a recursive state estimator that uses carefully selected sigma points to approximate how uncertainty passes through...
Unscented Kalman smoother - An unscented Kalman smoother estimates past hidden states using observations from an entire interval, applying sigma-point approximations to nonlinear state...
Unscented transform - The unscented transform is a deterministic method for approximating the mean and covariance of a random quantity after a nonlinear transformation.
Variational Bayes expectation maximization - Variational Bayes expectation maximization, often called variational Bayesian EM or VBEM, is an iterative approach to approximate Bayesian inference that...
Variational expectation maximization - Variational expectation maximization, or variational EM, is an alternating optimization method for latent-variable models when exact posterior inference is...
Variational filtering sequential Monte Carlo - Variational filtering sequential Monte Carlo refers here to combining particle-based sequential inference with variational optimization of proposals or...
Variational message passing - Variational message passing, or VMP, is a way to organize variational inference as local computations and messages in a probabilistic graphical model.
Variational recurrent neural network - A variational recurrent neural network, or VRNN, is a sequence model that combines recurrent neural-network memory with latent random variables at each time...
Vector-Jacobian product - A vector-Jacobian product, or VJP, combines sensitivities at a function's outputs with its derivatives to produce sensitivities at its inputs.
Vector-quantized generative adversarial network - A vector-quantized generative adversarial network, usually called VQGAN, learns a discrete codebook for image representations while using reconstruction...
Vector-quantized variational autoencoder - A vector-quantized variational autoencoder, or VQ-VAE, is a generative representation model that maps encoder outputs to entries in a learned discrete...
Von Mises-Fisher distribution - The von Mises-Fisher distribution is a probability model for directions represented by unit-length vectors.