Does NUTS choose the number of leapfrog steps?

It dynamically determines trajectory expansion subject to its stopping rules and implementation limits.

Is warmup part of the reported posterior sample?

It is normally excluded from retained inference draws because it supports adaptation and initialization.

Does the last simulated point have to become the next draw?

No. Valid candidate selection is part of the algorithm.

AI Glossary / Probabilistic machine learning / Advanced

No-U-turn sampler

Definition

The No-U-turn sampler (NUTS) is a Hamiltonian Monte Carlo method that adapts the length of each simulated trajectory instead of requiring a fixed number of integration steps. It builds and checks trajectories for signs of turning back, then selects a valid candidate using rules designed to preserve the target distribution.

What it means

Bayesian sampling aims to explore a distribution, not merely locate its highest point. Hamiltonian Monte Carlo uses gradients and auxiliary momentum to move through parameter space. Short trajectories can waste opportunities to travel; excessively long trajectories can loop back toward positions already explored.
NUTS addresses this trajectory-length choice. It does not remove every modeling or computation decision. Step size, scaling, posterior geometry and diagnostic checks still matter. A completed run is not by itself evidence that the samples adequately represent the desired posterior.

How it works

The sampler introduces momentum and uses a numerical integrator, commonly leapfrog, to simulate movement through the target's potential landscape. It expands a trajectory through repeated doubling, choosing forward or backward extensions. Turning checks inspect relevant parts of the resulting tree.
In the original unit-mass formulation, a check compares the displacement between a subtree's endpoints with each endpoint's momentum. A negative inner product signals that extending that end would reduce the distance between endpoints. Candidate-selection rules matter: simply stopping an ordinary forward simulation at its first turn and keeping that endpoint is not the full algorithm.
The original method also describes step-size adaptation using dual averaging. Adaptation is used during warmup and then fixed for retained sampling in that scheme. Implementations can differ in trajectory selection and metric handling; details should be checked before reproducing a particular version.

Three practical examples

Hierarchical estimation: an analyst fits a model with continuous group effects and uses NUTS to explore their joint posterior. Reparameterization may still be needed when the geometry is difficult.
Model comparison: a researcher compares parameter summaries from several independent chains, inspecting effective sample sizes and convergence diagnostics rather than counting iterations alone.
Computation planning: a team records gradient evaluations and trajectory depths alongside runtime. This helps explain why two models with the same number of retained draws require different amounts of work.

Think of it this way

Imagine exploring a curved track with momentum. Continuing around the track indefinitely eventually brings you back toward familiar territory. NUTS looks for evidence that extending the current excursion is becoming repetitive. However, it must also choose the next stopping location according to a sampling rule, rather than simply picking the most distant or most attractive point.

Worked example

Consider an original illustration of turning geometry, not a complete NUTS run. The target is a one-dimensional standard normal distribution. With unit mass, the potential energy is q squared / 2 and kinetic energy is p squared / 2. Start at position q = 0 with momentum p = 1.
For exact continuous dynamics, q(t) = sin(t) and p(t) = cos(t). Total energy remains 1/2. Compare the initial point with the point at t = pi/3. The displacement is square root of 3 divided by 2, approximately 0.8660, and the later momentum is 1/2. Their product is square root of 3 divided by 4, approximately 0.4330, which is positive. Extending forward increases the distance from the starting point locally.
At t = 2pi/3, the position is again approximately 0.8660, but momentum is -1/2. The displacement-momentum product is approximately -0.4330. The position alone is identical to the earlier position, yet the motion is now back toward the start. For these endpoint pairs, the initial momentum product remains positive; the later endpoint supplies the negative turning signal.
At t = pi, the trajectory returns to q = 0 with momentum -1. Continuing that far would erase the positional separation achieved earlier. This illustrates why trajectory length matters. Actual NUTS uses discrete integration, recursive checks and candidate selection, so these times are not a claim about its exact stopping point or accepted draw. Energy conservation here follows from the stipulated exact dynamics, not from assuming numerical integration is error-free.

Illustrative case study

Fictional company Cedar Analytics estimates regional demand using a hierarchical model. Its analysts run multiple chains and examine both parameter summaries and sampling diagnostics. One version requires long trajectories and produces warning signs in a narrow part of the posterior.
The team investigates parameter scaling and an alternative parameterization before simply requesting more iterations. It compares diagnostics and relevant predictive checks after refitting. The objective is dependable inference for the chosen model, not minimizing tree depth at any cost. This is a fictional workflow and does not report a successful deployment or measured performance improvement.

Common misconceptions and mistakes

NUTS is not an optimizer that returns a best parameter vector. Its draws are generally correlated, and a large draw count does not imply the same number of independent observations. Automatic trajectory selection does not guarantee exploration of separated modes.
A U-turn and a divergent trajectory are different concepts: one concerns turning geometry, while the other signals problematic numerical behavior. Raising a trajectory-depth limit does not automatically resolve divergences or a poor parameterization. Ordinary gradient-based NUTS also does not directly update discrete unknowns through continuous dynamics; such models need an appropriate treatment of those variables.

Business use

NUTS can support probabilistic models where uncertainty over continuous parameters is central to a decision. Its cost depends on gradient calculations and model geometry, so teams should budget for model diagnosis as well as sampling. Reliable summaries require checking the computation and assessing whether the model describes relevant data; excellent sampler diagnostics cannot repair a substantively inappropriate model.

Check your understanding

Why do the two example positions give opposite turning signals?

Answer: Their momenta point in opposite directions.

Why not keep the first turning endpoint automatically?

Answer: That shortcut omits the sampling construction needed to preserve the target distribution.

What should accompany posterior summaries?

Answer: Evidence about convergence, effective sample size, numerical issues and model adequacy.

Related terms

Hamiltonian Monte Carlo
Markov chain Monte Carlo
Effective sample size
Leapfrog integration
Dual averaging
Posterior distribution
Reparameterization

Learn next

Trace position and momentum for a Gaussian target, then study how numerical integration changes energy. Read the candidate-selection construction before implementing a turning-based sampler.

Sources

Hoffman and Gelman, The No-U-Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte Carlo, 2014, Sections 3.1–3.2 independently consulted: https://jmlr.org/papers/volume15/hoffman14a/hoffman14a.pdf . The exact-dynamics example and fictional case are original teaching illustrations.

Previous: No-regret learning Next: Node