Does RMHMC change the desired posterior?

Correctly constructed, it changes the proposal dynamics while preserving the intended target marginal.

Is the metric a covariance or inverse covariance for momentum here?

Under the stated convention, G(q) is the conditional momentum covariance; its inverse appears in the quadratic energy.

Is it automatically faster than ordinary HMC?

No. Improved exploration must outweigh metric and integration costs, which depends on the problem and implementation.

AI Glossary / Probabilistic machine learning / Advanced

Riemann manifold Hamiltonian Monte Carlo

Definition

Riemann manifold Hamiltonian Monte Carlo is an MCMC method that uses a position-dependent metric to adapt Hamiltonian sampling dynamics to local features of a target distribution. Unlike ordinary HMC with a fixed mass matrix, it allows the momentum distribution and geometry of motion to vary across parameter space.

What it means

A posterior can be narrow in some directions and broad in others, with those directions changing from one region to another. One fixed scaling may not describe that geometry well. RMHMC uses a local positive-definite matrix to guide proposals through such variation.
The objective remains sampling the target distribution, not finding only its highest-density point. Geometry-aware proposals may improve exploration, but the extra matrix calculations, derivatives, and numerical integration can be expensive. A sophisticated local geometry also does not automatically solve separated modes or a badly specified statistical model.

How it works

Let pi(q) be the desired density with respect to the chosen parameter coordinates and let G(q) be a smooth positive-definite metric. Draw momentum conditionally as p | q ~ Normal(0, G(q)). Up to constants independent of q and p, the Hamiltonian is H(q,p) = -log pi(q) + 0.5 log det G(q) + 0.5 p^T G(q)^(-1)p.
The log-determinant term ensures that integrating out momentum leaves pi(q). Both kinetic and metric-normalization terms depend on position, so their derivatives contribute to the dynamics.
The resulting Hamiltonian is generally nonseparable. Ordinary explicit leapfrog formulas for a fixed mass matrix cannot simply be reused unchanged. The original method uses a generalized integrator with implicit steps and a Metropolis correction. Numerical solver accuracy, reversibility, and the requirements of the chosen integration scheme must be respected.

Three practical examples

Bayesian parameter estimation: a researcher samples a posterior whose local scales and correlations change substantially across the region of interest.
Nonlinear system modeling: a team explores parameter uncertainty in a dynamical model where different parameter combinations produce similar outputs and curved posterior structure.
Sampling-method research: an analyst compares local-metric and fixed-metric samplers using effective sample size per unit time rather than acceptance rate alone.

Think of it this way

Think of navigating terrain with a map whose scale and preferred directions change locally. A navigator can adjust movement to the terrain instead of using one global stride rule. However, the map must be included consistently in the movement calculation; merely changing stride lengths without the associated corrections can lead to the wrong sampling behavior.

Worked example

Consider an original one-dimensional target pi(q) proportional to exp(-q^2/2), and choose illustrative metric G(q) = 1 + q^2. This positive metric is deliberately simple; it is not claimed to be an optimal metric for the target.
Ignoring a common additive constant, H(q,p) = q^2/2 + 0.5 log(1 + q^2) + p^2/[2(1 + q^2)]. At q = 0, p = 1, the energy is 0.5 and velocity dq/dt = p/G(q) = 1. At q = 1, p = 1, the energy is 0.5 + 0.5 log 2 + 0.25, approximately 1.09657, and velocity is 1/2.
These are two separate phase-space points, not a proposed energy-preserving trajectory. At the second point, the momentum derivative is -q - q/(1 + q^2) + p^2 q/(1 + q^2)^2 = -1 - 1/2 + 1/4 = -1.25. The extra terms come from the changing metric.
If the log-determinant term were omitted, integrating exp[-q^2/2 - p^2/(2G(q))] over p would produce a marginal proportional to pi(q) sqrt(G(q)). That would overweight some positions and change the intended target.

Illustrative case study

In a fictional project, Larch Engineering fits a nonlinear simulator to laboratory measurements. A fixed-metric sampler mixes slowly along a curved parameter region. Researchers evaluate an RMHMC implementation with a documented metric and compare it against a carefully reparameterized ordinary HMC model.
They record computation time, integration failures, effective sample sizes, and agreement between independent chains. They also test small cases where direct numerical integration provides reference moments. The case illustrates a controlled comparison: fewer correlated draws per iteration would not justify a method if each iteration costs substantially more or numerical errors distort the result.

Common misconceptions and mistakes

A position-dependent metric is not the same as freely adapting an MCMC algorithm from its entire past history. Given a fixed metric function, its value changes with the current position as part of the specified transition mechanism.
The Hessian of a negative log density is not always positive definite, so it cannot automatically serve as a valid metric everywhere. Omitting the metric determinant or its derivative generally changes the dynamics or target. An acceptance step also does not magically repair an arbitrary nonreversible or non-volume-preserving proposal without the appropriate correction. The numerical scheme and its assumptions matter.

Business use

Consider RMHMC when difficult local geometry is a demonstrated sampling bottleneck and the extra computation can be justified. Compare it with improved parameterization and well-tuned fixed-metric HMC before assuming that greater algorithmic complexity is necessary.
Document the target measure, metric construction, derivative calculations, integrator, solver tolerances, and diagnostics. Check analytic gradients against numerical derivatives in small examples. Assess effective sample size per second and stability across starting points. Preserve failed-run evidence, since apparently good summaries from a few successful chains can hide difficult regions.

Check your understanding

Why can the log determinant not be discarded as a constant in the example?

Answer: G(q) = 1 + q^2 varies with q. Its normalization factor is required so that integrating over momentum returns the standard-normal target rather than a target multiplied by sqrt(1 + q^2).

Related terms

Hamiltonian Monte Carlo
Momentum
Effective sample size
Riemannian metric
mass matrix
Fisher information
generalized leapfrog
Metropolis correction
posterior geometry

Learn next

Differentiate the scalar Hamiltonian and verify the velocity and momentum equations numerically. Then compare a constant metric with the position-dependent example, keeping the target density and integration-error checks explicit.

Momentum

Sources

Girolami and Calderhead, Riemann Manifold Langevin and Hamiltonian Monte Carlo Methods, original2011 article, Section6, university-hosted copy: https://statistics.fas.harvard.edu/sites/g/files/omnuum10116/files/statistics-2/files/xlm_jroystatsocb73-2_123-214_2011.pdf Xu and Ge, original2024 research abstract documenting practical integration challenges: https://proceedings.mlr.press/v235/xu24i.html Girolami and Calderhead Section 6, equations 13–18, independently consulted for metric normalization, dynamics and implicit integration.

Previous: Ridge regression Next: Risk-averse