Must the direction vector have length one?

No. Scaling the vector scales the JVP by the same factor. A unit vector is useful when interpreting change per unit distance.

Does a JVP require a stored Jacobian?

No. Forward-mode differentiation can compute the product directly.

Is it always cheaper than reverse-mode differentiation?

No. The useful mode depends on the number of directions or output weightings needed and the structure of the computation.

AI Glossary / Probabilistic machine learning / Advanced

Jacobian-vector product

Definition

A Jacobian-vector product, or JVP, multiplies a function's Jacobian at a chosen input by an input-direction vector. It describes the first-order change in every output when the input moves in that direction, and can often be computed without constructing the full Jacobian matrix.

What it means

A function may accept many inputs and return many outputs. Its Jacobian records how each output changes with each input. Storing all those derivatives can be expensive, especially for a large model.

Often the application only needs the effect of one coordinated perturbation. A JVP answers that narrower question directly. The direction vector can represent a parameter adjustment, a sensitivity scenario or a vector used inside a numerical solver.

The result is local. It describes an infinitesimal change around the specified input, not the exact effect of a large intervention. A derivative of a predictive model also does not automatically identify a causal effect in the real world.

How it works

For a differentiable function f from n inputs to m outputs, the Jacobian J has m rows and n columns. Multiplying it by a length-n vector v produces a length-m vector Jv.

For a small scalar epsilon, f(x + epsilon v) is approximately f(x) + epsilon Jv. Forward-mode automatic differentiation propagates both values and their directional changes through the operations of the program, applying the chain rule as it goes.

This can avoid materializing the m-by-n matrix. To obtain the entire Jacobian, one could evaluate directions corresponding to all input basis vectors, but that is a larger task. A vector-Jacobian product instead propagates an output weighting back toward inputs and is associated with reverse-mode differentiation.

Three practical examples

A simulation team measures how several predicted outputs respond locally to a coordinated change in input settings.
An optimization routine uses derivative products inside a matrix-free linear solve rather than storing a large derivative matrix.
A machine-learning researcher combines a JVP with differentiation of a gradient to calculate a Hessian-vector product for curvature analysis.

Think of it this way

Think of a control panel with many knobs and gauges. Instead of making a complete table for every knob-gauge pair, ask what happens when the knobs move together in one specified pattern.

The JVP is the local rate of gauge movement for that pattern. The analogy does not mean physically turning knobs proves the model correct, or that the same rates hold after a large change in settings.

Worked example

Checking a directional prediction

Consider the original function f(x1, x2) = (x1 squared + x2, x1 times x2, x1 − x2). At x = (2, 3), its output is (7, 6, −1).

The Jacobian rows are (4, 1), (3, 2) and (1, −1). Choose direction v = (1, −2). Multiplication gives Jv = (2, −1, 3): the first output increases at rate 2, the second decreases at rate 1, and the third increases at rate 3 along this direction.

With epsilon = 0.01, the perturbed input is (2.01, 2.98). The first-order prediction is (7.02, 5.99, −0.97). Direct evaluation gives (7.0201, 5.9898, −0.97).

The small differences in the first two outputs come from second-order terms. The third component is linear, so its directional prediction is exact. This example distinguishes the derivative product from a finite displacement while showing how a small perturbation can help check an implementation.

Illustrative case study

Fictional case: reviewing a sensitivity calculation

A fictional engineering group has a differentiable simulator with many adjustable inputs and three summary outputs. It wants the local effect of a proposed coordinated adjustment, rather than every separate partial derivative.

The group computes a JVP for the adjustment direction and compares it with direct simulator runs at several small step sizes. It checks variable units and confirms that the direction vector describes the intended changes.

When larger adjustments produce visibly different behavior, the group evaluates those scenarios directly. It reports the JVP as local model sensitivity and does not present it as a guaranteed operational outcome or a causal estimate.

Common misconceptions and mistakes

A JVP is not a full Jacobian and is not necessarily a gradient. For a scalar output it is a directional derivative; the gradient contains sensitivities for every input coordinate.

JVP and vector-Jacobian product have different input and output roles. Even when dimensions happen to match, swapping them changes the calculation. Shape compatibility alone is not enough to establish that a derivative operation is the intended one.

Automatic differentiation applies programmed derivative rules; it does not remove floating-point error or resolve every nondifferentiability. At a kink, a library may use a convention or subgradient-like rule. Branches, discrete operations and custom derivative definitions deserve particular attention.

Business use

JVPs are useful when a team needs a few local sensitivities or an algorithm needs derivative products repeatedly. They can reduce storage compared with building a complete Jacobian, but actual runtime depends on the function and differentiation support.

Document the evaluation point, direction, units and any held-fixed inputs. Compare small hand-calculable cases with the implementation. Use finite differences as a diagnostic while recognizing that overly large steps introduce approximation error and overly small steps can suffer cancellation. Separate computational correctness from the validity of the model being differentiated.

Check your understanding

In the worked example, what is the JVP for direction (2, −4) at the same input?

Answer: It is (4, −2, 6), twice the original product. The derivative is linear in the direction vector at a fixed evaluation point, even though the underlying function is nonlinear.

Related terms

Learn next

Compute a JVP and a vector-Jacobian product for the same rectangular Jacobian, checking dimensions carefully. Then apply a JVP to a gradient function to understand matrix-free Hessian products.

Sources

Primary implementation documentation: JAX, The autodiff cookbook: JVPs and VJPs: https://docs.jax.dev/en/latest/301/cookbook.html

JAX JVP interface: https://docs.jax.dev/en/latest/_autosummary/jax.jvp.html

The three-output calculation and simulator case are original teaching examples.

Previous: Jacobian Next: Jarvis&Patrick clustering