Can a Jacobian be rectangular?

Yes. Its shape follows the numbers of outputs and inputs.

Is it constant for every model?

No. A linear map has a constant Jacobian, while a nonlinear map generally does not.

Does a large derivative prove a causal effect?

No. It describes sensitivity within the specified mathematical model, which is not by itself evidence about real-world causation.

AI Glossary / Mathematics and optimization / Intermediate

Jacobian

Definition

The Jacobian is the matrix of first-order partial derivatives of a vector-valued function. It describes how small changes in several inputs affect several outputs near a particular point. For a differentiable mapping from n inputs to m outputs, the usual convention gives an m-by-n matrix: one row per output and one column per input.

What it means

A model can have many adjustable inputs and return several quantities. A single derivative is then insufficient to describe all local sensitivities. The Jacobian collects the derivative of each output with respect to each input into an organized table.
Its meaning is local. A derivative measured at one input point need not describe behavior far away. If the mapping is nonlinear, the Jacobian can change from point to point. It can reveal directions that strongly affect outputs, directions with little first-order effect, and how intermediate transformations pass sensitivity through a larger calculation.

How it works

For f=(f_1,...,f_m), the entry in row i and column j is the partial derivative of f_i with respect to x_j. At a differentiability point x, a small perturbation h gives f(x+h) approximately equal to f(x)+J_f(x)h, with an error smaller than first order as h approaches zero.
The chain rule multiplies compatible Jacobians: for a composition g(f(x)), the Jacobian is J_g evaluated at f(x), multiplied by J_f evaluated at x. Derivatives can be obtained analytically, by automatic differentiation, or approximately by finite differences. The required method depends on the function and numerical setting.

Three practical examples

A robotics calculation can relate small joint changes to local changes in a tool's position or orientation coordinates.
An image-alignment algorithm can use derivatives of transformed image measurements when adjusting its parameters.
A machine-learning system can propagate local derivatives through layers or inspect sensitivity of selected outputs to inputs.

Think of it this way

Think of a local sensitivity panel: each column shows what a small nudge to one input does to all outputs. The analogy is limited because the panel is valid around a specified operating point. Turning a control a long way can move the system into a region where the old sensitivities no longer apply.

Worked example

Consider an original mapping f(x,y)=(x²+y,xy). Its Jacobian has rows (2x,1) and (y,x). At (2,3), the output is (7,6) and the Jacobian has rows (4,1) and (3,2).
Apply a small change h=(0.1,−0.2). The predicted output change is J h=(0.4−0.2,0.3−0.4)=(0.2,−0.1). The linear approximation therefore predicts output (7.2,5.9).
Direct evaluation at (2.1,2.8) gives f=(7.21,5.88). The approximation errors are (0.01,−0.02), arising from the omitted terms h_x² and h_x h_y. Halve the change to (0.05,−0.1): the omitted terms become (0.0025,−0.005), one quarter as large. This illustrates the second-order remainder of this particular polynomial example, not a universal error rate for every differentiable function.
The determinant at (2,3) is 4 times 2 minus 1 times 3, giving 5. For sufficiently small regions near that point, its absolute value describes local area scaling. At (0,0), however, the Jacobian has rows (0,1) and (0,0) and determinant zero. First-order sensitivity cannot provide a locally invertible two-dimensional linear approximation there.
Finally, combine the two outputs into g(u,v)=u+2v. Its Jacobian is the row (1,2). The composed derivative at (2,3) is (1,2)J=(10,5), matching direct differentiation of x²+y+2xy. Multiplication order and dimensions make the chain rule concrete.

Illustrative case study

In a fictional example, Fern Vision tunes a two-parameter alignment model. A large update based on a local derivative makes the alignment worse, even though the derivative calculation is correct. The team compares predicted changes with actual function evaluations at progressively smaller steps.
It finds a range where the local approximation is useful and adjusts its optimization procedure accordingly. This invented case illustrates why a correct Jacobian does not justify an arbitrary step size. It does not claim that a particular line search or optimizer will always solve the alignment problem.

Common misconceptions and mistakes

A Jacobian matrix is not the same as its determinant. The determinant is defined only for a square Jacobian and summarizes particular local scaling information. Rectangular Jacobians remain useful even though they do not have a determinant.
Do not confuse the Jacobian with the Hessian, which contains second derivatives of a scalar function. For a scalar output under the row-output convention, the Jacobian is the transpose of the column gradient.
Having all partial derivatives at a point does not alone guarantee differentiability there. Also, a zero first derivative need not mean the output is globally unaffected. For f(x)=x² at zero, the derivative vanishes even though finite changes in x change the output.

Business use

Jacobian calculations support fitting, calibration, sensitivity analysis, and many other numerical components in AI applications. A useful explanation specifies the input units, output units, evaluation point, and layout convention. A sensitivity of two means different things when the input is measured in seconds rather than milliseconds.
For large models, storing the entire Jacobian may be expensive. A computation may need only its product with a vector or a transposed product, which automatic differentiation can often evaluate without materializing every entry. Regardless of implementation, compare local predictions with representative perturbations when the approximation is central to a decision.

Check your understanding

A function takes four inputs and returns three outputs. Under the convention used here, what is its Jacobian's shape?

Answer: Three rows by four columns, with each row describing one output's partial derivatives.

Related terms

Learn next

Recalculate the worked perturbation and its exact output, then reverse the perturbation direction. Check the chain-rule dimensions before multiplying. Finally, compare a first-order approximation at a regular point with one at a point where the Jacobian loses rank.

Chain rule

Sources

Primary terminology source: Alexander Jung and collaborators, The Aalto Dictionary of Machine Learning, Jacobian matrix entry: https://aaltodictionaryofml.github.io/ADictML_Main.pdf . Original list context: https://people.csail.mit.edu/fredo/comp-photo-book/glossary.html . The matrix convention was checked in Aalto; all polynomial calculations and the fictional case are original.

Previous: Jacobi method Next: Jacobian-vector product