Is dogleg derivative-free?

The usual least-squares dogleg method uses residual derivatives through the Jacobian.

Does it always take two steps?

No. The two segments define a candidate path; the algorithm selects one trial displacement along it.

Is it the same as Levenberg - Marquardt?

No. Both are used in nonlinear least squares, but they construct and control trial steps differently.

AI Glossary / Mathematics and optimization / Advanced

Powell's dog leg method

Definition

Powell's dogleg method is a trust-region optimization technique that selects a step along a two-segment path connecting a steepest-descent-based step to a Newton or Gauss - Newton step. It is particularly associated with nonlinear least-squares problems. The bend in that path gives the method its name. The aim is to use an effective local model without taking a step beyond the region where the model is considered trustworthy.

What it means

A nonlinear least-squares problem minimizes a sum of squared residuals, such as differences between predicted and measured values. Linearizing the residuals produces a quadratic model. A full Gauss - Newton step may work well near a solution but can be too aggressive elsewhere. A step along the negative gradient offers a simpler local direction. Dogleg combines these choices geometrically inside a trust region. It should not be confused with Powell's derivative-free direction-set method, which is a separate optimization algorithm despite sharing the same mathematician's name.

How it works

For an unscaled Euclidean trust region, construct the Gauss - Newton step and the point that minimizes the quadratic model along the negative-gradient ray, often called the unconstrained Cauchy step. If the full Gauss - Newton step lies within the allowed radius, use it. If the radius is too small to reach the Cauchy point, use the boundary point along the negative-gradient direction. Otherwise choose the boundary intersection of the segment from the Cauchy point to the Gauss - Newton point. Evaluate the actual objective at the trial point and compare actual reduction with predicted reduction. That agreement guides acceptance and adjustment of the radius. Implementations may use variable scaling and additional safeguards, so their geometric norm must be specified.

Three practical examples

A camera-calibration system can use a dogleg trust-region strategy to fit parameters from image residuals when the formulation and linear solver support it.
A scientific fitting program can minimize nonlinear measurement residuals while limiting steps that a local approximation predicts too optimistically.
A robotics estimation study can compare dogleg with Levenberg - Marquardt under equal accuracy and computational requirements. These are solver choices inside an estimation problem; neither choice makes incorrect observations or an unsuitable model reliable.

Think of it this way

Think of following a route with one bend: first head in a locally promising direction, then turn toward the destination suggested by a more detailed map. Stop where the permitted travel radius ends. The analogy explains the path shape, but the actual directions come from residual derivatives and a quadratic model, not from a literal spatial route or a general guarantee that the destination is globally best.

Worked example

Use the objective f(x, y) = (x − 1)² + 0.5(y − 1)², which is one half the squared norm of residuals [√2(x − 1), y − 1]. Start at (0, 0). The gradient is g = (−2, −1), and the quadratic model matrix is B = diag(2, 1). The full Gauss - Newton step solves Bp = −g, giving p = (1, 1), whose norm is √2. The negative-gradient ray's minimizing multiplier is (g·g)/(g·Bg) = 5/9, so its Cauchy point is (10/9, 5/9), with norm 5√5/9, approximately 1.242. For a trust-region radius of one, both points are outside, so dogleg selects p = (2/√5, 1/√5), approximately (0.8944, 0.4472). Its norm is exactly one and the objective becomes approximately 0.1639, down from 1.5. For radius two, the full step (1, 1) is allowed and reaches objective zero. An intermediate radius between approximately 1.242 and 1.414 uses the bent segment. Because this example is already quadratic, model and actual reductions agree exactly apart from rounding; nonlinear problems need the explicit agreement check.

Illustrative case study

In a fictional project, Elm Vision fits a small calibration model. An unrestricted Gauss - Newton trial sometimes predicts a much larger improvement than the nonlinear objective actually delivers. The team evaluates a dogleg strategy, records the ratio of actual to predicted reduction, and checks parameter scaling before comparing solver behavior. It validates the returned calibration on separate measurements. The case illustrates why trust-region diagnostics matter without asserting a real accuracy gain or treating a solver's successful termination as proof that the camera model is correct.

Common misconceptions and mistakes

The dogleg path is a practical approximation to the trust-region subproblem, not generally its exact global solution. A smaller trust region does not correct wrong derivatives. The basic formulas require suitable curvature or rank conditions; a singular or poorly conditioned Jacobian needs appropriate handling. The Gauss - Newton model is not the exact Hessian of every nonlinear least-squares objective. Also, a trust-region radius limits movement in a chosen scaled norm, so a numerical radius cannot be interpreted without knowing variable units and scaling. A local convergence result is different from globally minimizing a nonconvex fitting problem.

Business use

Use the method when nonlinear least-squares structure is available and derivative computation and linear solves are practical. Check the residual definition, weighting and Jacobian before tuning trust-region settings. Monitor rejected steps, predicted-versus-actual reduction and residual behavior, not only iteration count. Compare computational work and final fit against an appropriate alternative, since different linear-solver requirements can dominate performance. Verify how the selected implementation handles bounds, rank deficiency and robust losses. Keep fitting error separate from external predictive accuracy; a better numerical solution can still overfit or reveal a misspecified model.

Check your understanding

The full Gauss - Newton step lies inside the trust region. Must dogleg shorten it along the gradient direction?

Answer: No. In the standard construction it can use the full step, subject to the outer algorithm's evaluation and acceptance rules.

Related terms

Trust region
Jacobian
Gauss–Newton method
Cauchy point
nonlinear least squares
Levenberg–Marquardt

Learn next

vary the trust-region radius in the quadratic example and plot the selected steps. Then replace one linear residual with a nonlinear residual and compare predicted and actual reductions to see why radius adaptation becomes necessary.

Trust region

Sources

Sources: Ceres Solver authors, Solving Non-linear Least Squares, Trust Region Methods and Dogleg sections, https://ceres-solver.readthedocs.io/latest/nnls_solving.html . The two-variable calculation and fictional calibration case are original illustrations using an explicitly unscaled Euclidean norm.

Previous: Potts model Next: Powell's method