Can every linear system be solved by plain Jacobi?

No. Its diagonal must support the update, and its iteration must converge for the chosen problem and start.

How does it differ from Gauss-Seidel?

Jacobi uses a frozen previous vector throughout a sweep; Gauss-Seidel uses newly available component values.

Is weighted Jacobi identical to ordinary Jacobi?

No. A relaxation parameter changes the update and can change its convergence properties.

AI Glossary / Mathematics and optimization / Intermediate

Jacobi method

Definition

The Jacobi method is an iterative algorithm for solving a linear system Ax=b. Each new component is calculated from the corresponding equation using only the previous iteration's values for the other components. This simultaneous-update structure makes the method simple and parallelizable, but convergence depends on the matrix; a nonzero diagonal alone does not guarantee success.

What it means

A linear system couples several unknown quantities. Jacobi temporarily treats the other unknowns as fixed at their latest completed estimates and calculates a replacement for each component. After every replacement has been calculated, the algorithm advances to the new vector.
This entry concerns Jacobi iteration for linear equations. The name Jacobi also appears in other numerical algorithms, including an eigenvalue method, so context matters. Here the aim is to reduce the discrepancy between Ax and b, not to diagonalize a matrix or calculate its eigenvectors.

How it works

Separate A into its diagonal D and remaining entries R, so A=D+R. Provided each diagonal entry is nonzero, compute x_new=D⁻¹(b−R x_old). The diagonal inverse means dividing each component by its corresponding diagonal coefficient, not constructing a general dense inverse.
Keep old and new values separate throughout a sweep. Using freshly updated values immediately changes the algorithm toward Gauss-Seidel. For convergence from every starting vector, the iteration matrix −D⁻¹R must have spectral radius below one. Strict row diagonal dominance is a useful sufficient condition. Symmetric positive definiteness by itself does not guarantee convergence of ordinary unweighted Jacobi.

Three practical examples

A numerical methods class can use Jacobi to demonstrate fixed-point iteration and residual monitoring on small linear systems.
A sparse numerical application can exploit independent component updates when its matrix and convergence behavior make the method appropriate.
A multigrid implementation can use a suitably weighted Jacobi variant as a relaxation component, with parameters selected for the problem.

Think of it this way

Think of several team members revising coupled estimates from the same frozen snapshot. Everyone submits a new estimate before the next round begins. The analogy is limited because agreeing on a synchronized process does not ensure the estimates converge; the numerical relationships determine whether the iteration settles down.

Worked example

Consider an original system 5x+y=11 and x+4y=6. Its exact solution is (2,1). Jacobi gives x_new=(11−y_old)/5 and y_new=(6−x_old)/4. Start at (0,0).
The first iterate is (2.2,1.5). The second is ((11−1.5)/5,(6−2.2)/4)=(1.9,0.95). The third is (2.01,1.025). Each component uses the previous pair, so the second coordinate of the first iterate is 1.5, not the value 0.95 that would result from immediately using the newly computed x=2.2.
At the third iterate, Ax has components 11.075 and 6.11. The residual b−Ax is therefore (−0.075,−0.11), whose largest absolute component is 0.11. The exact error is (−0.01,−0.025), a different vector. Residual monitoring checks the equations; it should not be mislabeled as directly measuring solution error.
The iteration matrix has rows (0,−1/5) and (−1/4,0). Its eigenvalues are plus and minus sqrt(1/20), with magnitude approximately 0.2236, below one. This supports convergence from any initial vector for this system.
For a counterexample, take a three-by-three matrix with diagonal entries 1 and every off-diagonal entry 0.75. Its eigenvalues are 2.5, 0.25, and 0.25, so it is symmetric positive definite. Nevertheless, for b=0 and initial vector (1,1,1), Jacobi produces (−1.5,−1.5,−1.5), then (2.25,2.25,2.25). The error grows by a factor of 1.5 in magnitude. Positive definiteness alone was not enough.

Illustrative case study

In a fictional example, Cedar Simulation implements a parallel linear solver. A small diagonal-dominant demonstration works, so the team initially assumes the same method will handle every symmetric positive-definite input.
Testing the counterexample above reveals the mistaken assumption. The team adds checks appropriate to its supported problem family and benchmarks alternative solvers. It also separates convergence failure from implementation failure. This invented case illustrates why an algorithm's mathematical conditions belong in its evaluation plan; it does not claim that one replacement solver is universally best.

Common misconceptions and mistakes

Parallel updates do not automatically make Jacobi the fastest solver. A cheap iteration can still require many rounds, and communication or synchronization can dominate distributed implementations.
Do not overwrite the old vector in a way that lets later components read new values while earlier ones read old values. That changes the iteration and its behavior. Also distinguish ordinary Jacobi from weighted Jacobi, which blends the old vector with a Jacobi update using a relaxation parameter.
A small change between successive iterates is not universally sufficient proof of an accurate solution. Inspect a suitably scaled residual and consider the conditioning of the system. For difficult problems, an apparent plateau can be a numerical limitation rather than successful convergence.

Business use

Jacobi is mainly an infrastructure technique for numerical computation. Its business relevance comes through simulation, scientific software, and other systems whose workloads include large linear solves. It can also serve as a transparent baseline against which a more sophisticated method is compared.
A useful performance report includes the matrix family, stopping criterion, iteration count, setup cost, and total runtime. Compare algorithms at equivalent accuracy requirements. For a production workflow, define what happens when iteration diverges or reaches its work limit; a returned vector should not silently be presented as a solved system.

Check your understanding

An implementation computes the new first component and immediately uses it in the second component's update. Is that the Jacobi sweep described here?

Answer: No. Jacobi requires all components in a new sweep to use the same previous vector.

Related terms

Fixed-point iteration
Residual
linear system
Gauss-Seidel method
spectral radius
diagonal dominance
weighted Jacobi
multigrid

Learn next

Reproduce three iterates and residuals for the two-variable example. Then run the positive-definite counterexample algebraically and explain why its growing sequence contradicts a blanket convergence claim based only on positive definiteness.

Sources

Primary numerical reference: Templates for the Solution of Linear Systems, The Jacobi Method: https://www.netlib.org/linalg/html_templates/node12.html . Source-list entry consulted: https://aaltodictionaryofml.github.io/ADictML_Main.pdf . The latter's sufficient-condition claim about symmetric positive definiteness is too broad for ordinary Jacobi; the original counterexample above explains the limitation.

Previous: Jackknife variance estimates for random forest Next: Jacobian