Does every Cauchy sequence converge?

It converges within its space when that space is complete. Otherwise its candidate limit may be absent.

Can small consecutive changes be misleading?

Yes. The harmonic partial sums illustrate how individually shrinking increments can accumulate indefinitely.

Does a convergent sequence have to be Cauchy?

Yes, in a metric space. Once terms are sufficiently close to the limit, they are close to each other.

AI Glossary / Mathematics and optimization / Intermediate

Cauchy sequence

Definition

A Cauchy sequence is a sequence whose terms eventually become arbitrarily close to one another. For every positive tolerance, however small, there is a point after which the distance between any two terms is below that tolerance. The definition compares terms with each other without first identifying a proposed limit.

What it means

The word eventually is essential. Early values may be widely separated, but sufficiently late values must form a tightly controlled group. It is not enough for one pair of values, or even consecutive values, to be close.

The definition depends on a distance, or metric. For real numbers, distance is the absolute difference. For vectors, a chosen norm can supply the distance. This makes the concept useful when studying sequences of parameter vectors or approximations, although an algorithm's observed behavior over a finite run cannot by itself establish the infinite property.

How it works

Write the sequence as x_1, x_2, and so on. It is Cauchy when, for every epsilon greater than zero, there exists an index N such that the distance between x_m and x_n is less than epsilon for all m and n at least N.

Every convergent sequence is Cauchy. The reverse implication requires a complete space: one in which every Cauchy sequence has a limit belonging to that space. The real numbers and finite-dimensional real vector spaces with their usual norms are complete.

This distinction matters because the limit may be missing from a restricted space. The sequence can satisfy all the pairwise closeness requirements while approaching a point that the space excludes. A proof therefore needs both the Cauchy property and the appropriate completeness assumption before concluding convergence within the space.

Three practical examples

1. Numerical analysis: prove that successive approximations settle toward a real solution by bounding the entire remaining tail, even before writing the solution explicitly.

2. Optimization theory: analyze whether parameter iterates become mutually close under a specified norm. This differs from checking whether their objective values stop changing visibly.

3. Function approximation: investigate whether a sequence of functions is Cauchy under a chosen function-space metric. Pointwise closeness and uniform closeness are different requirements, so the metric must be stated.

Think of it this way

Think of a group of late arrivals gathering inside an increasingly small meeting area. Once the required stage is reached, every later arrival must be close to every other later arrival.

The analogy has a boundary: inspecting today's arrivals cannot prove where all future arrivals will go. A Cauchy argument supplies a rule covering the entire future tail, rather than relying on a short period of apparent stability.

Worked example

Consider x_n=3−2^(−n), with n beginning at 1. The first terms are 2.5, 2.75, 2.875, and 2.9375.

For m and n at least N, both correction terms lie between zero and 2^(−N). Consequently, the absolute difference between x_m and x_n is at most 2^(−N). Given a tolerance of 0.01, choosing N=7 works because 2^(−7)=1/128=0.0078125, which is below 0.01.

For any smaller positive tolerance, choose a larger N with 2^(−N) below it. This establishes the Cauchy property for every tolerance, not only for the displayed numbers. Since these are real numbers, completeness ensures a real limit; the formula also directly identifies that limit as 3.

Contrast this with the partial sums of 1+1/2+1/3+…. Consecutive increments approach zero, yet the difference between the nth and 2nth partial sums is at least n/(2n)=1/2. Arbitrarily late terms can therefore remain separated by at least one half. Small individual steps alone are insufficient.

Illustrative case study

Fictional case: an analytics team sees tiny parameter changes in its iterative estimator and initially labels the process convergent. A reviewer asks whether many small changes could accumulate into continuing drift.

For a simplified version, the team derives a bound on the sum of all future step lengths. That bound tends to zero, establishing the required closeness of every pair of sufficiently late iterates. The team separately checks that the parameter space is complete. The exercise strengthens the mathematical explanation; it does not prove that the estimator is accurate or optimal.

Common misconceptions and mistakes

Do not confuse a Cauchy sequence with a sequence whose adjacent differences merely approach zero. Do not infer convergence inside an arbitrary space without checking completeness.

Another mistake is treating Cauchy behavior of loss values as Cauchy behavior of parameter vectors. Different vectors may have the same loss. Finally, a finite stopping tolerance is a practical decision rule, not an infinite-sequence proof and not a guarantee of a good solution.

Business use

For a business team, the concept helps evaluate claims that an iterative method is mathematically stable. Ask which objects are being compared, which distance is used, and what assumptions justify convergence.

Operational monitoring still needs separate checks for accuracy, resource limits, and sensitivity to data changes. A sequence can converge reliably to an unsuitable answer. Mathematical convergence and business usefulness address different questions.

Check your understanding

In the space of numbers strictly between zero and one, is x_n=1/(n+1) Cauchy, and does it converge within that space?

Answer: It is Cauchy under ordinary distance, but its real limit is zero, which the space excludes. This illustrates why completeness matters.

Related terms

Metric space
Convergence
Norm
complete space
numerical approximation
stopping criterion
optimization iterates

Learn next

study completeness and the difference between convergence of parameters and convergence of objective values. Then examine geometric tail bounds: they turn a bound on future step lengths into a useful Cauchy argument.

Convergence

Sources

Aalto Dictionary of Machine Learning, entries on Cauchy sequences and complete spaces: https://aaltodictionaryofml.github.io/ADictML_Main.pdf . Relevant definitions were read. The geometric sequence, harmonic comparison, and fictional review scenario are original teaching examples.

Previous: Cauchy-Schwarz inequality Next: Causal analysis