Does the Speed Prior simply prefer the shortest program?

No; computational time also matters.

Does it always choose the fastest available application?

No; it is a prior over computationally generated data, not a shopping rule for software.

Is it an ordinary probability distribution over a fixed list of model names?

No; the original formulation uses a semimeasure over output prefixes.

AI Glossary / Statistics and probability / Advanced

Speed prior

Definition

The Speed Prior is an algorithmic probability proposal introduced by Jürgen Schmidhuber that incorporates computational resources into a preference for simple explanations. It assigns greater prior emphasis to data that can be generated efficiently, rather than considering description length alone. Its formal construction uses a universal computation scheme and resource-weighted contributions from programs. It belongs to theoretical work on inductive inference; it is not a standard setting that automatically makes an ordinary AI model run faster.

What it means

A short description can hide an enormous computation. A brief program might take an impractical time to generate the observations it describes, while a somewhat longer program produces them quickly. The Speed Prior treats this difference as relevant to simplicity. It modifies the way computational explanations are weighted before observing further evidence. Its conclusions depend on the formal reference machine, resource allocation and assumptions about the data source, so it should not be presented as an empirically established law that nature always favors the fastest explanation.

How it works

The original construction considers successive phases of a universal search procedure called FAST. Each phase allocates computation to candidate program prefixes, with shorter prefixes receiving favorable treatment. A phase-specific contribution to an output prefix is combined with a geometrically decreasing phase weight. This creates a semimeasure over finite binary strings: the combined mass of two one-bit extensions can be less than the mass of their common prefix. Conditional predictions use these output-prefix quantities. The formal machinery is more specific than assigning an arbitrary inverse-runtime score to a short list of models.

Three practical examples

Example 1: A theoretical researcher compares two ways to express simplicity when one short program is extremely slow. Example 2: An AI foundations seminar studies how resource-bounded universal search changes an inductive prior. Example 3: A research prototype examines a restricted finite program language to explore time-sensitive hypothesis weighting. The third example is an approximation within a chosen system, not the full universal construction. None of these examples establishes a practical speed improvement for an existing business model.

Think of it this way

Think of choosing between two recipes that explain how to make the same dish. One fits on a tiny card but requires years of preparation; the other is longer but can be completed promptly. A resource-aware notion of simplicity notices the difference. The analogy explains the motivation, while the actual Speed Prior weights computational descriptions and output prefixes through a precise mathematical construction.

Worked example

In the formal phase-mixture representation, the phase weights are 2 to the power minus i for i = 1,2,3 and so on. The first four weights are 1/2, 1/4, 1/8 and 1/16, summing to 15/16. The remaining weight is 1/16, or 0.0625. Because each phase-specific contribution for a given prefix is at most one, omitting every phase after four leaves an absolute contribution bounded by 0.0625. After eight phases, the corresponding bound is 1/256, approximately 0.00390625. This geometric calculation is not the Speed Prior value for any particular string: those values also require the program contributions. Nor does a small absolute tail guarantee a small relative error. If a prefix mass were only 0.001, a 0.0625 absolute bound would be much too loose for an accurate relative estimate.

Illustrative case study

In this fictional case study, Horizon Research runs a reading group on theoretical prediction. A participant proposes replacing its production model-selection process with a runtime-based score and calling it the Speed Prior. The group instead documents the formal assumptions, builds a small illustrative program-language experiment and labels it a restricted demonstration. Operational models continue to be judged with measured accuracy, latency and cost. The project uses the theory to clarify ideas about computational simplicity without claiming that a miniature experiment inherits universal prediction guarantees.

Common misconceptions and mistakes

Common mistakes include equating the Speed Prior with inverse inference latency, claiming it ignores program length, and assuming that computable approximation means inexpensive computation. The original work gives a finite approximation procedure for a requested precision, but universal enumeration can be prohibitively costly. Another error is treating speculative physical interpretations as established consequences about the actual world. Theorems apply under their mathematical premises. The semimeasure distinction also matters: ordinary probability intuitions may need adjustment when some computational paths do not produce a further output bit.

Business use

For business readers, the main value is conceptual: model simplicity can involve both representation and resources. This does not make the Speed Prior an immediate deployment recipe. Practical trade-offs between accuracy, cost, latency and maintainability should be measured in the intended environment. If a research team uses a time-weighted model prior inspired by the idea, it should state the restricted hypothesis class and exact weighting rule. Inspiration from a universal theory does not transfer its guarantees to an unrelated implementation.

Check your understanding

What do the first four geometric phase weights sum to?

Answer: 15/16.

What absolute tail bound follows from stopping after eight phases?

Answer: 1/256 under the stated bounded-contribution argument.

Does that calculate a particular string's prior?

Answer: No; its phase-specific program contributions are still needed.

Why distinguish absolute and relative error?

Answer: A small absolute bound can be large relative to a very small prefix mass.

Related terms

Kolmogorov complexity
Related terms include algorithmic probability
Solomonoff induction
universal search
Levin complexity
semimeasure and resource-bounded inference

Learn next

Learn next how a semimeasure differs from a probability measure and how program-prefix weights enter universal inference. Then examine FAST's allocation of computation and the distinction between finite approximability and feasible runtime. Read formal prediction guarantees together with their source assumptions and precision requirements.

Sources

Sources: Schmidhuber, The Speed Prior: A New Simplicity Measure Yielding Near-Optimal Computable Predictions, https://people.idsia.ch/~juergen/coltspeed/coltspeed.html; Speed Prior construction, https://people.idsia.ch/~juergen/coltspeed/node2.html; Close Approximation, https://people.idsia.ch/~juergen/coltspeed/node5.html. These primary author materials define the proposal. The geometric-tail illustration is original and independently checked; the research-group case is fictional.

Previous: Speech synthesis Next: Speedy search