AI Glossary / Statistics and probability / Intermediate

Probability of direction

Definition

Probability of direction, usually abbreviated pd, summarizes which side of a specified reference value has more posterior probability. For a parameter theta and reference value zero, it is the larger of P(theta > 0 | data) and P(theta < 0 | data). It describes posterior certainty about direction under the chosen model. It does not measure the magnitude, practical importance, or causal interpretation of an effect.

What it means

A Bayesian analysis produces a posterior distribution rather than just one estimated coefficient. Probability of direction reduces that distribution to a statement about its sign. If 95% of the posterior lies above zero, the positive direction has 95% posterior support.

The reference value must match the parameter's scale. Zero is natural for a difference, while one is often the no-change value for a ratio. State the favored direction alongside pd: the same numerical value could describe evidence for an increase or a decrease. Whether that direction is desirable depends on the business question.

How it works

Choose the parameter and its meaningful reference value before interpreting the result. Compute the posterior probability strictly above the reference and strictly below it, then retain the larger value. With equally weighted posterior draws, estimate these probabilities by counting draws on each side and dividing by the total number of draws.

For a continuous posterior with no probability mass at the reference, the two sides sum to one, so pd ranges from 0.5 to 1. If a discrete or mixture posterior assigns probability to the reference itself, the range can extend below 0.5. Preserve that mass instead of silently removing it.

Check posterior sampling quality and sensitivity to modeling choices. More draws improve numerical estimation of a given posterior; they do not correct a poor likelihood, an inappropriate prior, or biased observations. Report the posterior effect estimate and an uncertainty interval alongside directional probability.

Three practical examples

  • An experiment analyst summarizes posterior support for a positive change in completion rate after an interface change.
  • An operations team reports whether an estimated latency difference is more probably above or below zero.
  • A Bayesian regression dashboard displays directional support for a coefficient while separately showing its size and uncertainty.

Think of it this way

Think of a balance with posterior probability on either side of a marked reference point. Probability of direction tells you how much weight sits on the heavier side. It does not tell you how far the weight lies from the reference, so it cannot by itself describe effect size.

Worked example

In an original teaching example, suppose the posterior for a processing-time improvement is normal with mean 0.20 seconds and standard deviation 0.10 seconds. Positive values mean faster processing. The probability below zero is the standard normal probability below −2, approximately 0.02275.

Therefore P(improvement > 0 | data) is approximately 0.97725, and pd is 97.725% in the positive direction.

Now suppose a useful improvement must exceed 0.30 seconds. The posterior probability of exceeding this business threshold is the standard normal probability above 1, approximately 0.15866, or 15.866%.

The same posterior therefore has strong directional support but much weaker support for meeting the stated practical target. These calculations use an assumed posterior distribution, not a fitted real experiment. They also show why replacing the business threshold with zero can change the question being answered.

Illustrative case study

Fictional case study: Cedar Workflow compares two automated routing designs. Its Bayesian report shows a high probability that the new design improves speed. The product owner initially interprets that result as sufficient justification for a costly migration.

The analysis team adds the posterior distribution of the improvement and the probability of exceeding a pre-agreed minimum worthwhile gain. The migration discussion then considers expected benefits, uncertainty, and implementation costs. This fictional case illustrates how a directional summary can inform a decision without substituting for its practical criteria.

Common misconceptions and mistakes

A high pd is not the probability that the entire model is correct. It is also not automatically the probability of a meaningful effect or the probability that a future individual outcome improves. Those require different quantities.

Values near 0.5 for a continuous posterior indicate directional uncertainty; they do not establish that the effect is exactly zero. Conversely, a parameter constrained by the model to be positive can have pd equal to one without the data supplying strong evidence against a useful null model. Approximate correspondences with frequentist p-values should not erase the different conditioning and assumptions involved.

Business use

Probability of direction offers an accessible way to communicate Bayesian coefficient results. It can help teams identify whether estimated changes favor an increase or decrease while avoiding an exclusive focus on a point estimate.

A responsible dashboard should also name the reference value, parameter units, model, and relevant practical threshold. If decisions concern financial returns, service quality, or resource use, calculate those consequences explicitly. Monitor whether stakeholders confuse a large directional probability with a large benefit; the worked example shows that the two can differ substantially.

People also ask

Is pd always at least 50%?

Only when there is no posterior mass at the reference value. Discrete or mixture distributions can behave differently.

Does pd require posterior draws?

No. It can be calculated analytically when the posterior probabilities are available, or estimated numerically.

Is pd independent of the prior?

Its calculation uses the posterior, but that posterior generally depends on the prior and likelihood.

Check your understanding

Of 2,000 equally weighted posterior draws, 1,900 are negative and 100 are positive, with none zero. What is pd?

Answer: It is 1,900/2,000 = 95%, favoring a negative effect. This alone does not establish whether the negative effect is practically important.

Related terms

  • Bayesian inference
  • Effect size
  • Bayes factor
  • Posterior distribution
  • credible interval
  • region of practical equivalence
  • reference value
  • Monte Carlo error

Learn next

Learn to summarize posterior distributions with estimates and intervals. Then compare probability of direction with the probability of exceeding a practical threshold and with a region of practical equivalence. Study prior sensitivity and posterior sampling diagnostics before relying on fine numerical differences in reported probabilities.

  • Posterior sampling

More in Statistics and probability

  • Poisson distribution - The Poisson distribution is a probability distribution for a count taking the values 0, 1, 2 and so on.
  • Policy capturing - Policy capturing is a research method for describing how people combine information when making judgments or decisions.
  • Poly-Weibull distribution - The poly-Weibull distribution is a continuous time-to-event distribution constructed by adding several Weibull-type hazard functions.
  • Pólya urn model - A Pólya urn model is a probability model in which sampling changes the probabilities of future samples through a specified reinforcement rule.
  • Polynomial regression - Polynomial regression models an outcome using powers of one or more numerical predictors.
  • Polynomial trend coefficient - A polynomial trend coefficient is a numerical weight used with other such weights to test a specified trend across ordered group means.

Sources

The bayestestR authors document probability of direction, its direct calculation, reference values, and treatment of discrete or mixture posteriors: https://easystats.github.io/bayestestR/reference/p_direction.html

The normal-posterior calculation, draw-count exercise, and fictional case are original. Directional probability is explicitly distinguished from effect magnitude, model validity, and practical benefit.

Previous: Probability model Next: Probability of error

  • Home
  • Daily AI Updates
  • Blog
  • AIFBL Academy
  • AI Glossary
  • Products and Tools
  • Store
  • Free CFO AI Prompt Pack
  • About Shihan Sheriff
  • Privacy Policy
  • Terms of Use
  • Refund Policy
  • Cookie Policy
  • Earnings Disclaimer