AI Glossary / Statistics and probability / Intermediate

Hurdle model

Definition

A hurdle model describes an outcome using one component for whether it is zero or positive and another for its size once it is positive. In a count hurdle model, the positive component is a zero-truncated count distribution. This separates the occurrence of an event from its frequency among cases with at least one event.

What it means

A customer may make no purchases during a month, or make one or more. The factors associated with making any purchase need not affect the number of purchases in the same way.
A hurdle model represents these two questions separately. Crossing the hurdle means moving from zero to a positive outcome. It is a statistical decomposition, however, and does not by itself prove that two distinct causal mechanisms generated the observations.

How it works

Define the outcome and observation period carefully. Zero must represent an observed absence of events, rather than missing information, an excluded case, or a different exposure window.
Let q(x) be the probability of a positive count for covariates x. A binary model, often logistic regression, can describe this probability. The probability of zero is then 1 minus q(x). Some software instead parameterizes the zero probability, so check the direction before interpreting coefficients.
For positive outcomes, specify a count distribution truncated at zero. If f(y) is an ordinary count probability function, the positive part assigns probability f(y)/(1 - f(0)) for y greater than zero. Truncation renormalizes the remaining probabilities to sum to one.
The full model gives P(Y = 0) = 1 - q and P(Y = y) = q*f(y)/(1 - f(0)) for positive y. Common positive-count families include Poisson and negative binomial. Different covariates can be used in the two components when justified.
For a Poisson positive component with underlying rate lambda, the conditional positive mean is lambda/(1 - exp(-lambda)), not lambda. The unconditional mean multiplies this positive mean by q. Confusing these quantities can substantially distort a forecast.
With separate parameter sets and ordinary independent observations, the likelihood separates into the binary occurrence part and the truncated positive-count part. This makes fitting and interpretation convenient, but it does not eliminate uncertainty in either component.
A zero-inflated model differs: its count component can also generate zeros, alongside a separate zero-generating state. In an ordinary hurdle model, the positive component cannot generate zero. Many observed zeros alone do not establish which description is appropriate.
Hurdle models can accommodate fewer as well as more zeros than a corresponding ordinary count model, depending on their specification. They are not restricted to situations with excess zeros. Evaluate the full predictive distribution rather than selecting a model solely from the zero percentage.
Check predicted zero rates, positive-count behavior, tail probabilities, and overall forecasts on relevant held-out data. Sparse positive observations, separation in the binary part, dependence across repeated observations, and exposure differences all require attention. A more complex model can fit the training data while generalizing poorly.

Three practical examples

  • An AI service forecasts whether an account will submit any requests and, conditional on activity, how many requests it will submit.
  • A retailer separately models any monthly purchase and purchase count among active customers.
  • A maintenance team models whether a machine records any service events and the positive count over a clearly defined exposure period.

Think of it this way

Think of a venue with a doorway and an activity counter. The first model concerns whether someone enters. The second concerns the number of activities among those who enter. Multiplying the chance of entry by the average activity of entrants gives expected activity across everyone.

Worked example

Suppose a count hurdle model gives q = 0.4 and uses a zero-truncated Poisson positive component with underlying lambda = 2. Then P(Y = 0) = 0.6.
The ordinary Poisson zero probability is exp(-2), approximately 0.135335. The positive component therefore divides ordinary positive probabilities by 1 - exp(-2), approximately 0.864665.
The full probability of exactly one event is 0.4*[2*exp(-2)/(1 - exp(-2))], approximately 0.125214. The probability of at least two events is 0.4 - 0.125214 = 0.274786. These probabilities sum with 0.6 to one.
The conditional mean among positive cases is 2/0.864665, approximately 2.313035. The overall expected count is 0.4*2.313035 = 0.925214, not 0.4*2 = 0.8.
Across 100 comparable accounts, the expected total is about 92.52 events. This is an expectation under the model, not a guarantee that the realized total will equal that value.

Illustrative case study

In this fictional case, Fern Cloud Services notices many accounts make no requests while active accounts have variable request counts. Its team fits a hurdle model and discovers that an onboarding indicator is associated mainly with the probability of activity. It reports the positive-count and overall forecasts separately, validates both components on a later period, and avoids claiming that the observational association proves onboarding causes greater use.

Common misconceptions and mistakes

The positive component must account for truncation; fitting an ordinary Poisson model to positive-only data is generally a different likelihood. The underlying Poisson rate is not the positive conditional mean. A hurdle model is not interchangeable with a zero-inflated model.
Two fitted components do not prove two real-world causes. More zeros do not automatically justify a more complex model. Also, a coefficient in one component is not generally the same as its effect on the overall expected count.

Business use

The decomposition can help teams distinguish participation forecasts from intensity forecasts and plan capacity accordingly. Report both parts and the combined result, including uncertainty. Compare the model with simpler count baselines, and check whether small active segments or rare large counts dominate the business risk.

People also ask

Can the two parts use different predictors?

Yes, when the specification is justified.

Can the positive part predict zero?

Not in the ordinary count hurdle construction.

Is this only for excess zeros?

No; the independently modeled zero probability can also represent zero deflation.

Check your understanding

Why is the example's positive mean greater than two?

Answer: Conditioning a Poisson variable on being positive removes zero outcomes and renormalizes the distribution.

What is the overall mean?

Answer: The probability of a positive outcome multiplied by its conditional mean, approximately 0.925214 here.

Related terms

  • Poisson regression
  • Conditional expectation
  • Zero-truncated distribution
  • zero-inflated model
  • negative binomial regression
  • two-part model

Learn next

Learn truncated distributions and binary regression. Then compare hurdle and zero-inflated likelihoods, exposure handling, predictive calibration, and uncertainty for combined forecasts.

  • Binary regression

More in Statistics and probability

  • Hot deck imputation - Hot deck imputation fills a missing value using an observed value from a suitable donor record in the same current dataset or survey collection.
  • Hotelling's t-squared statistic - Hotelling's t-squared statistic, conventionally written T-squared or T^2, measures a multivariate departure while accounting for covariance among variables.
  • Huber regression - Huber regression fits a regression model using a loss that is quadratic for small residuals and linear for sufficiently large residuals.
  • Hybrid Bayesian network - A hybrid Bayesian network is a Bayesian network containing both discrete and continuous random variables.
  • Hypergeometric distribution - The hypergeometric distribution gives the number of items of a specified type in a fixed-size simple random sample drawn without replacement from a finite...
  • Hyperprior - A hyperprior is a prior distribution placed on a parameter that controls another prior distribution.

Sources

Zeileis, Kleiber and Jackman, Regression Models for Count Data in R: https://cran.r-project.org/web/packages/pscl/vignettes/countreg.pdf statsmodels, Hurdle and truncated count models: https://www.statsmodels.org/stable/examples/notebooks/generated/count_hurdle.html

Previous: Hungarian algorithm Next: Hybrid agent

  • Home
  • Daily AI Updates
  • Blog
  • AIFBL Academy
  • AI Glossary
  • Products and Tools
  • Store
  • Free CFO AI Prompt Pack
  • About Shihan Sheriff
  • Privacy Policy
  • Terms of Use
  • Refund Policy
  • Cookie Policy
  • Earnings Disclaimer