AI Glossary / Causal inference and experimentation / Advanced

Highly adaptive lasso

Definition

Highly adaptive lasso, or HAL, is a flexible regression method that represents complex relationships using a large collection of basis functions and controls their combined complexity with an L1 constraint or penalty. In its common zeroth-order form, those basis functions are indicator rules built from observed covariate values and their combinations.

What it means

Ordinary linear regression asks a small set of coefficients to describe a relationship. HAL can assemble a much richer function from many threshold-based building blocks. Its theoretical properties make it useful in statistical learning and in estimating component functions for causal analyses. That flexibility does not mean the method discovers causation or needs no assumptions. It is an estimation tool whose usefulness depends on the target, data and conditions under which it is applied.

How it works

Construct indicator basis functions from covariates, potentially including interactions, and fit a loss-appropriate regression with L1 regularization. The constraint is connected to a bounded sectional variation norm, a mathematical restriction on the function class. Tuning controls complexity; the choice appropriate for prediction need not be the choice required for a particular inferential procedure. Large basis expansions can be computationally demanding. Research on HAL-based inverse probability weighting uses additional undersmoothing and regularity conditions to obtain inferential properties; an arbitrary fitted HAL model does not inherit every such result.

Three practical examples

• An analyst models service time as a nonlinear function of workload and case complexity, comparing HAL's held-out performance with simpler alternatives.
• A causal evaluation uses HAL to estimate treatment probabilities from baseline variables. The study still needs a defensible treatment comparison and adequate overlap.
• A research team includes HAL among candidate learners for an outcome regression, recording both predictive performance and computational cost rather than assuming that greater flexibility is always better.

Think of it this way

Think of building a landscape from many adjustable steps. Some steps activate when one measurement crosses a threshold; others activate only when several conditions hold together. The regularization budget limits how much the fitted landscape can change overall. The resulting shape can be flexible without requiring the analyst to specify every bend in advance. Whether it captures the right pattern must still be assessed using appropriate evidence.

Worked example

Consider an original miniature basis expansion with workload x and complexity z. Define b1=1 when x≥5, b2=1 when z≥3, and b3=b1×b2. Suppose a fitted illustrative function is 10+2b1+3b2−b3. At x=6 and z=4, all indicators equal one, giving 14. At x=4 and z=4, only b2 is active, giving 13. The sum of absolute non-intercept coefficients is 2+3+1=6. This shows threshold bases, an interaction and an L1 coefficient budget. It is not a complete HAL fitting procedure: a real fit generates candidate bases from data, selects coefficients through optimization and tunes the complexity under the chosen setup.

Illustrative case study

A fictional maintenance company wants to predict repair duration from pre-job measurements. A straight-line model misses changes around workload thresholds, so analysts evaluate a HAL-based model alongside other candidates. They separate training and evaluation records by the operational structure needed to avoid leakage, document the generated basis size and monitor runtime. They also inspect errors on unusual jobs where data are sparse. If the model later supplies an outcome regression for a causal study, the team revisits the study design and inference requirements. Predictive success in the first project is not treated as proof of causal validity in the second.

Common misconceptions and mistakes

HAL is not simply ordinary lasso applied to a few raw variables; the rich basis construction is central to the approach. It is also different from the separately named adaptive lasso. A sparse coefficient vector does not necessarily make the full fitted function easy for non-specialists to interpret. Another mistake is reading asymptotic results as a guarantee for every sample size or covariate dimension. Practical screening or restrictions on interactions can affect the fitted model, so implementation choices should be documented rather than hidden behind the method's name.

Business use

HAL is most relevant when an organization has statistical expertise and a need for flexible estimation within a carefully designed analysis. It can be evaluated as a predictor or as a component of a broader estimator. A business-facing assessment should compare accuracy, stability, runtime and maintenance effort with available alternatives. The project should also distinguish prediction objectives from inference objectives. A method selected for low validation error may need different treatment when the goal is a confidence interval for a causal quantity rather than a prediction for the next record.

People also ask

Does HAL mean the same thing as adaptive lasso?

No. They are distinct methods despite the similar names.

Are its basis functions always linear in the original variables?

No. The common indicator expansion represents nonlinear threshold patterns and interactions.

Does using HAL remove confounding?

Not by itself. It can estimate a component model, but causal identification requires additional assumptions and a suitable design.

Check your understanding

Why did the first worked prediction equal fourteen?

Answer: All three indicator terms were active, including the negative interaction coefficient.

What distinguishes the toy calculation from fitting HAL?

Answer: Its coefficients and bases were supplied rather than learned and tuned.

Why should a prediction-focused tuning choice be reconsidered for inference?

Answer: The inferential procedure may impose different conditions on estimation error and bias.

Related terms

• Lasso
• Nuisance function
• indicator basis
• nonparametric regression
• sectional variation norm
• targeted learning

Learn next

Construct a small indicator design matrix and examine how an L1 penalty changes the fitted function. Then study the distinction between prediction tuning and undersmoothing for a specified inferential method before applying theoretical results to practice.

More in Causal inference and experimentation

• Hamming distance - Hamming distance counts the positions at which two aligned sequences of equal length contain different symbols.
• Hamming loss - Hamming loss measures the fraction of individual label decisions that are incorrect.
• Heterogeneous treatment effect - A heterogeneous treatment effect is a causal effect that varies across individuals, groups or contexts.
• Holdout cross-validation - Holdout cross-validation is an imprecise phrase commonly used for evaluating a model on a held-out portion of data, sometimes repeated across several random...
• Human evaluation - Human evaluation is the assessment of an AI system's outputs or behavior by people using a defined task and judgment procedure.
• Human-operator-intervention - Human-operator intervention is a person's action or input to help an otherwise autonomous system handle an exceptional situation or complete a task.

Sources

Hejazi, Coyle and van der Laan, hal9001: Scalable highly adaptive lasso regression in R: https://joss.theoj.org/papers/10.21105/joss.02526 Ertefaie, Hejazi and van der Laan, Nonparametric inverse probability weighted estimators based on the highly adaptive lasso, abstract and paper record: https://arxiv.org/abs/2005.11303 The basis calculation and maintenance scenario are original educational illustrations.

Previous: Higher-order Markov model Next: Highway network