Does smearing require normal log errors?

No; its appeal is empirical retransformation without that specific parametric assumption.

Is one factor always valid?

No; the relevant expected exponential error may depend on predictors.

Does exponentiating a predicted log value always give a median?

No; that requires appropriate conditional median assumptions.

Can smearing fix a poorly specified conditional mean model?

Not generally; retransformation addresses one mathematical issue and does not establish that the fitted relationship or residual distribution is correct.

AI Glossary / Statistics and probability / Advanced

Smearing retransformation

Definition

Smearing retransformation is a method for estimating an expected response on its original scale after fitting a regression on a transformed scale. Introduced by Naihua Duan, it averages inverse-transformed predictions across an empirical distribution of residuals. For a log-response model with a common error distribution, it multiplies the exponentiated fitted value by the average exponentiated residual. This addresses nonlinear retransformation, but it does not automatically correct model misspecification, selection bias or every form of changing error distribution.

What it means

The mean of an exponentiated quantity is generally not the exponential of its mean. Consequently, simply exponentiating a predicted log outcome does not usually estimate the arithmetic mean outcome. Smearing uses the observed residual distribution to account for this difference without imposing a normal error distribution. The target matters: an arithmetic mean, median and geometric mean are different summaries. Exponentiating a conditional mean of log Y is not automatically a conditional median unless additional assumptions about the log-error median hold.

How it works

Fit an appropriate transformed-response model and obtain residuals on that transformed scale. For a log model, exponentiate each residual and average those values to estimate a smearing factor when a common factor is justified. Multiply the exponentiated fitted log value by that factor. More generally, add residuals to the transformed prediction, apply the inverse transformation and average. If the error distribution changes with predictors, estimate an appropriate conditional correction rather than assuming one global factor. Assess uncertainty and predictive calibration on the original outcome scale.

Three practical examples

Example 1: An analyst models positive repair costs on a logarithmic scale but needs expected costs for budgeting. Example 2: An AI forecasting pipeline predicts log demand and must produce an estimate of mean demand in original units. Example 3: A process model uses a transformation to stabilize a response and then estimates an original-scale mean. Each application needs a clear target and a suitable residual distribution. The method's name does not justify arbitrary transformation choices or automatic use of a single pooled correction.

Think of it this way

Think of translating several plausible transformed-scale outcomes back into the original units and then averaging them. Translating only their average can give a different answer because the translation is curved. Smearing keeps that curvature visible by carrying residual variation through the inverse transformation. The calculation still depends on whether those residuals represent the uncertainty appropriate to the new case.

Worked example

Suppose a fitted log-response value is log 100, so naive exponentiation gives 100. Use two illustrative log residuals, minus log two and plus log two. Their exponentials are 0.5 and two, giving smearing factor (0.5 + 2)/2 = 1.25 and corrected mean prediction 125. Equivalently, inverse-transforming the two possible log outcomes gives 50 and 200, whose average is 125. Now suppose another predictor group has residuals minus log three and plus log three. Its factor is (1/3 + 3)/2 = 5/3, giving approximately 166.67 at the same fitted log value. Pooling the equally sized groups produces factor 35/24 and prediction approximately 145.83, which matches neither group's correction. These tiny residual sets illustrate arithmetic and conditional heterogeneity, not reliable estimation from two observations.

Illustrative case study

In this fictional case study, Redfern Repairs compares an AI cost model with a log-linear baseline. The original evaluation rewards accuracy on the log scale, but budgeting requires mean costs in currency units. Analysts apply a justified retransformation and check calibration separately for low- and high-variability job types. They find that a single pooled factor can conceal group differences, so they investigate a conditional approach and validate it on held-out work orders. The report distinguishes prediction targets rather than claiming one transformation solves every cost-model problem.

Common misconceptions and mistakes

Common mistakes include exponentiating a log prediction and calling it an arithmetic mean, confusing the smearing factor with a universal normal-theory correction, and ignoring heteroscedasticity. Another error is assuming that zero average log residual implies an average exponentiated residual of one. Nonlinearity prevents that conclusion. A few large residuals can strongly influence the factor, so tail behavior and finite moments matter. Residuals from a fitted model also require care when estimating uncertainty or evaluating performance without leakage.

Business use

Businesses use smearing when decisions depend on expected totals or average costs expressed in meaningful original units. It can make transformed regression useful without assuming a fully specified parametric error distribution. AI teams should compare original-scale calibration and loss, not just transformed-scale fit. Document the transformation, residual source, grouping or conditional correction and target summary. Prediction intervals require a separate justified procedure; a smearing-adjusted point estimate alone does not describe the full uncertainty of an individual future outcome.

Check your understanding

What factor is obtained from residuals minus and plus log two?

Answer: 1.25.

Why is the corrected prediction 125 rather than 100?

Answer: The average of the inverse-transformed outcomes differs from the inverse transformation of their log average.

Why is a pooled factor questionable in the second example?

Answer: The two groups have different residual distributions and therefore different conditional corrections.

Does the example estimate a reliable real-world factor?

Answer: No; it is a deliberately small arithmetic illustration.

Related terms

Conditional means
Related terms include log-linear model
retransformation bias
Jensen's inequality
conditional mean
heteroscedasticity
residual distribution
geometric mean and lognormal distribution
subject to its own regularity and representativeness assumptions

Learn next

Learn next how nonlinear transformations separate means and medians. Derive the original-scale expectation for a log model, then compare a constant correction with predictor-dependent corrections. Evaluate calibration and uncertainty using an appropriate resampling or modeling strategy, ensuring that fitting the transformation and correction remains inside the training process.

Sources

Sources: Naihua Duan, Smearing Estimate: A Nonparametric Retransformation Method, Journal of the American Statistical Association, https://www.tandfonline.com/doi/abs/10.1080/01621459.1983.10478017; university-hosted paper, https://people.stat.sc.edu/hoyen/pastteaching/stat704-2022/Notes/Smearing.pdf. USGS, Estimation of Low-Flow Characteristics, https://pubs.usgs.gov/wsp/2418/report.pdf, provides a primary application of the exponentiated-residual average. The residual-based examples are original direct evaluations of the exponential transformation. The fictional repair-cost project illustrates target alignment and conditional error variation rather than claiming measured superiority of a specific estimator.

Previous: SMDP Next: Smoothed Bellman error embedding