Is carryover always harmful?

No. Persistence may be desirable in practice, but it can complicate an experiment intended to isolate immediate treatment effects.

Does randomizing order solve the problem?

It helps with design balance and confounding, but does not guarantee removal of differential carryover.

Can a washout erase learning?

Not reliably. Persistent changes may require a different design rather than a longer interval.

AI Glossary / Statistics and probability / Intermediate

Carryover effect

Definition

A carryover effect occurs when an earlier treatment, condition or experience continues to influence a later measurement. It is especially important in crossover and repeated-measures studies, where the same participants encounter several conditions. The later outcome can then reflect both the current condition and what happened previously, complicating interpretation of the current treatment's effect.

What it means

Carryover is one way that observations depend on experimental history. A previous drug may remain active, an earlier interface may teach a transferable skill, or a prior task may create fatigue. These effects can differ by treatment and sequence. They are not necessarily the same as a period effect, such as a general improvement over time affecting everyone regardless of earlier treatment.

How it works

Before choosing a repeated-exposure design, consider whether effects are temporary and whether the underlying condition remains sufficiently stable. Randomize treatment sequences where appropriate and plan intervals or resets that address plausible persistence. A washout period allows some residual effects to dissipate, but it cannot erase permanent learning or irreversible changes. Specify how period, sequence and within-person dependence will be handled. Some designs cannot cleanly distinguish particular carryover effects from direct treatment effects. A nonsignificant carryover test does not establish that carryover is absent, especially with limited data. Design decisions should not rely solely on such a test.

Three practical examples

Interface evaluation: users who learn shortcuts in one version may perform better in a later version because of transferable practice.
Experimental research: a treatment with a lingering effect can influence a participant's response in a subsequent crossover period.
Training assessment: exposure to one instructional method may permanently change later task performance, making a simple within-person reversal unsuitable for isolating the second method.

Think of it this way

Think of tasting a strongly flavored food before judging a mild one. The second experience is shaped by the first even though the food has changed. Waiting or cleansing the palate may help, but the appropriate reset depends on what persists and how long it lasts.

Worked example

Consider a balanced two-period experiment with sequences AB and BA. In a simplified noiseless model, baseline performance is 50 points, treatment A adds 10 points and treatment B adds 4 points. There is no general period effect. Assume A leaves a carryover of 6 points in the next period, while B leaves none. For sequence AB, performance is 60 under A and 50 + 4 + 6 = 60 under B, giving an observed A-minus-B difference of zero. For sequence BA, performance is 54 under B and 60 under A, giving a difference of six. Averaging these within-person contrasts equally produces three points. The stipulated direct effect difference is 10 - 4 = six points. Balanced ordering therefore does not remove the bias from this differential carryover. If both carryovers were zero, both sequence-specific contrasts would equal six in this simplified example. The numbers demonstrate a mechanism; they are not estimates from a trial or a claim that all balanced designs have the same bias.

Illustrative case study

Fictional case: Cedar Learning compares two AI tutoring interfaces by having every participant use both. A pilot reveals that the first interface teaches a problem-solving strategy that participants retain during the second session. The team recognizes that a longer break may not remove acquired knowledge. It considers a parallel-group design and revised tasks, documents the trade-offs and specifies the analysis before the main study. This fictional case illustrates how persistent learning can affect design choice and does not report an observed improvement from either interface.

Common misconceptions and mistakes

Counterbalancing can help separate some order and period effects, but it does not automatically eliminate treatment-specific persistence. Carryover need not be biological and can involve memory, fatigue, expectations or behavior. A washout interval is a design feature, not proof that the effect has disappeared. Discarding later-period data only after seeing a carryover test can create additional analysis problems. Observing different sequence means does not by itself identify the precise carryover mechanism. The same individuals being measured twice also creates dependence even when no carryover exists, so the two issues must be handled separately.

Business use

Businesses evaluating products, workflows or training should consider whether the first exposure changes how people respond to the next. Record order, timing and previous experience. Use independent groups when persistent learning makes crossover comparisons unsuitable, while recognizing the different precision and sample-size implications. Where repeated exposure remains appropriate, justify resets, sequence allocation and the analysis model. For AI systems, consider both human learning and system adaptation: a personalized model or retained conversation history can also make a later condition depend on earlier interactions.

Check your understanding

Q: Why does the balanced worked example underestimate the direct A-minus-B effect?

Answer: A: A's residual benefit raises the later B outcome in sequence AB, reducing that contrast. B has no corresponding residual benefit in sequence BA, so the two distortions do not cancel.

Related terms

Sequence effect
Counterbalancing
crossover design
repeated measures
period effect
washout period
learning effect and within-person correlation

Learn next

add a common second-period improvement to the worked example and examine what balanced sequencing cancels. Then introduce different carryover amounts and identify which effects can be estimated under the chosen design before selecting an analysis model.

Sources

Sources: Howell, Statistical Methods for Psychology glossary: https://www.uvm.edu/~statdhtx/methods8/Glossary/Glossary.html ; Penn State STAT 509, crossover designs and carryover: https://online.stat.psu.edu/stat509/Lesson12 ; CONSORT extension for randomized crossover trials: https://pmc.ncbi.nlm.nih.gov/articles/PMC6667942/ ; Cochrane Handbook, analysis issues in crossover trials: https://www.cochrane.org/authors/handbooks-and-manuals/handbook/current/chapter-23 .

Previous: Carrier cloud Next: CART