Can a sequence effect improve performance?

Yes; transferred learning can help, while fatigue or interference can hurt.

Is random order enough?

It helps address allocation concerns but does not erase carryover.

Must every participant receive every condition?

No; a between-participant design may be preferable for some questions.

AI Glossary / Statistics and probability / Advanced

Sequence effect

Definition

A sequence effect occurs when experiencing one condition influences a person's response to another condition presented later. In repeated-measures research, the order of exposure can therefore affect the comparison being made. This entry uses the supplied glossary's meaning: an earlier level of an independent variable changes the response to a later level.

What it means

People do not necessarily start each condition with a clean slate. They may remember a task, learn a strategy, become tired or change their expectations. A response recorded during condition B can consequently reflect both B and what happened during condition A.

Specify the mechanism and comparison; terminology varies. A carryover effect concerns influence remaining from an earlier condition. A period effect concerns change associated with the measurement occasion, such as general practice or fatigue, irrespective of the particular preceding condition.

How it works

In A-then-B testing, condition and order are confounded.

Researchers may allocate participants to different orders, such as AB and BA, and record responses separately by period and sequence. Equal representation can balance a common period effect in a simple additive comparison. It does not automatically eliminate carryover that differs between conditions.

Consider whether earlier exposure can realistically stop influencing later responses. A break or washout period may help in suitable settings, but learned knowledge can persist. Where carryover cannot be addressed adequately, a different design may be needed. Analysis must also account for the dependence between observations from the same participant.

Three practical examples

1. AI-interface testing: users learn a task strategy with the first interface and retain that strategy when trying the second, affecting the comparison.

2. Survey research: an earlier question makes a topic salient and changes how a respondent interprets a later question. The earlier content, not just its position, may matter.

3. Training evaluation: participants learn a method during one training condition, so later performance under another condition cannot be treated as if that learning never occurred.

Think of it this way

Think of comparing two maps after walking the same route twice. On the second walk, you may remember the turns. Your speed reflects both the map and your earlier experience; swapping the maps across walkers helps examine order, but does not make memory disappear.

Worked example

Consider two fictional interfaces, A and B, evaluated with comparable tasks. Suppose the expected score for a fresh participant is 50 with A and 54 with B. The stipulated direct difference is therefore 4 score points in favor of B.

For this constructed example, add 2 points to every second-period score because of general practice. Also suppose that using A teaches a transferable strategy that adds another 6 points when B follows it. Using B leaves no corresponding carryover when A follows. These assumptions define the example.

In sequence AB, the expected first-period A score is 50. The second-period B score is 54 + 2 + 6 = 62. The within-sequence B-minus-A difference is 62 - 50 = 12 points. It contains the direct difference, practice and A's carryover.

In sequence BA, B is first and scores 54. A comes second and scores 50 + 2 + 0 = 52. The B-minus-A difference is now 54 - 52 = 2 points. Both comparisons use B minus A.

Assume equal numbers of comparable participants are assigned to the two sequences. The equally weighted B mean is (62 + 54)/2 = 58. The A mean is (50 + 52)/2 = 51. Their difference is 7 points, which is also the average of the two sequence-specific differences: (12 + 2)/2 = 7.

Balancing the orders has canceled the common practice contribution to the overall contrast. It has not recovered the stipulated direct difference of 4. Half the B observations receive the extra 6 points from A's carryover, adding 3 points to the balanced contrast. Thus 4 + 3 = 7.

Now remove carryover while keeping the 2-point practice effect. Sequence AB would produce A = 50 and B = 56, giving a difference of 6. Sequence BA would still produce B = 54 and A = 52, giving a difference of 2. The average, (6 + 2)/2 = 4, matches the stipulated direct difference under these revised assumptions.

This illustrates a limitation of counterbalancing; observed sequence differences do not uniquely identify carryover. Real data can contain sampling variation, participant differences, unequal completion and other time-dependent changes.

No hypothesis test or uncertainty interval is calculated here. The scores are stipulated expected values used to trace the contributions algebraically.

Illustrative case study

Fictional Willow Design initially reports a 12-point advantage for B after testing everyone in order AB. Its reviewer notices that users learned a reusable strategy while using A.

Willow considers both presentation orders and examines responses by period. The worked example shows why even the balanced seven-point contrast would not automatically isolate the direct interface difference. The team plans an evaluation that addresses learning explicitly before claiming that B is intrinsically better. This invented case illustrates a design problem.

Common misconceptions and mistakes

"Using the same participants removes every source of bias." Earlier exposure can change later responses.

"Counterbalancing eliminates all sequence effects." It can balance some influences while differential carryover remains.

"A break resets participants completely." Memory and learned skills may persist.

"Different results across orders prove carryover." Other explanations and sampling uncertainty must be considered.

Business use

Document the order in which users see products, prompts, training materials or survey questions. Keep that information alongside outcomes.

For AI-assisted analysis, check that the report separates direct condition differences from practice and carryover assumptions. A larger sample does not resolve a design in which those contributions remain inseparable. Match the design to the operational question, including whether real users will encounter a tool fresh or after relevant training.

Check your understanding

Why is B's score 62 in sequence AB?

Answer: Its baseline 54 gains 2 practice points and 6 carryover points.

Why is the balanced contrast 7 rather than 4?

Answer: A's carryover adds 3 points to B's overall mean.

What does removing carryover change?

Answer: The balanced contrast returns to 4 under the example's additive assumptions.

Related terms

Order effect
Carryover effect
Counterbalancing
Confounding
period effect
repeated measures
crossover design
practice effect

Learn next

Replace A's six-point carryover with a two-point carryover and keep the other assumptions. Recalculate both sequence contrasts and the balanced difference, explaining which contribution remains.

Sources

David C. Howell, Statistical Methods for Psychology glossary, Sequence effect: https://www.uvm.edu/~statdhtx/methods8/Glossary/Glossary.html

Office for Health Improvement and Disparities, Crossover randomised controlled trial: comparative studies, order, carryover and period considerations: https://www.gov.uk/guidance/crossover-randomised-controlled-trial-comparative-studies

Previous: Sequence analysis in social sciences Next: Sequence form