Is a paradigm the same as a protocol?

A protocol specifies the exact implementation; a paradigm is the broader experimental framework.

Can a paradigm have many versions?

Yes, but changes should be documented and their consequences examined.

Does an established paradigm guarantee causality?

No. Causal interpretation depends on the actual manipulation, assignment, controls, and assumptions.

AI Glossary / Statistics and probability / Intermediate

Paradigm (experimental)

Definition

An experimental paradigm is a recognizable framework for conducting a type of experiment, including its central task, manipulation, comparison conditions, and outcome measures. Researchers can implement different versions while retaining the framework's defining logic. In behavioral research, examples include the Stroop task, visual search, and task-switching procedures.

What it means

A paradigm connects an abstract research question with an observable procedure. Rather than asking vaguely whether conflicting information affects attention, a study presents controlled stimuli, specifies a response, and compares performance across conditions.
The term here means an experimental approach, not a broad scientific worldview or a programming style. It is also broader than a single software script. Two scripts can implement the same paradigm differently, and those differences can affect what the experiment measures.

How it works

Begin with the construct or mechanism of interest and identify the contrast that could provide evidence about it. Specify what participants or systems experience, what they must do, and what is measured.
Define the conditions precisely. In a Stroop-style color task, for example, the written word and its display color may agree or conflict, while the participant responds to the display color. The contrast concerns how that conflict affects performance under the chosen instructions.
Control features that could offer alternative explanations. These include stimulus frequency, difficulty, response mapping, presentation order, practice, and equipment timing. Randomization or counterbalancing should follow the design rather than being assumed from the paradigm's name.
Record both the intended manipulation and the actual implementation. Instructions, stimuli, software versions, exclusions, timing rules, and outcome calculations allow others to understand and reproduce the procedure.
Pilot the task to confirm that people understand it and the measurement system works. A short demonstration can illustrate an effect without providing a reliable individual assessment or a sufficiently powered research study.
Analyze results according to the design, retaining repeated measurements and clustering. Examine accuracy alongside speed when faster responses might reflect more mistakes. A task effect is evidence about the specified contrast; interpreting it as a unique psychological mechanism requires further argument.
For AI evaluations, adapt the task cautiously. A model may solve a superficially similar task through different cues, training exposure, or shortcuts. Similar performance does not by itself establish the same internal process as a human participant.

Three practical examples

Interface research: a team adapts a conflict task to study whether competing labels slow users' decisions under controlled conditions.
Cognitive experimentation: researchers use a visual-search framework to vary distractors and measure response time and accuracy.
AI assessment: evaluators define a repeatable task family with controlled conditions to investigate sensitivity to irrelevant information, while checking for shortcut strategies.

Think of it this way

Think of an experimental paradigm as a recipe's organizing method. It identifies what is being compared and how the result is produced. Changing ingredients, timing, or preparation can preserve the general method while changing the outcome, so the exact recipe still matters.

Worked example

Consider a fictional conflict-task pilot with four participants. Their mean correct-response times in a consistent-label condition are 510, 540, 550, and 520 milliseconds. Their corresponding means in a conflicting-label condition are 590, 620, 610, and 600 milliseconds.
The within-person differences are 80, 80, 60, and 80 milliseconds. The average conflict cost is therefore (80 + 80 + 60 + 80)/4 = 75 milliseconds.
The consistent-condition average is 530 milliseconds, and the conflicting-condition average is 605 milliseconds, giving the same 75-millisecond contrast. This arithmetic defines an outcome for the chosen paradigm.
It does not show that the pilot has established a general attentional trait. The number of participants is small, the reliability of their condition means depends on the trials, and accuracy has not been supplied. If conflicting trials also produce more errors, reporting only correct-response speed would omit part of the performance pattern.
The study would need prespecified processing rules and an analysis appropriate to participant and trial variation before making inferential claims. The calculated difference is an illustrative descriptive result.

Illustrative case study

Illustrative fictional case: Fern Research wants to evaluate whether AI-generated hints distract users during a classification task. It adapts a conflict paradigm, documenting when hints agree or disagree with relevant information.
A pilot reveals that disagreeing hints are also longer. The team revises the materials to separate information conflict from reading length, then records both response accuracy and time. It describes the experiment as a specific hint-conflict task rather than a comprehensive test of intelligence. The case is fictional and illustrates construct and implementation discipline.

Common misconceptions and mistakes

Do not assume that naming a familiar paradigm makes a study valid. A modified task may change the construct, difficulty, or strategy it elicits. Do not treat a demonstration's visible effect as proof of reliable measurement for individuals.
Avoid confusing experimental control with real-world generalizability. Do not infer identical human and AI mechanisms from similar scores. Also distinguish the paradigm, the particular protocol, the statistical analysis, and the theory used to interpret the results.

Business use

Experimental paradigms help businesses frame focused evaluations of interfaces, training, and human-AI interaction. Their value is a clear, repeatable comparison linked to a decision. Teams should explain which operational behavior the task represents, which important features it leaves out, and how laboratory findings will be checked in realistic use.

Check your understanding

What is the descriptive effect in the worked example?

Answer: A 75-millisecond average increase in correct-response time under conflict.

Why is that not a complete assessment?

Answer: Reliability, accuracy, sampling, design, and interpretation remain to be evaluated.

Related terms

Construct validity
Counterbalancing
Replication (statistics)
Experimental design
protocol
operational definition
Stroop effect
task switching
reaction time
replication

Learn next

Next study operationalization and construct validity. Then compare alternative implementations of a task, inspect their timing and scoring rules, and examine whether their results generalize to the intended setting.

Construct validity

Sources

PsyToolkit, Stroop task: https://us.psytoolkit.org/experiment-library/stroop.html — developer documentation showing a concrete paradigm, implementation choices, outcomes, and demonstration limits. Gijsbert Stoet, PsyToolkit: A Novel Web-Based Method for Running Online Questionnaires and Reaction-Time Experiments: https://repository.essex.ac.uk/32449/1/Stoet_PsyToolkit-1.pdf — author paper describing task paradigms and implementations. Both primary sources were accessible; the pilot and fictional case are original.

Previous: Paper key Next: Parallel algorithm