Can a single-arm study contain many participants?

Yes; arm count and sample size are different.

Does adding a historical benchmark make it randomized?

No; the comparison remains externally supplied and depends on its suitability.

Is a single-arm study always useless?

No; it can provide valuable descriptive or feasibility evidence and, in particular settings, contribute to broader evidence.

Does a narrow confidence interval eliminate design limitations?

No; it describes specified sampling uncertainty, not every source of bias or causal ambiguity.

AI Glossary / Statistics and probability / Intermediate

Single-arm study design

Definition

A single-arm study design follows one group receiving an intervention without an internally assigned concurrent comparison arm. It can describe outcomes, feasibility and adverse events under that intervention, but it does not directly observe what would have happened to comparable participants without it. A study may compare its results with a benchmark or external information, yet that comparison requires additional assumptions. Single-arm refers to the treatment-group structure, not to the number of participants or measurements.

What it means

Observing improvement after an intervention is different from estimating the intervention's effect. Outcomes may change because of natural progression, regression to the mean, other services, selection or changes in measurement. A single-arm study can estimate a response proportion or average outcome in its enrolled group under a clear analysis plan. To attribute a difference to the intervention, it needs credible comparative information and assumptions that the design itself does not supply through random assignment to a control arm.

How it works

Define the population, intervention, outcome, follow-up period and target of estimation. Specify how missing data, withdrawals and events that alter outcome interpretation will be handled. Collect outcomes consistently and use an analysis appropriate to the endpoint. If a benchmark or external comparison is planned, justify its relevance in terms of eligibility, timing, measurement and other factors. Report the observed intervention-group result separately from any estimated comparative effect. A larger sample can improve sampling precision without resolving bias from an unsuitable comparison.

Three practical examples

Example 1: An early feasibility study follows one group using a new service to assess recruitment, adherence and operational problems. Example 2: A pilot AI support program records participant outcomes before and after implementation without a concurrent control group. Example 3: A clinical development program may study an intervention in one treatment group when a comparative design is difficult, with its evidential limitations explicitly evaluated. These examples serve different purposes and should not be treated as equally strong demonstrations of efficacy.

Think of it this way

Think of watching one road after a traffic change without simultaneously observing a suitable comparison road. You can record traffic on that road accurately, but you do not directly see its alternative history without the change. Historical records can provide context, yet weather, construction and travel habits may also have changed. The missing comparison is a question about interpretation, not a problem solved simply by counting more cars.

Worked example

Suppose a single-arm study enrolls 100 participants and observes a defined response in 30, giving 30%. Under a simplified independent Bernoulli model with complete outcomes, the estimated standard error is the square root of 0.30 times 0.70/100, approximately 0.04583. A basic normal-approximation 95% interval is about 21.02% to 38.98%; other interval methods may be preferable depending on the setting. If a historical benchmark is 20%, the observed difference is ten percentage points. That arithmetic does not establish a ten-point causal benefit, because comparability and benchmark uncertainty remain unresolved. If ten participants instead lacked outcomes, reporting 30/90 = 33.33% among observed participants would answer a different question from 30/100 and could be biased if missingness relates to response.

Illustrative case study

In this fictional case study, Pine Vale Assistance pilots an AI-supported onboarding service in one department. The team records completion rates, user difficulties and time spent, then compares them cautiously with a previous year's records. It discovers that eligibility rules and staffing changed between periods. The pilot is reported as evidence about implementation and observed outcomes, with historical differences described as contextual rather than definitive effects. A subsequent evaluation is planned with a comparison structure better suited to the causal question.

Common misconceptions and mistakes

Common mistakes include describing a before-and-after difference as the intervention's effect without qualification, ignoring changes in case mix and assuming a historical benchmark is known without error. Another error is confusing a single arm with a single subject. Missing outcomes and selective enrollment can materially alter the estimated response. Independent outcome assessment may reduce some measurement bias but does not create an untreated comparison. A precise estimate of the intervention-group outcome can coexist with substantial uncertainty about comparative benefit.

Business use

Businesses use single-arm pilots to learn about feasibility, workflow and user experience before larger evaluations. Such studies can reveal implementation failures or help refine measurement, but adoption claims should match their design. AI teams should state whether the goal is describing outcomes or estimating a causal improvement. Keep the denominator, follow-up window and missing-data handling visible. Where comparative conclusions matter, assess whether a randomized or otherwise credible controlled evaluation can address the unanswered question.

Check your understanding

What is directly estimated by 30 responses among 100 participants?

Answer: The response proportion under the study's intervention and population definition.

Why is subtracting a historical 20% not automatically a treatment effect?

Answer: The groups and conditions may differ and the counterfactual is not internally randomized.

What changes if the denominator becomes 90 observed participants?

Answer: The estimate targets observed cases and may be affected by missingness.

Does a single arm imply one person?

Answer: No.

Related terms

Estimand
Related terms include uncontrolled study
pilot study
historical control
external control
randomised controlled trial
selection bias and regression to the mean

Learn next

Learn next how a causal estimand differs from a descriptive outcome summary. Compare concurrent and external controls, then study how eligibility, follow-up and missingness affect comparability. When planning a pilot, define which feasibility questions it can answer and which effectiveness claims will require a stronger subsequent design.

Causal estimand

Sources

Sources: European Medicines Agency, Establishing efficacy based on single-arm trials submitted as pivotal evidence in a marketing authorisation, https://www.ema.europa.eu/en/scientific-guidelines/establishing-efficacy-based-single-arm-trials-submitted-pivotal-evidence-marketing-authorisation; adopted reflection paper, https://www.ema.europa.eu/en/documents/scientific-guideline/reflection-paper-establishing-efficacy-based-single-arm-trials-submitted-pivotal-evidence-marketing-authorisation-application_en.pdf. The numerical example and fictional service pilot are educational illustrations, not clinical recommendations or an assertion that any specific regulatory submission would be accepted.

Previous: Sinewave synthesis Next: Single customer view