No; the standard version uses a normal approximation.

Is it related to a chi-square test?

For a two-by-two table without continuity correction, the corresponding Pearson chi-square statistic equals the pooled Z statistic squared.

AI Glossary / Statistics and probability / Intermediate

Two-proportion Z-test

Definition

A two-proportion Z-test is a large-sample hypothesis test comparing two independent population proportions. In its common form, it tests whether the proportions are equal by dividing the observed difference by a pooled standard error calculated under that null hypothesis. The resulting statistic is compared with a standard normal reference distribution.

What it means

Suppose two groups have different observed completion rates. Some difference can arise through random sampling or assignment even if their underlying success probabilities are equal.
The test assesses how unusual the observed difference would be under that equality model and the stated assumptions. It does not tell us the probability that the null hypothesis is true, and it does not by itself establish that a difference is important or caused by an intervention.

How it works

Define the binary outcome consistently for both groups. Record success counts x1 and x2 and denominators n1 and n2. The sample proportions are p-hat1 = x1/n1 and p-hat2 = x2/n2.
Specify the hypotheses before interpreting the data. The usual null is p1 = p2. The alternative may be two-sided, p1 not equal to p2, or directional if the question and analysis plan justify that direction.
Under equality, estimate the common proportion by pooling successes and observations: p-hat = (x1+x2)/(n1+n2). The null standard error is sqrt[p-hat(1-p-hat)(1/n1 + 1/n2)].
Calculate Z = (p-hat1 - p-hat2)/SE-null. For a two-sided test, the approximate p-value is 2 Phi(-absolute Z), where Phi is the standard normal cumulative distribution function. The absolute value matters when the observed difference is negative.
Check that the normal approximation is reasonable. Expected success and failure counts under the null should not be too small. Rules of thumb are diagnostic aids rather than universal guarantees; rare events or small samples can require alternative inference.
Check independence and the sampling design. Repeated outcomes from the same person, paired before-and-after responses, clustered allocation, and heavily weighted survey data are not automatically covered by the simple independent-binomial formula. A paired binary comparison may require McNemar's test instead.
Distinguish testing from interval estimation. A common elementary confidence interval uses separate sample proportions in an unpooled standard error. More accurate score-based interval methods are often useful, especially near boundaries. The pooled null test and an unpooled Wald interval need not be exact inverses.
Report the difference in proportions, uncertainty, and practical implications. Account for multiple comparisons or repeated interim testing when those are part of the analysis. A threshold crossing in one unadjusted test does not validate an entire experiment or remove concerns about missing outcomes and measurement quality.

Three practical examples

A randomized website experiment compares the proportion of accounts completing setup under two interfaces.
A quality analyst compares defect proportions from two independently sampled production processes.
An evaluation team compares independent groups' success rates while checking that repeated tasks from the same user do not violate the assumed unit of analysis.

Think of it this way

Think of two jars of colored beads. The observed color proportions may differ even if both jars were filled from the same mixture. The test compares that observed gap with the variation expected from the numbers of beads sampled, under a shared-mixture hypothesis.

Worked example

Consider an original example with 60 successes among 200 independent observations in group A and 40 among 200 in group B. The observed proportions are 0.30 and 0.20, giving a difference of 0.10, or 10 percentage points.
Under the equality null, the pooled estimate is (60+40)/(200+200) = 0.25. The standard error is sqrt[0.25 x 0.75 x (1/200 + 1/200)], approximately 0.0433013. Therefore Z = 0.10/0.0433013, approximately 2.30940.
The two-sided normal-approximation p-value is approximately 0.02092. At a prespecified 5% level for this single test, the result rejects equality under the independent-binomial assumptions. Under the null, each group has expected counts of 50 successes and 150 failures, supporting the large-sample approximation here.
For a separate illustrative 95% unpooled Wald interval, the standard error is sqrt[(0.30 x 0.70 + 0.20 x 0.80)/200], approximately 0.0430116. The interval is 0.10 plus or minus 1.96 times that value, approximately [0.01570, 0.18430]. It is an approximate interval for the difference, not a probability that either group's rate is correct.

Illustrative case study

Fictional case: Harborview Apps compares an AI onboarding feature with its existing interface. The first dataset contains several sessions per account, but assignment occurred at the account level.
Analysts rebuild the outcome as one seven-day completion indicator per assigned account and verify that the denominators include all eligible assigned accounts with observed outcomes. They report the absolute rate difference and implementation limitations, rather than treating session-level rows as independent evidence.

Common misconceptions and mistakes

Do not confuse percentage points with relative percentage change. Do not replace the null pooled standard error with an unrelated formula while calling the result the same test. A p-value is not the probability that the groups are equal. Nonrejection does not demonstrate equivalence. The simple test is not appropriate for every pair of percentages, especially when denominators or dependence structures differ fundamentally.

Business use

The test can support product experiments, quality monitoring, and operational comparisons with binary outcomes. Define the unit and outcome window clearly, preserve denominators, and separate statistical evidence from economic value. Observational comparisons need additional justification before a difference is attributed to a specific action.

Check your understanding

What is the observed absolute difference in the example?

Answer: Ten percentage points.

Why is the pooled estimate 0.25?

Answer: It combines 100 successes across 400 observations under the equality null.

Would repeated sessions from one account automatically be independent?

Answer: No.

Related terms

Standard error
A/B testing
Chi-square test
Confidence interval
Binomial proportion
pooled estimate
Fisher's exact test
McNemar's test

Learn next

Construct a two-by-two count table and derive the pooled Z statistic. Then compare large-sample, score-based, and exact approaches for smaller or more extreme counts.

Sources

NIST/SEMATECH, comparing two proportions of defectives: https://www.itl.nist.gov/div898/handbook/prc/section3/prc33.htm Penn State, two-way tables and pooled-proportion testing: https://online.stat.psu.edu/stat504/Lesson03 NIST Dataplot, difference of proportions hypothesis test: https://www.itl.nist.gov/div898/software/dataplot/refman2/auxillar/diffprci.htm

Previous: Two-phase locking Next: Two-sample hypothesis testing