Achieved power for the sign test

When to use it

Assess the sensitivity of a paired comparison that uses only the direction of each change, with no assumption about the shape of the differences.

Not for

Power is not monotone in n: one more pair can lower it, because the critical count moves in whole numbers. Check the sizes either side of yours.

Notation

$n$: Number of pairs with a non-zero difference.
$p$: P(D > 0), the probability a difference is positive.
$c$: Critical count from the Binomial(n, ½) null distribution.

Formulas

Critical count

c = F^{-1}_{\text{Bin}(n,\,1/2)}(\alpha/\text{tails})

The smallest count whose null cumulative probability reaches α/tails.

Exact binomial power

\text{Power}=P(B\le c-1)+P(B\ge n-c+1), \qquad B\sim\text{Bin}(n,p)

The rejection region is both tails of the count of positive differences [1].

Parameters

Number of pairs $n$

Values: Integer, at least 2

Pairs with a non-zero difference; ties are dropped.

Probability of superiority $p$

Values: Between 0 and 1, not 0.5

From pilot data, or for normal differences Φ(Δ/σ_D). 0.5 means no effect.

Significance level $\alpha$

Values: Between 0 and 1

The type I error rate for the specified test.

Test direction $s$

Values: 1 or 2

One-sided tests use the upper rejection tail. Two-sided tests split α equally between both tails.

Assumptions

Pairs are independent. Nothing is assumed about the shape of the differences.
One-sided tests use the upper rejection tail. Two-sided tests split α equally between both tails.

Worked examples

A worked input set

For 50 pairs and p = 0.70 at α = 0.05, two-sided: the critical count is c = 18, and power is the chance that 17 or fewer, or 33 or more, of the 50 differences are positive.

Sample size: 50

Open this example in the calculator

Validation

Power is the exact binomial rejection probability, the function the sign-test calculator solves, and agrees with an independent calculation in R.

How to cite

StatWisely. Achieved power for the sign test. Method document, engine version 1.0.0, reviewed 2026-10-03. https://statwisely.com/methods/power-sign/

Cite the original references for the formulas your design uses as well.

References

1.Noether GE. Sample size determination for some common nonparametric tests. Journal of the American Statistical Association. 1987;82(398):645–647.