Skip to main content
    Sample Size CalculatorBeta

    Method document

    Sample size to estimate an intraclass correlation

    How many participants are needed to estimate test–retest or inter-rater reliability, as an intraclass correlation (ICC) from k ratings of each participant, within a chosen margin. Covers non-response.

    Engine version 1.0.0 · Reviewed 9 October 2026

    Open in calculator

    See also: Sample size to show that an intraclass correlation exceeds a benchmark

    https://statwisely.com/methods/precision-icc/

    When to use it

    • A reliability study reporting the ICC of a score measured twice, or by several raters, with a confidence interval.

    Not for

    • Showing that the ICC exceeds a benchmark such as 0.75. Use the test of the ICC under “Detect an effect”, Measurement.

    Notation

    nn
    Number of participants
    kk
    Ratings of each participant (raters or occasions)
    ρ\rho
    Expected intraclass correlation
    ww
    Full width of the interval, 2E
    EE
    Margin of error
    z1−α/2z_{1-\alpha/2}
    Standard normal quantile for confidence 1 − α (1.960 for 95%)
    qq
    Non-response rate

    Formulas

    Participants for an interval of width w

    n=⌈8 z1−α/22 (1−ρ)2 [1+(k−1)ρ]2k (k−1) w2+1⌉n = \left\lceil \frac{8\, z_{1-\alpha/2}^{2}\,(1-\rho)^{2}\,[1+(k-1)\rho]^{2}}{k\,(k-1)\,w^{2}} + 1 \right\rceil

    The large-sample variance of the one-way ICC is 2(1 − ρ)²[1 + (k − 1)ρ]²/[k(k − 1)(n − 1)]; setting the interval’s width to w gives Bonett’s formula [1]. Agreement is best near ρ = 1 and with more ratings per participant.

    Non-response

    napproach=⌈n1−q⌉n_{\text{approach}} = \left\lceil \frac{n}{1 - q} \right\rceil

    The number to approach so that n respond. Applied last, after any clustering.

    Parameters

    Confidence level1−α1-\alpha

    Values: 90%, 95% or 99%

    95% is the convention in health research. A higher level widens the interval, so the same margin needs a larger sample.

    Margin of errorEE

    Values: Above 0 and below 1

    Half the interval’s width on the coefficient’s scale: 0.1 gives an interval from 0.6 to 0.8 around an ICC of 0.7.

    Expected ICCρ\rho

    Values: Between 0 and 1

    From a pilot or an earlier reliability study; the margin should be planned at a plausible value, not the hoped-for one.

    Ratings per participantkk

    Values: At least 2

    2 for a test–retest design; the number of raters when each rates every participant.

    Expected non-response rateqqOptional

    Values: 0% up to, but not including, 100%

    The share of people approached who will not take part; use the response rate of a comparable study. Approaching more people restores the number of respondents, not their representativeness: if non-responders differ, the estimate can still be biased.

    Assumptions

    • A one-way random-effects model: participants’ true scores and the rating errors are normal and independent, with the same error variance for every rating.
    • Non-response is unrelated to the outcome.

    Worked examples

    Test–retest reliability

    ICC 0.7 within ±0.1 from 2 ratings: n = 8 × 1.96² × 0.3² × 1.7² / (2 × 0.2²) + 1 = 100.9, rounded up to 101.

    Sample size: 101

    Open this example in the calculator

    Three raters

    With 3 ratings per participant: n = 68.

    Sample size: 68

    Open this example in the calculator

    A highly reliable instrument

    ICC 0.9 within ±0.05: n = 57; a reliable instrument is estimated precisely by few participants.

    Sample size: 57

    Open this example in the calculator

    Validation

    • Every worked example on this page is recomputed by the StatWisely engine whenever the software is built. If a number or formula here stopped matching the calculator, the release would be stopped until this document was corrected.
    • F-based intervals for the one-way ICC from simulated studies at the planned size had a mean half-width of 0.102 against a target of 0.10. An independent implementation in R recomputes these sizes and is compared with the engine’s.

    How to cite

    StatWisely. Sample size to estimate an intraclass correlation. Method document, engine version 1.0.0, reviewed 2026-10-09. https://statwisely.com/methods/precision-icc/

    Cite the original references for the formulas your design uses as well.

    References

    1. 1.Bonett DG. Sample size requirements for estimating intraclass correlations with desired precision. Statistics in Medicine. 2002;21(9):1331–1335. doi:10.1002/sim.1108