Sample Size Procedures Guide
Comprehensive reference for all statistical procedures available in our sample size calculator
General Guidelines for Sample Size Calculation
Key Considerations
- Effect Size: Larger effects require smaller samples
- Variability: Higher variability requires larger samples
- Power (1-β): Typically 80% or 90%; higher power needs more subjects
- Significance Level (α): Usually 0.05; stricter α requires more subjects
- Dropout Rate: Always account for expected attrition
- Multiple Testing: Adjust α if conducting multiple tests
Common Pitfalls to Avoid
- Using unrealistic effect sizes (too optimistic)
- Ignoring dropout/attrition rates
- Not accounting for unequal group sizes
- Forgetting to adjust for multiple comparisons
- Using wrong test for study design
- Inadequate pilot data for parameter estimates
Effect Size Interpretation
Small Effect
Cohen's d = 0.2, correlation coefficient r = 0.1, Cohen's w = 0.1
Subtle difference, large sample needed
Medium Effect
Cohen's d = 0.5, correlation coefficient r = 0.3, Cohen's w = 0.3
Noticeable difference, moderate sample
Large Effect
Cohen's d = 0.8, correlation coefficient r = 0.5, Cohen's w = 0.5
Substantial difference, smaller sample
Effect Size Formulae
Both variables are numerical
Correlation coefficient (r)
r = Σ(x - x̄)(y - ȳ) / √[Σ(x - x̄)²Σ(y - ȳ)²]
Small (r=0.10)
Medium (r=0.30)
Large (r=0.50)
Outcome: numerical
Risk factor: Binary (T-test)
Standardized mean difference
Cohen's d
d = (x̄₁ - x̄₂) / s
s = √[(n₁-1)s₁² + (n₂-1)s₂²] / (n₁+n₂-2)
Small (d=0.20)
Medium (d=0.50)
Large (d=0.80)
Outcome: numerical
Risk factor: categorical > 2 groups (ANOVA)
Eta-squared
η² = SS_treatment / SS_total
Small (η²=0.01)
Medium (η²=0.06)
Large (η²=0.14)
Both variables are categorical
Phi
Odds ratio
Relative risk
φ = √(χ² / n)
RR = (risk among exposed) / (risk among unexposed)
= r₁ / r₂
OR = [r₁/(1-r₁)] / [r₂/(1-r₂)]
Small (r=0.10)
Medium (r=0.30)
Large (r=0.50)
OR < 1.44 - Very small
1.44 ≤ OR < 2.48 - Small
2.48 ≤ OR < 4.27 - Medium
OR ≥ 4.27 - Large
Statistical Analysis Framework
Understanding the fundamentals of statistical analysis and sample size determination
Sample Size Determinants
Questions to be answered to know how to calculate the sample size:
Type of statistical analysis:
Type of analytical statistics:
Type of the Outcome variable and the risk factor:
Areas of Statistics
There are two main areas of Statistics:
Descriptive statistics
Provides tabular and graphical techniques and numerical measures for describing data.
Analytical (Inferential) statistics
Provides procedures for analyzing data and making decisions. Using the sample to infer about the population (parameter).
Types of Inferential Statistics
Estimation
The answer for the research question is a value or a range of values (Confidence interval)
Example:
"What is the prevalence of Diabetes in a certain population?"
Hypothesis testing
The answer for the research question is either yes or no ("Accept" or "Reject")
Example:
"Is there a significant difference in the prevalence of Diabetes between males and females in a certain population?"
Types of Variables
Understanding variable types is crucial for selecting the appropriate statistical test
Variables can be classified into two main categories:
Qualitative (Categorical)
Labels or names used to identify attributes of the sample elements. The labels can be numbers with no real numerical meaning.
It has only two categories or levels
The order of the categories is not essential (no ranking)
The categories can be ordered
Quantitative (Numerical)
Numbers (with real meaning), representing measurements, obtained from the sample elements.
Is measured on a numeric or a quantitative scale and can take countable values (how many)
Is measured on a numeric or a quantitative scale and can take uncountable real values (how much)
Required Parameters for Sample Size Calculation
What values do we need to calculate the sample size?
› When the primary outcome is categorical
| Parameter | Value |
|---|---|
| Significance level | 5% |
| Prevalence difference | From the literature or a pilot study or 50% |
| Power of the test | Usually 80% |
| N (sample size) | ? |
› When the primary outcome is numerical
| Parameter | Value |
|---|---|
| Significance level | 5% |
| Mean difference and standard deviation | From the literature or a pilot study |
| Power of the test | Usually 80% |
| N (sample size) | ? |
› When the primary outcome is categorical
| Parameter | Value |
|---|---|
| Confidence level | 95% |
| Prevalence | From the literature or a pilot study or 50% |
| Margin of error (precision) (half of the interval width) | Usually 5% (±5%) |
| N (sample size) | ? |
› When the primary outcome is numerical
| Parameter | Value |
|---|---|
| Confidence level | 95% |
| Mean and standard deviation | From the literature or a pilot study |
| Margin of error (precision) (half of the interval width) | e.g. (±5kg or ±10cm) |
| N (sample size) | ? |
Sample Size Formulae and Examples
Detailed formulas and practical examples for each statistical procedure
Proportion Tests
Sample size calculations for categorical outcomes and proportions
One-Proportion Test
Compare a single proportion to a hypothesized value (e.g., testing if conversion rate differs from 50%).
When to use: Testing if a proportion equals a specific value
Parameters: p₀ (null proportion), p₁ (alternative proportion), α, power
n = (Z_α + Z_β)² × [p₀(1-p₀) + p₁(1-p₁)] / (p₁-p₀)²
Example: Testing if a new drug's response rate (60%) differs significantly from the standard treatment (40%)
Two-Proportion Test
Compare proportions between two independent groups (e.g., comparing success rates of two treatments).
When to use: Comparing proportions between two independent groups
Parameters: p₁ (group 1 proportion), p₂ (group 2 proportion), α, power
n = 2p̄(1-p̄)(Z_α + Z_β)² / (p₁-p₂)²
Example: Comparing click-through rates between two website designs
Chi-Square Test
Test independence or association between two categorical variables in a contingency table.
When to use: Testing association between two categorical variables
Parameters: Effect size (w), degrees of freedom, α, power
n = (Z_α + Z_β)² / w²
Example: Testing if gender is associated with product preference
McNemar's Test
Compare paired proportions in before-after or matched-pairs study designs.
When to use: Paired binary outcomes (before/after, matched pairs)
Parameters: p₁₀ (discordant pairs type 1), p₀₁ (discordant pairs type 2), α, power
n = (Z_α√(p₁₀+p₀₁) + Z_β√(p₁₀+p₀₁-4p₁₀p₀₁))² / (p₁₀-p₀₁)²
Example: Testing attitude change before and after an intervention
Decision Tree
Follow this flowchart to determine the appropriate statistical procedure for your research question
Statistical Procedure Decision Tree
Use this visual guide to navigate through different statistical procedures