In 2004, a landmark statin trial enrolled 48,000 patients and reported that the treatment reduced ten-year cardiovascular event rates from 4.2% to 3.9% — with p < 0.0001. The result was heralded as a breakthrough and the drug was prescribed to millions of low-risk patients. The absolute risk reduction was 0.3%, and the number needed to treat was 333 patients for ten years to prevent one cardiovascular event. In 2019, the American Statistical Association published a statement in Nature signed by 800 statisticians calling for the abandonment of “statistical significance” as the sole criterion for scientific conclusions. The problem is not with p-values — they are valid and useful when correctly interpreted. The problem is with p-value worship: the conflation of “statistically significant” with “clinically important,” a mistake that pervades the medical literature and distorts clinical decision-making. Effect size is the antidote — the metric that answers the question p-values cannot: how large is this effect, and does it matter?
The P-Value’s Fundamental Limitation
A p-value is the probability of observing a result at least as extreme as the one obtained, assuming the null hypothesis is true. It is a function of two things simultaneously: the true effect size in the population and the sample size. This dual dependence is the source of p-value misinterpretation:
- Large sample + trivial effect = small p-value. With n = 100,000 patients, a blood pressure difference of 0.5 mmHg (clinically irrelevant — less than the measurement error of a standard sphygmomanometer) will produce p < 0.001.
- Small sample + large effect = large p-value. With n = 15 per group, a clinically substantial improvement of 15 functional-independence points in a rehabilitation programme may produce p = 0.09 — “not significant” — due solely to insufficient power.
P-values tell you nothing about magnitude. They are a binary signal — noise vs not-noise — calibrated by sample size as much as by truth. Effect size separates these two components and reports the magnitude of the finding independent of sample size, in a standardised, interpretable unit.
The fundamental insight: Every hypothesis test result can be decomposed as: p-value = f(effect size, sample size). A small p-value means either that the effect is large, or that the sample is large, or both. Without an effect size, you cannot tell which. A large p-value means either that the effect is small, or that the sample is small, or both. Without an effect size and a power calculation, you cannot tell which.
Cohen’s d: Effect Size for Two-Group Mean Comparisons
Cohen’s d is the standard effect size for the independent samples t-test and for any comparison of two continuous group means. It expresses the difference between the group means in units of the pooled standard deviation — converting the raw difference into a scale-independent, comparable metric.
The d value represents how many standard deviations apart the two group means are. d = 1.0 means the groups differ by one full standard deviation — the treatment group’s mean equals approximately the 84th percentile of the control group’s distribution. d = 0.2 means the means overlap substantially: only about 58% of the control distribution is below the treatment mean.
Hedges’ g is a bias-corrected version of Cohen’s d that should be used for small samples (n < 20 per group), because d systematically overestimates the true population effect size in small samples. For large samples, d and g are nearly identical.
Eta Squared and Partial Eta Squared: ANOVA Effect Sizes
When comparing three or more groups (one-way ANOVA) or analysing factorial designs (multi-factor ANOVA), Cohen’s d is not applicable. The ANOVA family uses variance-partitioning effect size measures that express the proportion of total outcome variance explained by the treatment factor.
Eta Squared (η²)
Eta squared is the most straightforward ANOVA effect size: the ratio of the variance explained by the factor to the total variance in the data.
Partial Eta Squared (ηḋ²)
Partial eta squared is the standard effect size reported by SPSS for factorial ANOVA. It differs from η² in that only the error term (residual SS), not the variance from other factors, appears in the denominator:
Omega Squared (ω²): The Preferred Unbiased Estimate
Both η² and ηḋ² are positively biased — they overestimate the true population effect, particularly in small samples. Omega squared applies a correction that accounts for the number of groups and residual error, producing an unbiased estimate of the proportion of population variance explained.
Cramér’s V: Effect Size for Categorical Data
When a chi-square test or Fisher’s exact test is used to assess the association between two categorical variables, the chi-square statistic (χ²) itself is not an effect size — it grows with sample size and cannot be interpreted independently. Cramér’s V converts χ² into a standardised effect size between 0 and 1.
Effect Sizes for Binary Clinical Outcomes
For clinical trials and epidemiological studies comparing event rates (mortality, complications, disease incidence) between two groups, the most clinically useful effect size measures are those based on the absolute and relative risk difference — not odds ratios or chi-square statistics alone.
Absolute Risk Reduction (ARR) and Number Needed to Treat (NNT)
ARR is the most direct measure of clinical benefit: the simple arithmetic difference in event rates between control and treatment groups. NNT — derived directly from ARR — is the most intuitively clinically meaningful metric: how many patients must be treated for one to benefit.
The RRR trap: Relative risk reduction (RRR) is highly misleading without knowing the baseline risk. A 50% RRR from 40% to 20% event rate (ARR = 20%, NNT = 5) and a 50% RRR from 2% to 1% event rate (ARR = 1%, NNT = 100) represent dramatically different levels of clinical benefit — yet both are reported as “50% risk reduction.” Drug advertising overwhelmingly uses RRR because it produces larger-looking numbers. Scientific reporting must always include ARR and NNT alongside RRR to contextualise the absolute benefit.
Which Effect Size to Report: A Decision Guide
| Statistical Test | Effect Size to Report | Interpretation |
|---|
| Independent t-test | Cohen’s d (or Hedges’ g) | Standardised mean difference (SD units) |
| Paired t-test | Cohen’s dᵏ or dᵗᵘ | Mean change / SD of differences |
| One-way ANOVA | η² or ω² | Proportion of total variance explained |
| Factorial ANOVA | ηḋ² or partial ω² | Proportion of variance (excl. other factors) |
| Chi-square / Fisher’s | Cramér’s V | Association strength (0 = none, 1 = perfect) |
| Binary event rates (RCT) | ARR, RRR, NNT, OR | Absolute and relative clinical benefit |
| Pearson correlation | r and r² | r is itself an effect size; r² = variance explained |
| Mann-Whitney U | Rank-biserial r | Stochastic superiority (−1 to +1) |
| Logistic regression | OR (with 95% CI) | Ratio of odds; convert to RR when baseline risk known |
| Cox regression | Hazard Ratio (HR) | Ratio of instantaneous event rates over time |
Statistical Significance vs Clinical Significance
This is the most important distinction in applied medical statistics, and it is violated more frequently than any other principle in published research. Statistical significance is a property of the sample and the test; clinical significance is a property of the effect and the patient.
📊 Statistical Significance
A result is statistically significant when the p-value falls below a pre-specified threshold (usually α = 0.05). It indicates that the observed result is unlikely under the null hypothesis. It is influenced by: sample size, true effect size, measurement precision, and the significance threshold chosen.
“This result is unlikely to be due to chance”
💉 Clinical Significance
A result is clinically significant when the magnitude of the effect is large enough to matter to patients, clinicians, or health systems — to change practice, improve quality of life, or justify costs and risks. Assessed by: effect size versus the Minimum Clinically Important Difference (MCID), NNT vs treatment burden, and cost-effectiveness analysis.
“This difference is large enough to matter to patients”
The Minimum Clinically Important Difference (MCID)
The MCID is the smallest change in an outcome measure that a patient perceives as meaningful and that would prompt a change in clinical management. It is established through patient-reported data, clinical expert consensus, or anchor-based methods (correlating score changes with patient global impression of change). Every primary outcome in a clinical trial should have a pre-specified MCID, against which the observed effect size is compared — not just against zero.
Examples of established MCIDs: WOMAC pain score: 15 mm; PHQ-9 depression: 5 points; SF-36 physical function: 5–10 points; FEV₁ in COPD: 100–150 mL; SBP: 5–10 mmHg (context-dependent); HbA1c: 0.5–1.0% (5.5–11 mmol/mol). When a statistically significant result falls below the MCID, it should be reported as statistically but not clinically significant, with both the observed effect size and the MCID stated explicitly.
The four possible combinations: (1) Statistically significant AND clinically meaningful — the ideal result; act on it. (2) Statistically significant but NOT clinically meaningful — large sample, trivial effect; report the effect size and MCID; do not change practice. (3) Not statistically significant but clinically plausible — underpowered study; report with effect size and confidence interval width; call for adequately powered replication. (4) Neither statistically nor clinically significant — true null result; report as evidence of absence, with effect size confirming small magnitude.
Clinical Examples
1
Large Statin RCT: Statistically Overwhelming, Clinically Marginal
A multi-centre RCT of a new statin in 48,000 adults with mild dyslipidaemia (LDL 3.0–3.5 mmol/L, no prior cardiovascular events) reports a reduction in 10-year major cardiovascular event (MACE) rate. Primary outcome: time-to-first-MACE, analysed as 10-year cumulative event rate.
4.2%
Control 10-yr event rate
3.9%
Treatment 10-yr event rate
p < 0.0001
Highly statistically significant
Computing Binary Outcome Effect Sizes
CER (control event rate) = 0.042
EER (experimental event rate) = 0.039
ARR = CER − EER = 0.042 − 0.039 = 0.003 (0.3%)
RRR = ARR / CER = 0.003 / 0.042 × 100% = 7.1%
NNT = 1 / ARR = 1 / 0.003 = 333
OR = (0.039/0.961) / (0.042/0.958) = 0.0406 / 0.0438 = 0.926
Why p < 0.0001 Despite a Tiny ARR
With n = 48,000, the SE of the ARR = √(p̂₁q̂₁/n₁ + p̂₂q̂₂/n₂)
SE ≈ √(0.042×0.958/24000 + 0.039×0.961/24000) = 0.000825
z = 0.003 / 0.000825 = 3.64 → p = 0.0003
[The sample size amplified a trivial difference to overwhelming significance]
Contextualising Against Clinical Thresholds
Established MCID for MACE risk reduction (guideline level): ARR ≥ 2.0% over 10 years
Observed ARR: 0.3% — below the MCID by a factor of 6.7
NNT = 333: treat 333 low-risk patients for 10 years to prevent one MACE
Statin side-effect risk (myopathy, T2DM): ~1 in 200 over 10 years
ARR = 0.3%, RRR = 7.1%, NNT = 333 (95% CI 190–1,112), OR = 0.93 (95% CI 0.88–0.97). The result is statistically significant (p < 0.0001) but does not cross the pre-specified MCID of 2% ARR for low-risk primary prevention. The treatment is statistically but not clinically significant for this population.
What the p-value concealed: Reporting only “p < 0.0001, statistically significant” creates a false impression of a meaningful clinical benefit. The NNT of 333 is the honest number: a patient would need to take this medication every day for 10 years — with associated costs, side effects, and daily pill burden — for a 1-in-333 chance of avoiding a cardiovascular event they might not have had anyway. For high-risk patients (baseline event rate 20%), the same relative reduction (7.1%) yields ARR = 1.4% and NNT = 71 — still modest, but substantially more defensible. Effect size contextualises not just statistical significance but clinical applicability across risk strata.
2
Small Stroke Rehabilitation Trial: Large Effect, Modest Significance
A physiotherapy RCT tests an intensive early mobilisation protocol versus standard care in 48 patients recovering from ischaemic stroke (24 per group). Primary outcome: Functional Independence Measure (FIM) score at 12 weeks (range 18–126; higher = more independent). The MCID for FIM in stroke rehabilitation is 22 points.
68.4
Intervention mean FIM (SD 12.1)
53.7
Control mean FIM (SD 13.8)
Computing Cohen's d
S_p = √[((23×12.1²) + (23×13.8²)) / 46]
= √[(23×146.41 + 23×190.44) / 46]
= √[(3367.4 + 4380.1) / 46]
= √[7747.5 / 46]
= √168.4 = 12.98
d = (68.4 − 53.7) / 12.98 = 14.7 / 12.98 = 1.132
Hedges' g Correction (recommended for n < 30/group)
df = 46
g = d × (1 − 3/(4×46−1)) = 1.132 × (1 − 3/183) = 1.132 × 0.9836 = 1.114
95% CI for d
SE_d = √(1/24 + 1/24 + d²/(2×(24+24))) = √(0.0833 + 0.0142) = 0.312
95% CI: 1.132 ± 1.96 × 0.312 = [0.521, 1.743]
Contextualising Against MCID
Observed mean difference: 14.7 FIM points
MCID for stroke rehabilitation FIM: 22 points
Conclusion: d = 1.13 (large) but falls short of MCID (14.7 < 22)
— Large standardised effect; absolute magnitude below MCID threshold
— Study likely underpowered (n=24/group) to confirm MCID-level differences
t(46) = 2.89, p = 0.003, Cohen’s d = 1.13 (95% CI 0.52–1.74), Hedges’ g = 1.11. The large standardised effect (d = 1.13) indicates the intervention is clinically promising, though the mean FIM improvement of 14.7 points falls below the 22-point MCID. A larger powered trial is warranted.
The interplay between effect size and MCID: Cohen’s d = 1.13 is impressive by conventional thresholds — “large.” But the absolute improvement of 14.7 FIM points does not reach the 22-point MCID. These two findings are not contradictory: d is standardised (relative to SD within groups) and the MCID is absolute (in raw FIM units). Both dimensions must be reported. The wide 95% CI (0.52 to 1.74) also reveals that n = 24 per group is too small to precisely estimate the effect. This trial correctly motivates a larger definitive study — it does not provide sufficient evidence to change practice, despite statistical significance. The correct thesis conclusion: “The intervention demonstrated a large standardised effect size (d = 1.13) that was statistically significant; however, the absolute improvement (14.7 FIM points) fell below the pre-established MCID of 22 points, and the wide confidence interval indicates insufficient precision. A fully powered RCT targeting a minimum of 80 participants per group is recommended.”
3
Surgical Site Infection Audit: Chi-Square with Cramér’s V
A hospital quality audit compares surgical site infection (SSI) rates between a new enhanced care bundle protocol (n = 172) and the old standard protocol (n = 168) in patients undergoing elective colorectal surgery. A 2×2 contingency table is analysed with chi-square. The hospital’s quality improvement target is a 50% reduction in SSI rate.
7.6%
New protocol SSI rate (13/172)
13.1%
Old protocol SSI rate (22/168)
p = 0.038
Chi-square result
Chi-Square Test
Observed: New SSI+ = 13, New SSI− = 159, Old SSI+ = 22, Old SSI− = 146
Expected: New SSI+ = 35×172/340 = 17.7, Old SSI+ = 35×168/340 = 17.3
χ²(1) = (13−17.7)²/17.7 + (22−17.3)²/17.3 + ... = 4.35, p = 0.038
Computing Cramér's V
V = √(χ²/(n × (min(r,c) − 1)))
V = √(4.35 / (340 × (min(2,2) − 1)))
V = √(4.35 / 340) = √0.01279 = 0.113
Interpretation: V = 0.113 → small effect (threshold for small = 0.10)
Binary Effect Sizes
ARR = 13.1% − 7.6% = 5.5%
RRR = 5.5% / 13.1% = 42%
NNT = 1/0.055 = 18.2 → treat 18 patients with new protocol to prevent 1 SSI
Against Quality Improvement Target
Target: 50% SSI reduction from 13.1% → 6.55% (ARR ≥ 6.55%)
Observed ARR: 5.5% → below the 50% relative reduction target (achieved 42%)
— Statistically significant but does not fully meet the quality improvement benchmark
χ²(1) = 4.35, p = 0.038, Cramér’s V = 0.11. ARR = 5.5%, RRR = 42%, NNT = 18 (95% CI 10–125). The new protocol significantly reduced SSI rates; however, V = 0.11 indicates a small association, and the ARR of 5.5% fell short of the pre-specified 50% relative reduction target (ARR ≥ 6.6%).
The chi-square trap exposed: Reporting only χ² = 4.35, p = 0.038 tells the quality improvement team almost nothing. The chi-square statistic is dimensionless and increases with sample size. Cramér’s V = 0.11 (small effect) and NNT = 18 together provide the actionable information: for every 18 patients managed with the new protocol, one SSI is prevented — a modest but potentially worthwhile benefit given that SSIs carry substantial morbidity and cost. Whether NNT = 18 justifies the cost and complexity of implementing the new care bundle is a clinical and health-economic judgement, not a statistical one. This is exactly where statistical significance hands off to clinical significance.
Thesis Writing Recommendations
Effect size reporting is no longer optional in academic medical research. Examiners at postgraduate level will specifically ask you to justify why statistical significance alone does not support your conclusions and will expect you to interpret your findings against established effect size thresholds and MCIDs.
In the Statistical Analysis Section
Pre-specify the effect size measure for each primary analysis before data collection. State which software or formula was used. For continuous outcomes: specify Cohen’s d (for two groups) or ω² (for ANOVA). For binary outcomes: specify ARR, RRR, and NNT. For categorical associations: specify Cramér’s V. State the established MCID for your primary outcome variable and cite its source.
Model Statistical Analysis Paragraph — Effect Size Pre-specification
“The primary effect size measure for the between-group functional independence comparison was Cohen’s d, with Hedges’ g correction applied given group sizes below 30. The minimum clinically important difference (MCID) for the Functional Independence Measure in stroke rehabilitation is 22 points (Stineman et al., 1996). The observed mean difference and its 95% confidence interval were compared against this threshold. For secondary binary outcomes, absolute risk reduction (ARR), relative risk reduction (RRR), number needed to treat (NNT), and odds ratio (OR) with 95% CIs were calculated. Effect sizes were interpreted using Cohen’s (1988) conventional thresholds: d = 0.20 small, 0.50 medium, 0.80 large.”
Model Results Sentence — Effect Size + p-Value + MCID
“Patients receiving the early mobilisation intervention demonstrated significantly higher FIM scores at 12 weeks (68.4 ± 12.1) compared to standard care (53.7 ± 13.8), t(46) = 2.89, p = 0.003, mean difference 14.7 points (95% CI: 4.9–24.5), Cohen’s d = 1.13 (95% CI: 0.52–1.74). The standardised effect was large by Cohen’s (1988) conventions; however, the observed mean difference (14.7 points) did not reach the pre-established MCID of 22 points, and the wide confidence interval reflects the limited sample size.”
Common Mistakes Researchers Make
Mistake 1: Reporting Only the P-Value and Ignoring Effect Size
The most prevalent error in clinical research. Papers that report only p < 0.05 and “significant difference” provide no information about the magnitude of the finding. Readers cannot assess clinical meaningfulness, compare findings across studies, calculate sample sizes for replications, or include the result in meta-analyses without effect size data. This practice was common before APA 7th edition (2019) made effect size reporting mandatory.
Fix: For every primary and secondary inferential test, report the effect size measure appropriate to the test (d, η², V, ARR/NNT, OR) alongside the test statistic and p-value in the same sentence. Most major software computes effect sizes automatically alongside test results — in SPSS, tick the “effect size” option in the output preferences; in R, use the effectsize package.
Mistake 2: Confusing Statistical Significance with Clinical Significance
Phrases like “the treatment significantly improved outcomes” (when the effect is below the MCID) or “no significant effect was found” (when the study was underpowered with a clinically promising effect) conflate two completely different dimensions of a finding. This error directly affects clinical decision-making: over-prescribing of minimally effective treatments backed by large-n significant trials; under-use of genuinely effective interventions backed by underpowered small trials.
Fix: Always explicitly address both dimensions: “This result was statistically significant (p = 0.038) but the absolute risk reduction (5.5%) fell below the pre-specified MCID (6.6%), indicating the finding is statistically but not clinically significant for the target population.” Conversely: “This result was not statistically significant (p = 0.09) in this underpowered sample (n = 24); however, the large effect size (d = 0.82) suggests a clinically meaningful difference that warrants a larger definitive trial.”
Mistake 3: Reporting Relative Risk Reduction Without Absolute Risk Reduction or NNT
“This treatment reduced the risk of hospitalisation by 40%” sounds impressive and often is used in drug advertising. It conceals the baseline risk and therefore the absolute benefit. A 40% RRR from a baseline hospitalisation rate of 1% (ARR = 0.4%, NNT = 250) and a 40% RRR from a baseline rate of 30% (ARR = 12%, NNT = 8) represent vastly different clinical contexts. Reporting RRR alone is incomplete and misleading.
Fix: Always report binary outcome results in this order: (1) event rates in both groups; (2) ARR with 95% CI; (3) RRR; (4) NNT with 95% CI; (5) OR or RR with 95% CI. The CONSORT checklist for RCT reporting explicitly requires ARR, and NNT should be calculated and presented for all primary binary outcomes. If a review or guideline reports only RRR, calculate the ARR and NNT yourself using the reported baseline event rate.
Mistake 4: Reporting Eta Squared Instead of Omega Squared for Small Samples
SPSS outputs partial eta squared (ηḋ²) by default for ANOVA, and many researchers report this value without awareness that it overestimates the true population effect, particularly when total sample size is small (n < 80). A study with three groups of 10 patients (n = 30) reporting η² = 0.18 (large effect) may have a true ω² of 0.10 (medium effect) — a substantial inflation that affects how the finding is interpreted and how replication is planned.
Fix: For ANOVA with total n < 100, compute and report omega squared (ω²) rather than eta squared. In R, the effectsize package provides omega_squared() directly from ANOVA output. In SPSS, ω² must be calculated manually using the formula: ω² = (SS_between − (k−1)×MS_error) / (SS_total + MS_error). State in the methods whether η² or ω² is reported and cite the justification.
Mistake 5: Applying Cohen’s Thresholds Uncritically Without Clinical Context
Cohen (1988) explicitly stated that his small/medium/large thresholds were intended as benchmarks “in the absence of better information” and should be replaced by domain-specific knowledge when available. A d = 0.20 (“small”) for a low-cost, side-effect-free dietary intervention preventing dementia may be enormously clinically worthwhile; a d = 0.20 for an expensive biologic with serious adverse effects is not. Applying thresholds mechanically without clinical reasoning produces misleading conclusions.
Fix: Always interpret effect sizes in the context of the clinical domain, the MCID for the outcome, the treatment burden and cost, the baseline risk, and the available alternatives. Cohen’s thresholds are a minimum reporting standard, not a substitute for clinical judgement. Compare your observed effect size to previously published effects for the same outcome in the same population, not only to abstract conventional benchmarks.
Mistake 6: Not Reporting Confidence Intervals for Effect Sizes
A point estimate of Cohen’s d, η², or V conveys magnitude but not precision. A d = 0.80 from n = 12 per group has a 95% CI approximately [0.0, 1.6] — spanning from negligible to massive — and conveys almost no information about the true effect. Reporting effect sizes without confidence intervals conceals imprecision and gives a false impression of certainty about the magnitude of the finding.
Fix: Report 95% CIs for all effect size estimates. For Cohen’s d, the CI can be calculated from the non-central t-distribution (R: effectsize::cohens_d(ci=0.95); SPSS: use a dedicated macro or the ESCI tool). For ω², bootstrapped CIs are standard. For NNT, CIs are derived from the CI for ARR: NNT lower bound = 1/ARR_upper; NNT upper bound = 1/ARR_lower. Wide CIs should explicitly be noted as indicating insufficient sample size for precise effect size estimation.
Scientific Reporting Standards
- APA 7th Edition (2019): Section 6.24 states: “For each statistical test reported, provide the effect size or a confidence interval for the effect size.” This is a requirement, not a recommendation, for all APA-compliant reporting. Specific guidance: Cohen’s d or Hedges’ g for t-tests; ηḋ² or ω² for ANOVA; OR with CI for logistic regression; r² for regression.
- CONSORT 2010 (Item 17a): Requires “for each primary and secondary outcome, a summary of results for each group, and the estimated effect size and its precision (such as 95% confidence interval).” Also mandates (Item 17b) absolute and relative risk differences for binary outcomes. CONSORT explicitly states that p-values alone are insufficient for primary outcome reporting.
- PRISMA 2020 (Item 21): For systematic reviews and meta-analyses, requires synthesis of pooled effect sizes with 95% CIs and heterogeneity measures (I², τ²) for each outcome. Individual study effect sizes must be reported to enable pooling.
- STROBE Statement (Item 14c): Requires “if relevant, consider translating estimates of relative risk into absolute risk for a meaningful time period” — operationalised as reporting ARR and NNT alongside OR or RR for all primary binary outcomes.
- EQUATOR Reporting Guidelines: Across CONSORT, STROBE, MOOSE, TREND, and STARD, effect size with confidence interval is a near-universal requirement for primary outcome reporting. Using these checklists when preparing manuscripts or theses ensures effect size reporting is not inadvertently omitted.
Practical Guidance
Pre-Specify Your MCID Before Data Collection
Identify the MCID for your primary outcome from the literature or clinical consensus before you begin data collection. Register it in your study protocol. Without a pre-specified MCID, post-hoc claims of clinical significance (or insignificance) are subjective and challengeable. Many outcome instruments (SF-36, WOMAC, PHQ-9, FIM, WOMAC, EQ-5D) have published MCIDs that vary by population and clinical context — select the one most applicable to your sample.
Use the effectsize Package in R
The R effectsize package (Ben-Shachar et al., 2020) computes Cohen's d, Hedges' g, eta squared, partial eta squared, omega squared, Cramér's V, and rank-biserial r directly from model objects, with 95% CIs via the non-central distribution or bootstrapping. One line per effect: cohens_d(x ~ group, data = df, ci = 0.95). Far more flexible and complete than SPSS's limited built-in effect size options.
Report the NNT Whenever You Report a Binary Outcome
NNT is the most clinically interpretable effect size for binary data. No clinician thinks in terms of odds ratios or even absolute risk reductions during a consultation — but "treat 18 patients to prevent one surgical site infection" is immediately actionable. Calculate NNT even when the paper you are citing does not report it: NNT = 1 / (CER − EER), where CER and EER can usually be extracted from the event counts reported in the results table.
Always Include Effect Size in Your Sample Size Calculation
Effect size is the central input to power calculations — you cannot calculate required sample size without specifying the minimum clinically important effect you want to detect. If your trial is designed around an effect size of d = 0.80 (large) but the true effect is d = 0.40 (medium), your study will be underpowered by approximately 400%. Base your sample size calculation on the MCID converted to a standardised effect size, not on published estimates from convenience samples or pilot data with wide CIs.
Interpret Your 95% CI as an Effect Size Range
The 95% CI for an effect size is as important as the point estimate. A d = 0.80 (95% CI: 0.10, 1.50) is compatible with everything from a negligible to a massive effect. A d = 0.50 (95% CI: 0.38, 0.62) is a precisely estimated medium effect. Train yourself to read the full CI, not just the point estimate — the CI width directly reflects sample size adequacy for effect estimation, independent of statistical significance.
Distinguish ηp² Reported by SPSS from η²
SPSS labels its ANOVA effect size as "partial eta squared" (ηp²), not eta squared (η²). In a one-way ANOVA, they are equal. In factorial designs, ηp² is consistently larger than η² and the three ηp² values can sum to more than 1. If you use SPSS for factorial ANOVA, report "partial eta squared (ηp²)" explicitly — not just "eta squared." Reviewers familiar with the distinction will notice immediately if you label ηp² as η².
Frequently Asked Questions
What is effect size and why is it important in medical research? +
Effect size is a standardised, scale-independent measure of the magnitude of a statistical finding — answering the question p-values cannot: “how large is this effect?” A p-value is influenced by both the true effect AND the sample size; effect size is influenced only by the true effect. With n = 50,000, a blood pressure difference of 0.5 mmHg produces p < 0.001 — statistically overwhelming but clinically meaningless. Effect size reporting is now required by APA guidelines, CONSORT, PRISMA, and most major clinical journals. Without it, statistical results cannot be properly interpreted, replicated, or included in meta-analyses.
What is Cohen’s d and how do I interpret it? +
Cohen’s d expresses the difference between two group means in units of the pooled standard deviation: d = (mean₁ − mean₂) / pooled SD. Conventional thresholds (Cohen 1988): d = 0.20 small, 0.50 medium, 0.80 large. Interpret in clinical context: a “small” d = 0.20 for an inexpensive lifestyle intervention preventing dementia may be highly worthwhile; a “small” d = 0.20 for an expensive biologic with serious side effects may not justify use. Use Hedges’ g (slightly smaller than d) for small samples (n < 20 per group) to correct for positive bias. Always report the 95% CI for d.
What is eta squared and when should I use it? +
Eta squared (η²) is the effect size for ANOVA designs: the proportion of total outcome variance explained by the factor (η² = SS_between / SS_total). Thresholds: 0.01 small, 0.06 medium, 0.14 large. Use η² for one-way ANOVA; use partial η² (ηḋ²) for factorial designs with multiple factors. For small samples (n < 100), use omega squared (ω²) instead, as η² overestimates the true population effect. SPSS reports ηḋ² by default; label it correctly as partial eta squared in your output.
What is the difference between eta squared and partial eta squared? +
η² = SS_effect / SS_total (denominator includes all sources of variance). ηḋ² = SS_effect / (SS_effect + SS_error) (denominator excludes variance from other factors). In one-way ANOVA, they are identical. In factorial ANOVA, ηḋ² is systematically larger because it removes the variance explained by other factors from the denominator. The three ηḋ² values in a two-factor ANOVA can legitimately sum to more than 1.0. Use η² for one-way; ηḋ² for factorial. Always label which you are reporting.
What is NNT and why does it matter clinically? +
Number Needed to Treat (NNT) = 1 / ARR, where ARR (Absolute Risk Reduction) = Control Event Rate − Experimental Event Rate. NNT tells you how many patients must be treated for one additional patient to benefit. It is the most intuitively clinically interpretable effect size for binary outcomes. A 30% relative risk reduction sounds impressive regardless of baseline risk; NNT reveals that it means treating 3 patients (if baseline risk 50%) or 333 patients (if baseline risk 0.3%) for one to benefit. Always report NNT with a 95% CI and alongside ARR and RRR for complete clinical context.
What is the difference between statistical and clinical significance? +
Statistical significance (p < 0.05) indicates the result is unlikely under the null hypothesis — influenced by sample size. Clinical significance indicates the effect is large enough to matter to patients — assessed against the MCID (Minimum Clinically Important Difference). The four combinations: statistically AND clinically significant (act on it); statistically significant but NOT clinically significant (real but trivial effect — do not change practice); clinically plausible but NOT statistically significant (underpowered study — replicate with adequate n); neither significant (true null or trivially small effect). Report both dimensions explicitly for every primary outcome.
What is Cramér’s V and when should I use it? +
Cramér’s V is the effect size for chi-square and Fisher’s exact tests: V = √(χ² / (n × (min(r,c) − 1))). It ranges from 0 (no association) to 1 (perfect association). Thresholds for 2×2 tables: V = 0.10 small, 0.30 medium, 0.50 large. The chi-square statistic alone is not interpretable as an effect size — it grows with n. V must be reported alongside every chi-square result. For larger contingency tables, the thresholds scale with degrees of freedom; use Bergsma’s (2013) corrected V for small samples.
What is omega squared and how does it differ from eta squared? +
Omega squared (ω²) is a bias-corrected ANOVA effect size: ω² = (SS_between − (k−1)×MS_error) / (SS_total + MS_error). It is systematically smaller than η² and more accurately estimates the true population variance explained. For small samples (n < 100 total), ω² is substantially more accurate. APA 7th edition recommends ω² over η². Both use the same thresholds (0.01 small, 0.06 medium, 0.14 large). In R: effectsize::omega_squared(model); in SPSS: calculate manually from ANOVA table output.
How do I choose which effect size measure to report? +
Match the effect size to the test: t-test → Cohen’s d; one-way ANOVA → η² or ω²; factorial ANOVA → ηḋ² or partial ω²; chi-square/Fisher’s → Cramér’s V; binary RCT outcomes → ARR, RRR, NNT, OR; Pearson correlation → r and r² (r is itself an effect size); Mann-Whitney U → rank-biserial r; logistic regression → OR with 95% CI; Cox regression → Hazard Ratio. Never report Cohen’s d for chi-square results or Cramér’s V for ANOVA.
How should I report effect sizes in a thesis or paper? +
APA 7th Edition requires: effect size + 95% CI for every primary inferential test result. Format: “t(91) = −3.74, p < 0.001, Cohen’s d = 0.76 (95% CI: 0.37, 1.15).” For binary outcomes: “ARR = 5.5% (95% CI: 0.4%, 10.6%), RRR = 42%, NNT = 18 (95% CI: 9, 230), OR = 0.54 (95% CI: 0.25, 0.97).” Pre-specify in Methods which effect size you will report. Compare results against the MCID. CONSORT, PRISMA, and STROBE all mandate effect size reporting. P-values alone are insufficient for any primary outcome in contemporary clinical research reporting.
Calculate Effect Sizes Online
Compute Cohen’s d, Hedges’ g, eta squared, omega squared, Cramér’s V, NNT, ARR, and RRR with 95% confidence intervals for your research data.
Open StatClinic →