Start Here
Not sure which test to use? The Categorical Data Analysis Wizard asks a few questions and gives you a direct recommendation.
Open the Categorical Data Analysis Wizard →Categorical Data Analysis Hub
Testing associations and comparing proportions between categorical variables.
Related Calculators
Chi-Square TestTests whether there is a statistically significant association between two categorical variables in independent groups.Fisher's Exact TestProvides exact p-values for 2×2 contingency tables when expected cell frequencies are small.McNemar TestTests for a significant change in a paired binary outcome (e.g. positive/negative before vs. after, or two paired diagnostic tests) in the same subjects.Cochran's Q TestExtends McNemar's test to 3 or more related binary measurements on the same subjects (e.g. treatment response across 3 visits).OR & RR CalculatorCalculates the Odds Ratio (case-control studies) or Relative Risk (cohort studies / RCTs) to quantify the association between an exposure and an outcome.NNT CalculatorComputes the Number Needed to Treat (or Harm) from the absolute risk reduction between two groups.Exact Binomial TestComputes the exact probability of observing k successes in n trials against a hypothesized proportion, valid for any sample size.
Related Articles
Chi-Square vs Fisher's Exact Test: When Should You Use Each One?The expected-cell-frequency rule with worked examples.McNemar Test: Complete Guide for Paired Categorical DataTesting change in paired binary outcomes.Relative Risk vs Odds Ratio: What Is the Difference?When to report RR vs. OR, and how to convert between them.How to Calculate Odds Ratio in Medical ResearchStep-by-step calculation with practical examples.
Medical Research Examples
Chi-Square Test: Smoking status (yes/no) vs. COPD diagnosis (yes/no) in 200 patients: χ²(1)=14.2, p<0.001, Cramér’s V=0.27 -- a moderate association.
Fisher's Exact Test: A rare adverse-event comparison between two drugs in a small pilot trial (n=18, expected cell count=3): Fisher's exact p=0.04, OR=6.2.
McNemar Test: Wheeze present/absent before and after 8 weeks of inhaled steroids in 40 children (15 improved, 3 worsened): χ²(1)=7.6, p=0.006.
Cochran's Q Test: Ten patients evaluated for wheeze at three visits: Cochran's Q(2)=8.0, p=0.018 -- wheezing rates differ significantly across visits.
Common Mistakes
- Using chi-square when an expected cell frequency is below 5 instead of switching to Fisher’s exact test.
- Applying chi-square to paired/repeated data instead of McNemar or Cochran’s Q.
- Reporting Odds Ratio and Relative Risk interchangeably -- they diverge substantially when the outcome is common.
- Reporting only a p-value for a 2×2 table without an effect size (Cramér’s V, OR, or RR) and its 95% CI.
- Interpreting NNT without specifying the time horizon and patient population of the underlying trial.
- Ignoring multiple comparisons when running several pairwise McNemar tests after a significant Cochran’s Q.
Reporting Recommendations
Journals expect more than a p-value. For each test, report the effect estimate and its 95% confidence interval alongside the p-value:
Chi-Square Testχ² statistic, df, p-value, Cramér’s V (effect size), observed and expected frequencies
Fisher's Exact TestFisher's exact p-value, Odds Ratio, 95% CI
McNemar Testχ²(1) statistic (or exact binomial p-value for small samples), p-value, discordant pair counts
Cochran's Q TestQ statistic, df, p-value
OR & RR CalculatorOR or RR, 95% CI, p-value, absolute risk reduction (ARR), number needed to treat (NNT)
NNT CalculatorNNT (or NNH), 95% CI, absolute risk reduction
Exact Binomial TestExact p-value, 95% Wilson confidence interval for the observed proportion
Recommended Learning Order
- Chi-Square Test — the standard test for association between independent categorical variables
- Fisher's Exact Test — the safe alternative when expected cell counts are small
- Odds Ratio & Relative Risk — quantify the strength of an exposure-outcome association
- McNemar Test — move to paired/repeated categorical data at 2 time points
- Cochran's Q Test — most advanced: paired data across 3+ time points or raters
Frequently Compared Tests
Chi-Square vs. Fisher’s Exact vs. McNemar vs. Cochran’s Q
Chi-square/Fisher compare independent groups; McNemar compares paired proportions at 2 time points; Cochran’s Q extends McNemar to 3+ related time points.
Frequently Asked Questions
When do I use chi-square vs. Fisher’s exact test?
Use chi-square when all expected cell frequencies in your table are at least 5. Use Fisher’s exact test when any expected frequency is below 5 -- it is valid for any sample size.
When do I use McNemar instead of chi-square?
Use McNemar when the same subjects are measured twice (paired data), e.g. positive/negative before vs. after an intervention. Chi-square assumes independent groups.
What does Cochran’s Q add over McNemar?
Cochran’s Q extends McNemar’s test to 3 or more related (repeated) binary measurements on the same subjects.
When do I report Odds Ratio vs. Relative Risk?
Use Odds Ratio for case-control studies (where incidence can’t be directly calculated). Use Relative Risk for cohort studies and RCTs where you have true incidence data in both groups.
What is Number Needed to Treat (NNT)?
NNT is the number of patients who need to receive the treatment for one additional patient to benefit, calculated as 1 divided by the absolute risk reduction.
My sample is very small -- what test should I use for a single proportion?
Use the Exact Binomial Test rather than a chi-square approximation, especially with small samples or proportions near 0 or 1.