StatClinicClinical Statistics SuiteMeasure inter-rater agreement corrected for chance. Enter a 2×2 contingency table from two independent raters to compute kappa (κ), 95% confidence interval, and interpretation.
Enter the agreement table. Rows = Rater 1; Columns = Rater 2. Cell a = both positive; d = both negative; b and c = disagreements.
Cohen's Kappa (κ) measures agreement between two raters on a categorical classification task, adjusted for the level of agreement expected by chance. A κ of 1.0 indicates perfect agreement; 0.0 indicates agreement no better than chance; negative values indicate agreement worse than chance.
Formula: κ = (Po − Pe) / (1 − Pe) where Po = observed agreement and Pe = expected agreement by chance.
Standard Error: SE = √[Po(1 − Po) / (n(1 − Pe)²)] — used for the 95% CI: κ ± 1.96 × SE
Reporting (journal format): κ = 0.74 (95% CI: 0.62–0.86, N = 120), indicating substantial inter-rater agreement (Landis & Koch, 1977).
Cohen's Kappa (κ) measures the degree of agreement between two raters on a categorical variable, corrected for chance agreement. It is used in reliability studies for diagnostic criteria, image reading (radiology, pathology), scoring scales, and content coding in medical research.
Using Landis & Koch benchmarks: <0 = Poor; 0.01–0.20 = Slight; 0.21–0.40 = Fair; 0.41–0.60 = Moderate; 0.61–0.80 = Substantial; 0.81–1.00 = Almost Perfect. Most clinical research journals require κ ≥ 0.70 for acceptable reliability.
Percent agreement ignores chance matching. If both raters independently call 90% of subjects "positive," they will agree 81% of the time purely by chance. Kappa corrects for this. Reporting percent agreement alone without Kappa overestimates true reliability and is discouraged in peer-reviewed journals.
Report: κ = value (95% CI: lower–upper, N = total), with the verbal interpretation. Example: "Inter-rater agreement was substantial (κ = 0.74, 95% CI: 0.62–0.86, N = 120; Landis & Koch, 1977)."
Weighted Kappa is used for ordinal ratings (e.g., severity: none/mild/moderate/severe). It gives partial credit for near-misses: disagreeing by one level is penalised less than disagreeing by three levels. This calculator computes unweighted (simple) Kappa for binary or nominal categories.
Aim for at least 30 subjects minimum, ideally 50–100. The prevalence of each category also matters — if one category is very rare (prevalence paradox), Kappa may be low even when raters mostly agree. Aim for at least 15–20 cases per category for stable estimates.