The four questions
Statistical test selection feels overwhelming because textbooks present tests as a list. In practice the decision is a small tree:
- Outcome type — continuous (blood pressure, score), categorical (pass/fail, category counts), or ordinal (Likert ratings)?
- How many groups or conditions are you comparing — one, two, or three plus?
- Are measurements independent (different people per group) or paired (same people measured twice)?
- Are the assumptions met — roughly normal residuals and comparable variances for parametric tests, expected counts ≥ 5 for chi-square?
Continuous outcomes
Two independent groups: independent-samples t-test; if normality is doubtful and samples are small, Mann-Whitney U. Same subjects twice: paired t-test, or Wilcoxon signed-rank as the non-parametric fallback. One group against a known value: one-sample t-test. Three or more groups: one-way ANOVA, with post-hoc tests (Tukey) to find which pairs differ; Kruskal-Wallis when assumptions fail.
Relationships rather than differences: Pearson correlation for linear relationships between two continuous variables, Spearman when the relationship is monotonic but not linear or the data are ordinal. When you want to predict one variable from others, you are in regression territory.
Categorical outcomes
Comparing proportions across groups — did more people in condition A pass than in condition B? — is a chi-square test of independence on the contingency table. With small expected counts (any cell below 5), use Fisher’s exact test instead. Paired categorical data (the same people before/after) call for McNemar’s test, a distinction reviewers check.
Assumptions: when to worry
Parametric tests are more robust than their reputation. With n ≥ 30 per group, the central limit theorem covers moderate non-normality; below that, inspect a histogram and consider the non-parametric twin. Unequal variances matter more — Welch’s t-test (the default in most modern software) handles that and costs almost nothing when variances happen to be equal.
What actually invalidates analyses is rarely the distribution — it is dependence between observations (students nested in classrooms, repeated measures treated as independent). If your data have structure, you need mixed models, not a different two-sample test.
Where Kahubi fits
Kahubi’s statistics flow takes a CSV, suggests the appropriate test from your design, runs it (27+ tests from descriptives to PCA), charts the result and writes the APA-formatted results paragraph. The free calculators on this site run the individual tests in your browser — no signup, data never leaves the page.
Last updated 2026-07-09.