Permutation test

How to use (3 steps)

Paste data for two groups (A/B) or paired samples.
Choose statistic and settings (auto picks Exact or Monte Carlo).
Run, review the p-value and null distribution, then download a report.

Permutation / randomization test

Permutation test tool

This tool helps you compare two groups without assuming normality. Results depend on your test design and chosen null model.

Test type

Group A Group B

Load from file (.txt/.csv)

Load into

Statistic

Alternative

Method

Permutations

Reference α

α is for reference only. Avoid over-interpreting a single threshold.

Compute bootstrap CI

CI level

Bootstrap iterations

RNG

Include raw null stats in JSON (≤ 200k)

Results

Null distribution

The red line marks the observed statistic. The bars show the null distribution from permutations/sign flips.

Permutation test workflow

Use this page to compare two samples or paired observations with a non-parametric null distribution built from label shuffles or sign flips.

How it works

Choose the independent or paired design, select the statistic, define the one-sided or two-sided alternative, and run Exact or Monte Carlo permutations to build the null distribution.

When to use

Use it when you want a distribution-light significance check for small samples, skewed values, A/B experiments, classroom demonstrations, or paired before-after measurements.

Common mistakes to avoid

Using independent mode for paired measurements from the same unit.
Choosing a statistic after seeing which one gives the smallest p-value.
Reporting only the p-value without the observed difference and bootstrap confidence interval.
Treating Monte Carlo randomness as exact when the number of permutations is low.

Analysis workflow

Decide the design, statistic, and alternative before running. Then compare the observed statistic with the null distribution, p-value, confidence interval, and sample sizes.

FAQ

What does the p-value mean?

It is the probability (under the null) of seeing a statistic at least as extreme as the observed one. It does not prove causality or practical importance.

Exact vs Monte Carlo?

Exact enumerates all permutations when feasible. Monte Carlo approximates with random permutations.

Is my data uploaded to a server?

No. Everything runs locally in your browser.

Does a large p-value mean “random”?

Not necessarily. A large p-value only means your data is not surprising under the chosen null model. It does not prove randomness or correctness.

What should I decide before running?

Decide whether the samples are independent or paired, which statistic answers the question, and whether the alternative is one-sided or two-sided.

How to interpret permutation results

Null model

The null model assumes labels or signs can be rearranged without changing the structure of the data. That assumption must match the study design.

Statistic choice

Mean difference is easy to read, but median or absolute-difference statistics may be better when data are skewed or outlier-heavy.

Exact versus Monte Carlo

Exact mode enumerates all possible rearrangements when feasible. Monte Carlo mode samples many rearrangements, so the seed and trial count matter.

Reporting

Report the observed statistic, p-value, alternative, permutation count, and confidence interval. Add practical context before making a decision.