How to use (3 steps)
- Paste data for two groups (A/B) or paired samples.
- Choose statistic and settings (auto picks Exact or Monte Carlo).
- Run, review the p-value and null distribution, then download a report.
Permutation / randomization test
Permutation test tool
This tool helps you compare two groups without assuming normality. Results depend on your test design and chosen null model.
α is for reference only. Avoid over-interpreting a single threshold.
Results
Null distribution
The red line marks the observed statistic. The bars show the null distribution from permutations/sign flips.
How to use this tool effectively
This guide helps you use Permutation test in a repeatable way: define a baseline, change one variable at a time, and interpret outputs with explicit assumptions before you share or act on results.
How it works
The page applies deterministic logic to your inputs and shows rounded output for readability. Treat it as a comparison workflow: run one baseline case, adjust a single parameter, and measure both absolute and percentage deltas. If a result seems off, verify units, time basis, and sign conventions before drawing conclusions. This approach keeps your analysis reproducible across teammates and sessions.
When to use
Use this page when you need a fast estimate, a classroom check, or a practical what-if comparison. It works best for planning and prioritization steps where you need direction and magnitude quickly before investing in deeper modeling, manual spreadsheets, or formal external review.
Common mistakes to avoid
- Changing multiple parameters at once, which hides the true cause of output movement.
- Mixing units (percent vs decimal, monthly vs yearly, gross vs net) across scenarios.
- Comparing with another tool without aligning defaults, constants, and rounding rules.
- Using rounded display values as exact downstream inputs without re-checking precision.
Interpretation and worked example
Run a baseline scenario and keep that result visible. Next, modify one assumption to reflect your realistic alternative and compare direction plus size of change. If the direction matches your domain expectation and the size is plausible, your setup is usually coherent. If not, check hidden defaults, boundary conditions, and interpretation notes before deciding which scenario to adopt.
See also
FAQ
What does the p-value mean?
Exact vs Monte Carlo?
Is my data uploaded to a server?
Does a large p-value mean “random”?
What should I do first on this page?
Start with the minimum required inputs or the first action shown near the primary button. Keep optional settings at defaults for a baseline run, then change one setting at a time so you can explain what caused each output change.
How to use Permutation test effectively
How this tool helps
Tools are designed for quick scenario comparisons. They work best when you keep one question per run, define success criteria first, and avoid switching objectives mid-stream. This reduces decision noise and produces results you can defend in follow-up review.
Input validation checklist
Before running, verify that required values are in the right format, that optional flags are intentionally set, and that baseline assumptions reflect current conditions. Invalid assumptions are often mistaken for tool bugs, so validation is part of interpretation quality.
Scenario planning pattern
Build three rows: conservative, expected, and aggressive cases. Keep data sources transparent for each case and compare output spacing. The pattern helps you spot non-linear jumps and decide whether a model is stable under plausible variation.
When to revisit inputs
Revisit inputs when input scale changes, time window shifts, or downstream decisions add new constraints. If constraints change, your previous output remains a useful reference but should not be treated as final guidance.