p-values and Statistical Decisions

A $p$-value answers a specific question: under the null hypothesis, how surprising is the data we observed (or something more extreme)?

A small $p$-value indicates the observed result would be rare if $H_0$ were true.

What $p$-values are not:

$p$ is not $P(H_0 \mid \text{data})$. Inverting the conditional requires Bayes and a prior.$p < 0.05$ doesn't mean "true" — it means "significant at the $5\%$ level."A non-significant result doesn't prove $H_0$ — absence of evidence is not evidence of absence.

Multiple testing is a real-world hazard. Run $20$ independent tests at $\alpha = 0.05$ and you expect $1$ false positive even when nothing is going on. Bonferroni controls family-wise error rate, Benjamini–Hochberg controls false discovery rate, and both matter when screening many factors, strategies, or signals.