Central Limit Theorem

Why sums of independent variables look Gaussian, and when they don't.

The Central Limit Theorem is one of the most consequential results in probability — it explains why the Normal distribution shows up everywhere.

For independent identically distributed random variables $X_1, X_2, \dots$ with mean $\mu$ and finite variance $\sigma^2$ , the standardized sample average converges in distribution to a standard Normal:

\sqrt{n}\,(\bar{X}_n - \mu) \xrightarrow{d} N(0, \sigma^2)

In other words, once $n$ is reasonably large, the distribution of the sample mean looks Gaussian — regardless of the original distribution. Roll a die a hundred times and average; the average's distribution is approximately Normal even though a single roll is uniform.

There are caveats. CLT requires finite variance. It fails for Cauchy, and fails for power-law tails with index $\alpha \le 2$ . Convergence rate depends on the skewness and kurtosis of the underlying distribution; for highly skewed cases, $n$ in the hundreds may not be enough.

CLT is why so many estimators are approximately Normal, and why Normal-theory confidence intervals work even when the underlying data isn't Normal. It's the engine behind almost all of frequentist statistics.