3.7 Interpreting p-Values

What a p-value measures

A p-value is the probability of getting a sample statistic as extreme as (or more extreme than) the one observed, assuming the null hypothesis is true. It measures how surprising the data would be if H₀ were correct — it is NOT the probability H₀ is true.

Small vs. large p-values

A small p-value means the observed result would be unlikely under H₀, giving evidence against H₀ (supporting Hₐ). A large p-value means the result is consistent with H₀, so we fail to reject H₀ — this doesn't prove H₀ true, just that data don't contradict it.

Comparing to significance level

We compare the p-value to a pre-set significance level (α), often 0.05. If p-value < α, reject H₀ — results are statistically significant. If p-value ≥ α, fail to reject H₀. The conclusion should always be stated in context of the problem.

Key terms

p-value

Probability of getting a sample result as extreme or more extreme than observed, assuming H₀ is true.

Small p-value meaning

Strong evidence against H₀; result is unlikely if H₀ were true.

Large p-value meaning

Data are consistent with H₀; we fail to reject H₀ (does not prove it true).

Significance level (α)

The threshold p-value must be below to reject H₀; commonly 0.05.

Statistically significant

Describes a result where the p-value is less than α, leading to rejection of H₀.

Try an AP-style question

What is a p-value?

1. The probability that the null hypothesis H₀ is actually true
2. If H₀ is true, the chance of a result at least this extreme
3. The proportion of successes observed in the random sample
4. The probability that the alternative hypothesis Hₐ is true