Definition: The p-value is the probability, assuming the null hypothesis is true, of getting a result at least as extreme as the one observed in the sample.
What does the p-value mean?
It answers: if H0 were true, how likely is a sample result this extreme or more? A small p-value signals that the data are hard to explain by chance alone. If it is below the significance level alpha, usually 0.05, we reject H0.
How do you compute one?
Worked example: a coin lands heads 58 times in 100 flips. Test H0: p = 0.5. The standard error is square root of (0.25/100) = 0.05, so z = (0.58 – 0.50)/0.05 = 1.6.
The one-sided p-value is about 0.0548, and the two-sided p-value is about 0.1096. Since 0.1096 is larger than 0.05, we fail to reject H0: 58 heads is not unusual enough to call the coin unfair.
What mistakes occur?
Reading it as the probability that H0 is true. It is not. It is also not the chance the result happened “by luck.” Mathbot says: p-value is computed assuming H0, never about H0.