BrownMath.com → Statistics → HT: Top 10 Mistakes
Updated 21 Aug 2023

Top 10 Mistakes of Hypothesis Tests

Summary: Know your enemy! These are the most common mistakes students make in their hypothesis tests on quizzes. Know the right things to do instead!

See also: This article lists blunders. But there are also problems with the theory of hypothesis tests and how they are used by researchers today, as What’s Wrong with Hypothesis Tests? explains.

10. Hypothesis missing μ or p or has the wrong one
It’s really not hard if you just think about your data. Numeric data have means μ; binomial data have percents or proportions p.
9. Incorrect TI-83/84 inputs
If your H0 has “25” in it, that’s what you should put for μo. If your H1 or Ha has “≠” in it, that’s what should go in the hypothesis on your TI screen.

And hey! write down all your inputs.

8. H1 contains = or H0 doesn’t
H0 comes first and it must contain an = sign. (Some books use ≥, =, and ≤ in H0.) Ha or H1 comes second and it must contain >, ≠, or <.
7. H1 has > or < instead of ≠
If the problem asks you whether something “is” a number or whether two things are “different”, you need to test = and ≠ in your hypotheses. Don’t make assumptions that only > or < matters.

There’s no cool memory trick, because every problem is worded differently. Just make it a habit to read each problem carefully and notice whether it’s asking for a two-tailed test (≠) or a one-tailed test (> or <).

Another common problem is misreading “at least” as ≤ instead of the correct ≥, or misreading “no more than” as ≥ instead of the correct ≤. The Symbol Sheet has some common phrases for the inequalities, but again the best practice is just to read the problem carefully and think about what you’re writing.

6. Sample data in hypotheses
The hypotheses must always contain the number that’s part of the claim, never any number from the sample.

Think about it logically! You’re not testing the sample — you know the sample. You’re testing whether something is true about the general population that your sample came from.

A related problem is picking <, ≠, or > for your H1 by looking at the sample data. Again, remember that everything about the hypotheses is based on what you want to know, not on the data you actually find.

5. Using a z test instead of a t test
With numeric data (as opposed to counts of categories), be very sure you know the population standard deviation σ before you use a z test. If you don’t know the standard deviation of the population, you can’t use z and you must use t.

Again, no magic bullet here. You need to read the problem carefully.

4. Failing to check requirements
Our procedures need the sampling distribution to be normal, and you’ve learned procedures or rules of thumb to test that. If you don’t make the test, or if the data don’t pass the test, you can’t use a z test or t test.
When the p-value is small, your calculator may show it in scientific notation, such as 7.7321E-5. (This is how the calculator displays 7.7321×10-5.) Don’t pull a boneheaded move and write p-value = 7.7321. Like any probability, a p-value can’t be > 1. And if you write that, you fail to reject the H0 that you should reject.
2. Comparing p to α wrong
I see a lot of papers with α = 0.01 and p = 0.0275, then “p < α”. Everything else is worthless if you get this comparison backward!

Some students write the values of p and α above the symbols, or next to them: “p > α (0.0257 > 0.01)”. That’s perfectly acceptable, and it can help you make the comparisons correctly.

And the #1 mistake of hypothesis testing …

1. Reaching a conclusion when p > α
When p > α, you fail to reject H0 (and you don’t even mention H1). No conclusion is possible when p > α. If H0 was “the machine is okay” and H1 was “the machine is broken”, your only possible conclusion is

We can’t tell, at the ____ level of significance, whether the machine is okay or broken.

When p > α you have to write your conclusion in neutral language, not leaning one way or the other.

Don’t say the machine “might” be anything, or “could” be anything. And especially don’t say “we can’t prove it’s broken” or “we can’t prove it’s okay.” Both of those are true, but they’re only half the truth and they lead the reader to a wrong conclusion. (The most effective way to lie is to tell only part of the truth.)

See also: When p>α, you fail to reject H0