Top 10 Mistakes of Hypothesis Tests

Summary: Know your enemy! These are the most common mistakes students make in their hypothesis tests on quizzes. Know the right things to do instead!

See also: This article lists blunders. But there are also problems with the theory of hypothesis tests and how they are used by researchers today, as What’s Wrong with Hypothesis Tests? explains.

10. Hypothesis missing μ or p or has the wrong one

It’s really not hard if you just think about your data. Numeric data have means μ; binomial data have percents or proportions p.

9. Incorrect TI-83/84 inputs

If your H₀ has “25” in it, that’s what you should put for μ_o. If your H₁ or H_a has “≠” in it, that’s what should go in the hypothesis on your TI screen.

And hey! write down all your inputs.

8. H₁ contains = or H₀ doesn’t

H₀ comes first and it must contain an = sign. (Some books use ≥, =, and ≤ in H₀.) H_a or H₁ comes second and it must contain >, ≠, or <.

7. H₁ has > or < instead of ≠

If the problem asks you whether something “is” a number or whether two things are “different”, you need to test = and ≠ in your hypotheses. Don’t make assumptions that only > or < matters.

There’s no cool memory trick, because every problem is worded differently. Just make it a habit to read each problem carefully and notice whether it’s asking for a two-tailed test (≠) or a one-tailed test (> or <).

Another common problem is misreading “at least” as ≤ instead of the correct ≥, or misreading “no more than” as ≥ instead of the correct ≤. The Symbol Sheet has some common phrases for the inequalities, but again the best practice is just to read the problem carefully and think about what you’re writing.

6. Sample data in hypotheses

The hypotheses must always contain the number that’s part of the claim, never any number from the sample.

Think about it logically! You’re not testing the sample — you know the sample. You’re testing whether something is true about the general population that your sample came from.

A related problem is picking <, ≠, or > for your H₁ by looking at the sample data. Again, remember that everything about the hypotheses is based on what you want to know, not on the data you actually find.

5. Using a z test instead of a t test

With numeric data (as opposed to counts of categories), be very sure you know the population standard deviation σ before you use a z test. If you don’t know the standard deviation of the population, you can’t use z and you must use t.

Again, no magic bullet here. You need to read the problem carefully.

4. Failing to check requirements

Our procedures need the sampling distribution to be normal, and you’ve learned procedures or rules of thumb to test that. If you don’t make the test, or if the data don’t pass the test, you can’t use a z test or t test.

3. Misreading a small p-value

When the p-value is small, your calculator may show it in scientific notation, such as 7.7321E-5. (This is how the calculator displays 7.7321×10^-5.) Don’t pull a boneheaded move and write p-value = 7.7321. Like any probability, a p-value can’t be > 1. And if you write that, you fail to reject the H₀ that you should reject.

2. Comparing p to α wrong

I see a lot of papers with α = 0.01 and p = 0.0275, then “p < α”. Everything else is worthless if you get this comparison backward!

Some students write the values of p and α above the symbols, or next to them: “p > α (0.0257 > 0.01)”. That’s perfectly acceptable, and it can help you make the comparisons correctly.

And the #1 mistake of hypothesis testing …

1. Reaching a conclusion when p > α

When p > α, you fail to reject H₀ (and you don’t even mention H₁). No conclusion is possible when p > α. If H₀ was “the machine is okay” and H₁ was “the machine is broken”, your only possible conclusion is

We can’t tell, at the ____ level of significance, whether the machine is okay or broken.

When p > α you have to write your conclusion in neutral language, not leaning one way or the other.

Don’t say the machine “might” be anything, or “could” be anything. And especially don’t say “we can’t prove it’s broken” or “we can’t prove it’s okay.” Both of those are true, but they’re only half the truth and they lead the reader to a wrong conclusion. (The most effective way to lie is to tell only part of the truth.)

See also: When p>α, you fail to reject H₀