Research 4: Sample Size

Intro Paragraph.   

How big of a sample do you need? What's the right sample size? When it comes to sample size, larger is always better, and that's true. But sample size is limited by feasibility -- whether we find that many people and could get them to participate -- as well as cost and timing.

The other principle with sample size is every sample that you take is going to vary somewhat from what the true population is. So population has the true, right, correct answer, and our sample is going to vary somewhat from that true answer. The smaller your sample size, the bigger those margins of error.

This chart shows that sample size at the bottom and the margin of error along the side. High margin of error is bad, and low margin of error is good. You start out with a sample size of 50, and your margin of error is plus or minus 14 percent. That's an almost 30 percentage point range around your estimate. That is a huge margin of error. That does not give you a lot of confidence if you're trying to make an important business decision. It's probably worth it to spend a little more to get more confidence that the survey data is actually going to be reflecting what their population's doing.

If you double the sample size from 50 to 100, your margin of error goes down to about plus or minus 10 percent. That’s a pretty dramatic improvement. But plus or minus 10 percent is still a very large margin of error. That's why 100 is usuallya  minimum recommended for a quantitative study. However, it's the minimum. It still has a lot of error around it. If you can go from 100 to 200, now your error goes down to around plus or minus 7, so that's even better. If you can go up from 200 to 300, you get it down to 6 percent.

What happens as you add 100 and then another 100 in sample size, you'll see that you will start to get a smaller and smaller and smaller reduction in the amount of error that you get. When you're only at 100 sample size and you add 100 to get n=200, you get about a 3 pp reduction in error. If you add the same number of completes when you're at a sample size of 900 to get n=1000, you only get .2 percentage reduction in margin of error.

The answer to the question, "What's the right sample size?" there's no one fixed answer. It's as much as you can afford with the time, and more is always better, but there's a diminishing level of return. At a certain point, it may not be worth the extra money to get those little small reductions in error.

One point, this only applies if you had a good sample to begin with. If your sampling is messed up and then your slapping low sample sizes on top of it, you don't have very much confidence in the data.

Another question on how do you determine what's the right sample size. You have to think about what sub-groups within an overall umbrella. Let's say I want to understand anybody who buys toothpaste. But then within that group, if you need to measure attitudes/behaviors of people who buy toothpaste in drugstores, and those who have sensitive teeth, and moms of children over 8 years old. You need to make sure your sample is large enough to read all of these smaller subgroups.

Closing Header

Concluding Paragraph.