Why is Sample Size Important?

A good statistical study is one that is well designed and leads to valid conclusions. This however, is not always the case, even in published studies. In Cohen’s (1962) seminal power analysis of the journal of Abnormal and Social Psychology he concluded that over half of the published studies were insufficiently powered to result in statistical significance for the main hypothesis.

The power of a statistical test is the probability that a test will reject the null hypothesis when the null hypothesis is false. That is, power reflects the probability of not committing a type II error. The two major factors affecting the power of a study are the sample size and the effect size. The larger the sample size is the smaller the effect size that can be detected. The reverse is also true; small sample sizes can detect large effect sizes. While researchers generally have a strong idea of the effect size in their planned study it is in determining an appropriate sample size that often leads to an underpowered study. This poses both scientific and ethical issues for researchers.

A study that has a sample size which is too small may produce inconclusive results and could also be considered unethical, because exposing human subjects or lab animals to the possible risks associated with research is only justifiable if there is a realistic chance that the study will yield useful information. Similarly, a study that has a sample size which is too large will waste scarce resources and could expose more participants than necessary to any related risk. Thus an appropriate determination of the sample size used in a study is a crucial step in the design of a study.

More recent studies analysing the power of published papers has shown that, even still, there are large numbers of papers being published that have insufficient power. With the availability of sample size software such as nQuery and nTerim which can calculate appropriate sample sizes for any given power such issues should not be arising so often today.

Want to learn more? Read our guide below