Statsols Blog

Integrating Uncertainty in Parameter Estimates

In part 1/3 of this mini series, Head of Statistics at Statsols - Ronan Fitzpatrick - examines key trends in sample size determination which are being felt in trial design.

Integrating Uncertainty in Parameter Estimates

When we conduct a sample size calculation, we require a set of assumed values for the true parameters in the population being analysed in the study. For example, the usage of Cohen’s effect sizes is still a very common way to describe the expected standardised difference in the study. In most cases, researchers rely on a mix of pilot studies, previous studies and informed guesswork to assign a fixed point estimate to these parameters.

Integrating Uncertainty in Parameter Estimates Image.png

To deal with parameter uncertainty, the most common traditional approach has been a sensitivity analysis where a broad range of plausible values for a parameter or parameters is investigated to see its effect on the required sample size. However, the range of plausible values was itself usually based on ad-hoc considerations and researcher intuition. In response to this, several approaches have been proposed to allow researchers to more robustly integrate the uncertainty regarding parameters needed for sample size determination.

 

On the more traditional end, there have been explicit derivations of the sample size required given the amount of uncertainty found in a parameter estimate in another study or a pilot study. For example, Julious and Owen (2006) describe a method of adjusting the sample size based on the number of degrees of freedom associated with variance estimate in a previous study when using a two sample design comparing means. Other authors have suggested that sample size routines and software should automatically provide a lower and upper confidence bounds for the sample size estimate based on the sampling distribution of one or more of the parameters or on repeated simulations of the sample size scenario chosen.

In terms of more complex approaches, mixed Bayesian/likelihood methods allow researchers to explicitly integrate the degree of uncertainty in parameter estimates using statistical priors while still allowing the usage of traditional frequentist methods for the final analysis. Additionally, the increased acceptance of Bayesian methods for statistical analysis has led to a growing and vibrant literature on sample size determination for these methods which explicitly take into account parameter uncertainty and prior information. A good primer to these methods and more can be found in Chow, Shao and Wang (2008).

Despite its growing popularity, Bayesian analyses are still greatly outnumbered by frequentist analyses in most scientific fields. Thus methods which allow for greater explicit justification of parameter uncertainty while still allowing the usage of frequentist methods probably represent the greatest short-term opportunity to create a more holistic approach to sample size justification and place the tradition of sensitivity analysis onto a more solid footing.

You can read Part 2/3 The Effect of Adaptive Trials and Sample Size Re-estimation now.

References

Below you will find a selection of interesting papers and resources on the topics discussed in this mini series. These are only meant as an illustrative set of papers in the relevant fields and should not be considered a comprehensive summary of these areas.

  • Browne, Richard H. "On the use of a pilot sample for sample size determination." Statistics in medicine 14.17 (1995): 1933-1940.
  • Chow, Shein-Chung, Hansheng Wang, and Jun Shao. Sample size calculations in clinical research. CRC press, 2007.
  • Julious, Steven A. "Designing clinical trials with uncertain estimates of variability." Pharmaceutical Statistics 3.4 (2004): 261-268.
  • Julious, Steven A., and Roger J. Owen. "Sample size calculations for clinical studies allowing for uncertainty about the variance." Pharmaceutical statistics 5.1 (2006): 29-37.
  • Kieser, Meinhard, and Gernot Wassmer. "On the use of the upper confidence limit for the variance from a pilot sample for sample size determination."Biometrical Journal 38.8 (1996): 941-949.

Here is some other content from Statsols that you may be interested in.

15 Ways To Reduce Sample Size In Clinical Trials

Reducing sample size without losing power can be accomplished in one of three principles. This paper examines these methods and demonstrates 15 ways to reduce sample size in clinical trials.

15 Ways To Reduce Sample Size In Clinical Trials Download Header 650px.png 

Topics: Power & Sample Size Parameter Estimates