This is an excerpt from our webinar "Alternatives to the P-value and Power - The effect on Sample Size Determination." where our host Ronan Fitzpatrick, Head of Statistics at Statsols demonstrates an alternative to traditional sample size calculations by examining Confidence Interval (CI) and the role of precision sample size.

Confidence Interval -
Precision Sample Size

When the ASA released their statement on p-values and statistical significance, it created a new wave of discussion and debate on p-values and null hypothesis significance testing (NHST).

With numerous alternative approaches to statistical inference being championed and some calls to abandon statistical significance completely, it has never been a better time to catch up on these alternatives and see their potential effects on the areas of study planning and sample size determination.

Below is an extract from the webinar where our host Ronan Fitzpatrick, Head of Statistics at Statsols demonstrates an alternative to traditional sample size calculations by examining Confidence Interval (CI) and the role of precision sample size.

The full webinar is available here:  Alternatives to the p-value & power - The effect on sample size determination

Can't watch a video right now?
The slides and auto generated transcript for the webinar are below. 

Confidence Interval

A confidence interval (CI) is a type of interval estimate, computed from the statistics of the observed data, that might contain the true value of an unknown population parameter.

The confidence interval in the frequentist school is by far the most widely used statistical  interval and  the Layman's definition would be the probability that you will have the true value for a parameter such as the mean or the mean difference or the odds ratio under repeated sampling.

Sample size determination is targeting the interval width, that's quite closely tied to the idea of the standard error of the estimate which is  tied to your law of large number theory.

But of course, we're not focusing on power today. It's just a small note you may be interested in.

Confidence Interval Examples

Below are some simple confidence interval sample sizes because  the assumption here is that most of you are familiar with doing this.  The major thing that we're doing here is that we're changing the framework in the sense that you we have confidence level instead of significance level standard deviation would be the same thing if we were doing a one-sample t-test but now instead of power we're targeting the half width of the confidence interval here. This is a real paper just looking at Rosiglitazone for insulin sensitivity from diabetic medicine and this is just a simple example that we'll explore. You can find these examples in detail in the Webinar video above.

How nQuery can used to calculate sample size using the confidence interval for example 1.

How nQuery can used to calculate sample size using the confidence interval for example 2.

How nQuery can used to calculate sample size using the confidence interval for example 3A.

How nQuery can used to calculate sample size using the confidence interval for example 3B.

Other Frequentist Intervals

In many cases you'll probably as find sample size for confidence intervals is fine, but there are other frequentist intervals that are used or perhaps are underused that can target other types of parameter.

Two of the other frequentist intervals you  may be familiar with but maybe haven't used in a while is the prediction interval and the tolerance interval.

So the prediction interval is an interval to contain a future sample subject or parameter with a given probability. 

A tolerance interval is an  interval within which, with some confidence level, a specified proportion of a sampled population falls.

Watch the full webinar