An open portfolio of interoperable, industry leading products
The Dotmatics digital science platform provides the first true end-to-end solution for scientific R&D, combining an enterprise data platform with the most widely used applications for data analysis, biologics, flow cytometry, chemicals innovation, and more.
Statsols Head of Statistics & nQuery Lead Researcher, Ronan Fitzpatrick had a chance to sit down with Professor Stephen Senn to talk about Sample Size in Clinical Trial Design.
This video is an excerpt from a feature length video titled "nQuery Interviews Professor Stephen Senn". The full video interview is available to watch on-demand by clicking here.
Ronan Fitzpatrick is Head of Statistics at Statsols and the Lead Researcher for nQuery Sample Size Software. He is a guest lecturer for many institutions including the FDA.
Stephen Senn is a statistical consultant for the Pharmaceutical Industry. Stephen has worked as an academic in a statistical capacity at the Luxembourg Institute of Health, University of Glasgow and University College London. A leader of one of the work packages on the EU FP7 IDEAL project for developing treatments in rare diseases, his expertise is in statistical methods for drug development and statistical inference.
Ronan: So I suppose the first topic that I'm just interested in talking to you about is sample size as a means to better understand design decisions in clinical trials and in any study really. So I know sample size is kind of a standard thing that people ask for particularly like non statisticians they would ask conditions, they would ask I want a sample size for my study and then you usually just give them the sample size and that's the end of it.
You've expressed opinions previously that sample size calculations could actually be a really good mechanism to walk them through what could be interesting choices for their actual designs so rather them coming forward saying I'm doing this and I'm doing additional regression but you could actually talk with them using the sample size calculation as a mechanism to better understand and help them hopefully guide them to making the best decision for their trial.
Stephen: Yeah I think if you've had any experience of consulting I don't necessarily mean consulting externally but consulting internally with any particular organization as a statistician involving sample sizes you'll come across exactly that question how large should the study be and of course what you know as a statistician is you have to go through a number of questions that you have to establish. One of the things you have to establish is you have to establish what analysis will be done. There are many steps before that to establishing what the analysis would be you have to know something about the purpose of the trial about the measures that will be employed and so forth but the analysis is certainly central and in any sample size software including nQuery you have different modules depending upon the analysis that’s used.
So there's a very very close relationship between sample size and analysis but sometimes once the analysis is established people forget this so the analysis is then fixed and they do a particular sample size calculation but they don't then take the opportunity to try it for different types of analysis and this might in self be sometimes instructive as regards choosing analyses. I have to say of course I have an ulterior motive here because being a statistician I'm arrogant enough to know absolutely on every occasion what the perfect analysis is. Unfortunately some of the people I interact with don't know that and so what you can do is you can use sample size software to teach them how much their insistence upon an ignorant alternative will cost in terms of budget.
So, you say ok you don't like the idea of doing a survival analysis because you know hazard ratios well who understands that it's pretty difficult, pretty complicated, why don't we just fix the follow-up time and calculate how many people are alive or dead by a particular time so then you do the analysis that way and you show them well yes you can do that but that will add so many patients to the trial and that turns out to be worth a million and a half euros so can we talk about it now? Can we encourage you to think about this a bit more seriously than just saying I prefer this because I find it easier to interpret.
Ronan: Yeah and there's a number of common examples that you show of mistakes that are made in practices you've seen obviously over your many years working and consulting with people on the on the coalface so some of them will be what I think you call dichotomania. You've talked about this a lot do you want to explain where it showed up?
Stephen: So I'm slightly annoyed because I I think I consider that but it would be difficult to trace it but somewhere there's maybe a record that I'm the first person that used that dichotomania for the obsessive compulsive disorder whereby some people have to take continuous measurements and turn them into meaningless dichotomies but some people feel it's a necessary part of any valuable approach to interpreting clinical trials and what they don't understand is that frequently quite apart from the fact that the increase in the interpretability is often very debatable quite apart from that, the effect on sample size can be catastrophic.
First of all they're frequently done in terms of differences to baseline when a better use of the baseline will be as a covariant and secondly simply by dichotomizing a continuous covariant you increase the sample size by at least 60% if you choose the cut point wisely, if you choose the cut point badly it can be twice or three times. It's a pitman efficiency I'm talking about and there's a very nice paper by fedorov and Meninga I think Federoff certainly anyway is one of the authors in Pharmaceutical Statistics that describes in great detail what the losses are. A similar effort by Stephen Julius and me at the same time also discuss this amongst other matters but it's been well known for decades that this is a really bad thing to do in terms of efficiency yet people insist on doing it.
Ronan: You actually mentioned change from baseline as an example they're, using co-variants which are on of those things that there's a lot of different opinions about should you have these covariance or not and you know obviously the effect on power is only a small component of that choice but it can help you understand adding covariance that are useful that have strong explanatory power does increase the overall power versus yeah I a typical analysis.
Stephen: I think so I think the power and sample size are the main considerations that for example if one considers three common alternatives in trials in which you have continuous measures and you're not dichotomizing but you decide to use the continuous outcome in some sense, three common choices are either to ignore the baselines altogether just use the raw outcomes, to use difference from baseline change from baseline or to use analysis of covariance but it's very important to understand that the three approaches measure exactly the same thing at the end the causal measurement, the effective treatment, is exactly the same that you're trying to measure.
I mean the estimate will not be the same but what you're trying to measure estimand to use some rather more advanced vocabulary which suddenly we've been encouraged to lose by various guidelines and authors and so on. The estimand is the same one can see this by considering that in the perfect trial the baselines would be perfectly balanced and if they were perfectly balanced then in that case the three analyses would give you exactly the same answer and it also therefore follows that as the trial sample size goes to infinity the three analyses would give you the same answer. The only question is which of the three gets there more rapidly as the sample size increases and the answer is analysis of covariance that's the one that gets there faster.
No Comments Yet
Let us know what you think