This is an excerpt from our webinar "Innovative Strategies for Successful Trial Design."

Dealing with design constraints such as
Multi-centre/Hierarchical Effects

This video is an excerpt from our webinar: Innovative Strategies For Successful Trial Design

So let's move on to design constraints so how do we deal with design constraints. Real world constraints are a reality of designing any trial.  I think in phase 3 clinical trials you maybe have a bit more because of the resources available and obviously the importance of getting approval there's perhaps more freedom to kind of run roughshod over those issues and just throw more resources to basically get what you want in terms of having enough people to randomize each group and so forth, but obviously, if for pre clinical research or for academic research that's not something that's really feasible. Even within phase 3 you know there is a phase 3 clinical trials and bigger trials there is these resources and these constraints become a real issue that has to be dealt with and I think you know it's nice to design from first principles which is that we should have randomization, we should have everything on a fully randomized approach rather than cluster randomize. We prefer everyone to be from a kind of similar population if that's a representative population and so on, the same number of people from each center and so on.

Those kind of first principles are from like model out effectively are very useful for having an idea of the platonic ideal of your trial is definitely useful but the real-world issues that show up have to be accounted for. I think one note to make particularly in the context of clinical trials is that clinical trials are now a global business, global trials are much more common now, going from centers from very wide geographic spaces is very common now and a greater reliance on CRO’s to kind of help sponsors achieve that goal.

Some of that comes from a cost point of view, some of that's from a time point of view, some of that's just from we want more people point of view but it is a reality of clinical trials right now and of course that creates a lot of challenges and that's probably the perspective we're looking at it from this case when we're talking about constraints, that's obviously a negative framing but there is also the truth that as clinical trials become more global as we're getting this more wide sample of world. 

You have to deal with the fact that those samples might be from different type of populations but that does give you some opportunities as well in terms of generalizability, the subgroup analyses like your flexibility in terms of design. So despite the negative framing you know the way things are going at the moment does offer a lot of opportunities if you're particularly open-minded about them.

So in terms of that we're looking at here specifically in terms as an example of a real world constraint is this multi center hierarchical effect. In multi-center hierarchical affects what we're talking about here well they're kind of linked together it's basically the idea that generally you'll be bringing data from multiple centers, multiple clusters  like schools or hospitals depending on the type of trial and then each of those clusters, each of those units are more self similar within them than they might be across them. So you know people within a hospital have a certain standard of care versus another hospital. It's one of those issues that Stephen Senn, who we collaborated with, liked to emphasize.

One of the biggest sources of variation that doesn't get talked about in terms of treatment outcome in the real world, isn't you know which treatment you get or which drug you got it's the actual doctor which you got. I think Cox has a similar point and  you can see these graphs, for example, I think they were trying to show tonsillitis where people got their tonsils out and they showed well the biggest variance there wasn't some hospitals or some people believe tonsils are worse, some doctors are like that's a normal thing to do still.

So I think this kind of idea of  the center effect is incredibly important to deal with particularly as you're going to these much more global trials and you have if you're going from these very distinct populations how do you deal with that but that kind of reflects the fact that you know for a clinical trial, particularly a phase 3, it's not realistic to have enough people often time from a single clinical unit there's just too many people required for the power analysis or for other constraints safety analysis etc. Vaccine study, for example, will be a case where you need an incredibly large sample size typically to make  appropriate statistical inferences. The important thing to know is that if you have a center effect, if you are taking from multiple populations, multiple schools, multiple hospitals, multiple counties etc you do need to take account of that within the model and have to think about the decisions that are now open to you or which may not be open to you because of that structure.

Multi-centre/Hierarchical Effects

So I think one of the most important things which have the biggest effect is your randomization levels so one choice would be  within each of your clusters, each of your centers you just randomized 50% say to the treatment, 50% to the placebo or control but another choice might be to go take 50% of the centers and put 50% so that everyone gets the treatment and 50% everyone gets to control.

So the first would be what we call a multi center design or multi center trial so we're fairly confident that it more or less replicates a full randomization trial but the center effect is just the covariant in our model effectively so you know we're just going to treat that as something that we need to just account for.  Whereas, the latter is a cluster randomized approach and has a very significant difference because obviously now the fact that people are very similar within each cluster or are more likely to be similar within a cluster means that you're kind of losing some of the randomization from a certain perspective. If it was all male schools and female schools you can imagine the male school intervention, you'd expect that kind of distinct effect from that if you had too many male schools in one group than the other that would have a significant effect on how closely it approaches real randomization or true randomization.

There's a wider issue in how to deal with those Center effects,  should it be a fixed effect versus a random effect. I think generally the latter is probably true. In terms of the randomization level you know from a statistical model point of view that's like the biggest issue will just be probably defining how many degrees of freedom you will have for your treatment effect and generally kind of hierarchical generalized linear models, the perspective is usually the degrees of freedom is the degrees of freedom at the level that you're at, the level that you randomized at, for example, if you are doing a two-sample t-tests or equivalent approach where you did randomization within each cluster so 50% each group within each cluster the degrees of freedom  can be considered to be as if it was for a fully randomized approach minus the degree of freedom taken away for the center effect.

Whereas, if you were randomizing at the level of the cluster, like a cluster randomized design, the degrees of freedom actually then becomes the number of clusters which as you can imagine will tend to be much lesser than you'd expect for degrees of freedom for all of the subjects altogether within each cluster. Lets say you have ten clusters each people have 50 within them you know you're talking about 500 degrees of freedom minus 2 for the treatment effect, minus another one for the center effect, that’s around 500 degrees of freedom whereas if you're at the randomization level of the cluster  you only have nine degrees of freedom or eight degrees of freedom for a t-test. That could have a very substantial difference as you could imagine.

Worked Example

Then for the random effects  generally it kind of makes sense to treat it as a random effect because the very brief understanding of random effect is that it's where you consider a covariate to be a sample from the wider set of units at that level and that kind of makes sense if you're sampling in a set of centers, schools, hospitals etc. There is a small note to make that you know there is two definitions of random effects that are relevant for hierarchical modeling where you just have a random effect on the center effect alone which is basically just accounting for the fact that you're treating those standards as if they're from a random sample and that will generally reduce the overall variance. That is different from making hypotheses around each center having its own individual effect. If you want to predict that in future centers individual effect. The latter case is really just averaging the effect over all the centers there’s still a marginal effect over all the center's whereas you know if you were trying to predict  this cluster was like this or a future cluster will have an effect of Y it will be different from the other clusters that’s a treatment by cluster effect random effect and that's quite a very different one and just to note that that has much higher requirements in general for sample size calculations.

If you're interested in n of one trials, it's a very similar concept. I've covered that previously and the paper by Stephen Senn that that's based on ,where he came in and talked about it, has a very similar issue like the idea of predicting for individuals n of one trials very similar issue you're having this kind of treatment by n of one trial interaction very much requires a higher sample size, a higher burden of evidence basically. One note to make is that while you might expect a hierarchical effect you know to be a constraint to make you less efficient you can actually show that in cases it might actually increase or decrease your sample size requirement in terms of a proxy for efficiency. I'm I'm going to illustrate that in an example here in the moment.

Like any trial design, careful planning can mitigate or eliminate certain risks.
If you are exploring adaptive designs, one important factor is to select validated and trusted software that is designed for your adaptive trials. nQuery has dedicated adaptive trial design functionality that contains a selection of sample size tables designed specifically for areas of adaptive design.

Learn More About Adaptive Clinical Trials

We recently hosted a webinar examining Advantages & Disadvantages of Adaptive Sample Size Re-Estimation. You can watch this webinar on demand by clicking the image below.

In this webinar you’ll learn about:

• Advantages & Disadvantages of Adaptive Design
• Evaluating Unblinded SSR for your trials
• Evaluating Blinded SSR for your trials