Running an Optimization Trial during COVID-19: Adaptations and Considerations from the Field

Kate Guastaferro, PhD, MPH^✉ and Rachel D. Wells, PhD^✉; Optimization of Behavioral and Biobehavioral Interventions (OBBI) SIG Chairs

On top of heightened personal and professional stressors of the COVID-19 pandemic, scientists have needed to quickly adapt intervention studies for remote delivery and/or assessment, modify study objectives with funders, and even alter outcomes or endpoints. The complexity of adaptations and difficult decisions is largely contingent upon the experimental design in use. Here, we interviewed two SBM member-led research teams to learn how they adapted their active optimization trials during COVID-19.

Briefly, an optimization trial, a feature of the multiphase optimization strategy (MOST),¹ is an experiment designed to identify which components produce the best expected outcome while balancing constraints (e.g., time, cost). Often, the optimization trial uses a factorial experimental design wherein two or more factors (i.e., candidate intervention components) with two or more levels are manipulated with the purpose of identifying the individual effect of each factor on average on the desired outcome (main effect) as well as whether the effect of a factor varies depending on the level of other factors (interaction).

Drs. Sarabeth Broder-Fingert & Emily Feinberg (Boston University)

What is the purpose of your study?

Our study seeks to examine implementation strategies for Family Navigation, an evidence-based care management strategy.  Using a 2⁴ full factorial experiment we aim to recruit 304 families to determine what combination of four components (and at what level) have the greatest effect on completion of family behavioral health goals.²
 

What adaptations have you made and what have you had to consider in light of needed adaptations?

We have not had to directly alter the design of the optimization trial, but instead had to consider the factor designed to compare in-clinic versus community family navigation. During the COVID-19 pandemic, neither component could be delivered as designed – families are not coming into the clinic and Navigators cannot do home visits. Instead, families are receiving the “COVID version” wherein personal virtual visits are done via video or audio call.

We are considering three options. First, we could keep doing as we are and at some point, return to services as planned. Second, keep randomizing to all 16 experimental conditions, but in analyses collapse the factor (effectively not treating it as an experimental factor). These options allow us to deal with this unplanned scenario analytically. However, a third option might be to examine the core elements of the component and see if there are functional differences between the elements of the intervention delivered by the clinic or community health worker.
 

Dr. J. Nick Dionne-Odom (University of Alabama at Birmingham)

What is the purpose of your study?

The purpose of our study is to pilot test an optimization trial approach to develop and refine the decision partnering skills of family caregivers of persons with newly-diagnosed advanced cancer using a 2³ factorial design with 40 family caregivers of persons with newly-diagnosed advanced cancer.
 

What adaptations have you made and what have you had to consider in light of needed adaptations?

We haven't had to adapt the design in significant ways from what we originally planned. Our intervention has always been delivered primarily telephonically and so our general operations were able to perform robustly even after COVID-19. We did have to convert our in-person recruitment approach to one that was done wholly by mail and phone. 

We had to consider how to build interest in our study and trust with our interventionists without the aid of in-person contact.
 

We asked our optimization trialists to offer some advice, rooted in their experience and success in adapting their optimization trial to the COVID-19 pandemic:

Drs. Broder-Fingert & Feinberg: “We think two things prepared us well to adapt to the uncertain times of COVID-19. First, we had a really good, experienced project manager who was able to help execute, and trouble-shoot, a rigorous experimental design. Second, our familiarity with the component allowed us to consider how each component was implemented in each level of the factor and identify salvageable research questions. All in all, we are actually on track with our recruitment goals. Now that people are returning to the clinic, we are getting more referrals for mental and behavioral health needs.”

Dr. Dionne-Odom: “Organization and written detailed protocols for study processes are key to smooth operations. We have separate interventionists scripts, toolkit materials, and charting templates, and a robust fidelity monitoring plan of all recorded intervention sessions to ensure participants receive their assigned condition.” 

 

References

1. Collins LM. Optimization of Behavioral, Biobehavioral, and Biomedical Interventions: The Multiphase Optimization Strategy (MOST). New York: Springer; 2018.
2. Broder-Fingert S, Kuhn J, Sheldrick RC, et al. Using the Multiphase Optimization Strategy (MOST) framework to test intervention delivery strategies: A study protocol. Trials. 2019;20(1):1-15. doi:10.1186/s13063-019-3853-y