Analytic Models Evaluated for Individually Randomized Group Treatment Trials

Key Insights

• Researchers found that in individually randomized group treatment (IRGT) trials, the analytic model should account for correlations in outcome measures when participants receive an intervention from the same source.
• The study addressed a knowledge gap by examining analytic models for IRGT trials with complex clustering arising from participants interacting with multiple agents or single agents in nested and crossed designs.
• The study revealed substantial inflation in the type I error rate in studies with nested designs when the analytic model did not account for participants interacting with multiple agents.

In individually randomized group treatment (IRGT) trials, selecting an appropriate analytic model is crucial to avoid inflated type 1 error rates, according to a report from the NIH Pragmatic Trials Collaboratory’s Biostatistics and Study Design Core. The report, published in Statistics in Medicine, emphasizes the importance of accounting for correlations in outcome measures that arise when study participants receive an intervention from the same source.

Understanding IRGT Trials

IRGT trials involve randomly assigning individuals to study arms but delivering the intervention through shared "agents," such as clinicians, therapists, or trainers. Interactions between participants who share the same agent can lead to correlations in study outcomes. These delivery agents may be nested in or crossed with study arms, and participants may interact with a single agent or multiple agents. The absence of a systematic approach to identify appropriate analytic models for these complex designs prompted the NIH Collaboratory’s investigation.

Simulation Study Details

To address this knowledge gap, members of the NIH Collaboratory’s Biostatistics and Study Design Core conducted a simulation study. This study examined the performance of various analytic models for IRGT trials where complex clustering results from participants interacting with multiple agents or single agents in both nested and crossed designs.

Key Findings

The study revealed that in trials with nested designs, substantial inflation in the type I error rate occurred when the analytic model failed to account for participants interacting with multiple agents. This highlights the necessity of selecting analytic models that appropriately address the complex clustering inherent in IRGT trial designs.

Implications for Researchers

These findings underscore the importance of careful consideration when choosing analytic models for IRGT trials. Researchers should ensure that their chosen model accounts for the correlations in outcome measures that arise from shared intervention agents, particularly in nested designs, to avoid inaccurate conclusions.