Generalizing the Results of Randomized Controlled Trials

July 16, 2012 in Econometrics

The benefit of randomized controlled trials is that one can identify the causal effect of a certain treatment in the absence of selection effects (assuming the randomization is properly done).  However, RCT results often do not generalize to the larger population.  Reasons for this include:

  • The RCT population may not be representative of the population at large.  Frequently, RCT participants receive treatment for a disease which is their only disease; in the real world, patients suffer from multiple comorbidities and thus the RCT treatment may be less effective in practice.
  • Medication adherence is often lower in practice than during an RCT.  “For example, the Women Take Pride study assessed group versus self-directed behavioral interventions for women with heart disease (Janevic et al., 2003) and found much higher adherence rates for the preferred interventions (Long, Little and Lin, 2008).”
  • The patients who choose a certain treatment in practice may not be similar to the treatment patients are assigned to in the RCT.  For instance, sick patients outside an RCT may not choose an intensive treatment if the side effects are too severe. Read more