Empirical Evaluation of Very Large Treatment Effects of Medical Interventions

JMA: October 24/31, 2012

Most medical interventions have modest effects, but occasionally some clinical trials may find very large effects for benefits or harms.

Objective  To evaluate the frequency and features of very large effects in medicine.

Data Sources  Cochrane Database of Systematic Reviews (CDSR, 2010, issue 7).

Study Selection  We separated all binary-outcome CDSR forest plots with comparisons of interventions according to whether the first published trial, a subsequent trial (not the first), or no trial had a nominally statistically significant (P < .05) very large effect (odds ratio [OR], ≥5). We also sampled randomly 250 topics from each group for further in-depth evaluation.

Data Extraction  We assessed the types of treatments and outcomes in trials with very large effects, examined how often large-effect trials were followed up by other trials on the same topic, and how these effects compared against the effects of the respective meta-analyses. Read more