Cherry-picking by trialists and meta-analysts can drive conclusions about intervention efficacy
Name:
Publisher version
View Source
Access full-text PDFOpen Access
View Source
Check access options
Check access options
Abstract
Objective: To determine whether disagreements among multiple data sources affect systematic reviews of randomized clinical trials (RCTs). Study Design and Setting: Eligible RCTs examined gabapentin for neuropathic pain and quetiapine for bipolar depression, reported in public (e.g., journal articles) and non-public sources (clinical study reports [CSRs] and individual participant data [IPD]). Results: We found 21 gabapentin RCTs (74 reports, six IPD) and seven quetiapine RCTs (50 reports, one IPD); most were reported in journal articles (18/21 [86%] and 6/7 [86%], respectively). When available, CSRs contained the most trial design and risk of bias information. CSRs and IPD contained the most results. For the outcome domains “pain intensity” (gabapentin) and “depression” (quetiapine), we found single trials with 68 and 98 different meta-analyzable results, respectively; by purposefully selecting one meta-analyzable result for each RCT, we could change the overall result for pain intensity from effective (standardized mean difference [SMD]=-0.45; 95%CI -0.63 to -0.27) to ineffective (SMD=-0.06; 95%CI -0.24 to 0.12). We could change the effect for depression from a medium effect (SMD=-0.55; 95%CI -0.85 to -0.25) to a small effect (SMD=-0.26; 95%CI -0.41 to -0.1). Conclusions: Disagreements across data sources affect the effect size, statistical significance, and interpretation of trials and meta-analyses.Citation
Mayo-Wilson, E., Li, T., Fusco, N., Bertizzolo, L., Canner, J.K., Cowley, T., Doshi, P., Ehmsen, J., Gresham, G., Guo, N., Haythornthwaite, J.A., Heyward, J., Hong, H., Pham, D., Payne, J.L., Rosman, L., Stuart, E.A., Suarez-Cuervo, C., Tolbert, E., Twose, C., Vedula, S., Dickersin, K. (2017). Cherry-Picking by Trialists and Meta-Analysts Can Drive Conclusions About Intervention Efficacy. Journal of Clinical Epidemiology. DOI: 10.1016/j.jclinepi.2017.07.014Keyword
clinical study reportsdata comparison
publication bias
reporting bias
Systematic reviews (Medical research)
Data Accuracy
Identifier to cite or link to this item
http://hdl.handle.net/10713/7090ae974a485f413a2113503eed53cd6c53
10.1016/j.jclinepi.2017.07.014