Fragility index of network meta-analysis with application to smoking cessation data.

Publisher: Elsevier Country of Publication: United States NLM ID: 8801383 Publication Model: Print-Electronic Cited Medium: Internet ISSN: 1878-5921 (Electronic) Linking ISSN: 08954356 NLM ISO Abbreviation: J Clin Epidemiol Subsets: MEDLINE

Publication: New York : Elsevier
Original Publication: Oxford ; New York : Pergamon Press, c1988-

Network Meta-Analysis*; Smoking Cessation/*statistics & numerical data; Confidence Intervals ; Humans ; Meta-Analysis as Topic ; Odds Ratio ; Probability ; Reproducibility of Results ; Smoking Cessation/methods

Background and Objectives: The network meta-analysis (NMA) is frequently used to synthesize evidence for multiple treatment comparisons, but its complexity may affect the robustness (or fragility) of the results. The fragility index (FI) is recently proposed to assess the fragility of the results from clinical studies and from pairwise meta-analyses. We extend the FI to NMAs with binary outcomes.
Methods: We define the FI for each treatment comparison in NMAs. It quantifies the minimal number of events necessary to be modified for altering the comparison's statistical significance. We introduce an algorithm to derive the FI and visualizations of the process. A worked example of smoking cessation data is used to illustrate the proposed methods.
Results: Some treatment comparisons had small FIs; their significance (or nonsignificance) could be altered by modifying a few events' status. They were related to various factors, such as P-values, event counts, and sample sizes, in the original NMA. After modifying event status, treatment ranking measures were also changed to different extents.
Conclusion: Many NMAs include insufficiently compared treatments, small event counts, or small sample sizes; their results are potentially fragile. The FI offers a useful tool to evaluate treatment comparisons' robustness and reliability.
(Copyright © 2020 Elsevier Inc. All rights reserved.)

BMC Med. 2019 Sep 16;17(1):174. (PMID: 31526369)
J Clin Epidemiol. 2018 Aug;100:103-110. (PMID: 29339215)
Res Synth Methods. 2016 Sep;7(3):236-63. (PMID: 26754852)
Proc Natl Acad Sci U S A. 2013 Nov 26;110(48):19313-7. (PMID: 24218581)
Psychol Bull. 1960 Sep;57:416-28. (PMID: 13744252)
BMC Med. 2016 Jul 04;14(1):100. (PMID: 27377062)
BMJ. 2014 Sep 24;349:g5630. (PMID: 25252733)
J Clin Epidemiol. 2020 Aug;124:34-41. (PMID: 32298777)
Science. 2015 Aug 28;349(6251):aac4716. (PMID: 26315443)
Nature. 2019 Mar;567(7748):305-307. (PMID: 30894741)
N Engl J Med. 2008 Jan 17;358(3):252-60. (PMID: 18199864)
BJU Int. 2018 Jul;122(1):160-166. (PMID: 29569390)
Biom J. 2018 Nov;60(6):1040-1058. (PMID: 30062789)
Ann Intern Med. 2009 Aug 18;151(4):264-9, W64. (PMID: 19622511)
Value Health. 2011 Jun;14(4):417-28. (PMID: 21669366)
Lancet Oncol. 2019 Oct;20(10):e551. (PMID: 31578993)
Stat Med. 2002 Aug 30;21(16):2313-24. (PMID: 12210616)
Int J Epidemiol. 2017 Dec 1;46(6):1999-2008. (PMID: 29040566)
J Clin Epidemiol. 2016 Nov;79:46-54. (PMID: 27079845)
PLoS One. 2014 Jan 22;9(1):e86754. (PMID: 24466222)
Eur Heart J. 2017 Feb 1;38(5):346-348. (PMID: 28417139)
Milbank Q. 2016 Sep;94(3):485-514. (PMID: 27620683)
PLoS Med. 2014 Jul 08;11(7):e1001671. (PMID: 25003600)
Int J Epidemiol. 2012 Jun;41(3):818-27. (PMID: 22461129)
Int J Epidemiol. 2020 Jun 1;49(3):885-893. (PMID: 31764988)
JAMA. 2019 Jun 4;321(21):2067-2068. (PMID: 30946431)
Med Decis Making. 2013 Jul;33(5):607-17. (PMID: 23104435)
J Clin Epidemiol. 2014 Jun;67(6):622-8. (PMID: 24508144)
Diabetes Res Clin Pract. 2017 Dec;134:91-105. (PMID: 29037877)
Stat Interface. 2020;13(4):425-436. (PMID: 32742550)
Med Decis Making. 1998 Jan-Mar;18(1):37-43. (PMID: 9456207)
Nature. 2016 May 25;533(7604):452-4. (PMID: 27225100)
J Clin Epidemiol. 2019 Feb;106:41-49. (PMID: 30342086)
J Clin Epidemiol. 2020 Feb;118:86-92. (PMID: 31743750)
JAMA. 2014 Jan 22-29;311(4):405-11. (PMID: 24449319)
Biostatistics. 2009 Oct;10(4):792-805. (PMID: 19687150)
Ann Intern Med. 2015 Jun 2;162(11):777-84. (PMID: 26030634)
Res Synth Methods. 2012 Jun;3(2):80-97. (PMID: 26062083)
Lancet. 2004 Apr 24;363(9418):1341-5. (PMID: 15110490)
Res Synth Methods. 2016 Mar;7(1):6-22. (PMID: 26536149)
BMJ. 2001 Jan 27;322(7280):226-31. (PMID: 11159626)
Stat Med. 2013 Dec 30;32(30):5414-29. (PMID: 24123165)
J Clin Epidemiol. 2011 Feb;64(2):163-71. (PMID: 20688472)
Crit Care Med. 2016 Jul;44(7):1278-84. (PMID: 26963326)
Stat Med. 2004 Oct 30;23(20):3105-24. (PMID: 15449338)
J Stat Softw. 2017 Aug;80:. (PMID: 28883783)
Lancet Oncol. 2019 Oct;20(10):e552. (PMID: 31578994)
Lancet Oncol. 2019 Aug;20(8):1065-1069. (PMID: 31296490)
Epidemiology. 2016 Jul;27(4):562-9. (PMID: 27007642)
J Clin Epidemiol. 2020 Mar;119:47-56. (PMID: 31783099)
J Clin Epidemiol. 1990;43(2):201-9. (PMID: 2303850)
PLoS One. 2018 Sep 13;13(9):e0204056. (PMID: 30212588)
Stat Methods Med Res. 2017 Oct;26(5):2227-2243. (PMID: 26220535)
JAMA. 2018 Apr 10;319(14):1429-1430. (PMID: 29566133)
JAMA Surg. 2019 Jan 1;154(1):74-79. (PMID: 30422256)
PLoS Med. 2020 Apr 3;17(4):e1003082. (PMID: 32243458)
Res Synth Methods. 2012 Dec;3(4):312-24. (PMID: 26053424)
BMJ. 2019 Oct 9;367:l5342. (PMID: 31597627)
J Clin Epidemiol. 2020 Mar;119:140-142. (PMID: 31711911)
BMC Med Res Methodol. 2015 Jul 31;15:58. (PMID: 26227148)
J Eval Clin Pract. 2020 Jun 10;:. (PMID: 32524768)
J Clin Epidemiol. 2019 Jul;111:32-40. (PMID: 30940600)
Ann Intern Med. 1999 Jun 15;130(12):995-1004. (PMID: 10383371)
Ann Intern Med. 1986 Sep;105(3):445-7. (PMID: 3740684)
JAMA. 2016 Mar 15;315(11):1141-8. (PMID: 26978209)
Eur Heart J. 2017 Feb 1;38(5):338-345. (PMID: 27742808)
J R Stat Soc Ser C Appl Stat. 2018 Jan;67(1):127-144. (PMID: 29540936)
J Clin Epidemiol. 1991;44(12):1373-8. (PMID: 1753268)
Front Psychol. 2017 Jan 04;7:2030. (PMID: 28101074)
Nat Hum Behav. 2018 Jan;2(1):6-10. (PMID: 30980045)
Clin Trials. 2014 Apr;11(2):246-62. (PMID: 24096635)

R01 LM012982 United States LM NLM NIH HHS; UL1 TR001427 United States TR NCATS NIH HHS

Keywords: Fragility index; Network meta-analysis; Replicability; Statistical significance; Systematic review; Treatment comparison

Date Created: 20200714 Date Completed: 20210305 Latest Revision: 20211102

20240829

PMC7669670

10.1016/j.jclinepi.2020.07.003

32659361