Silence is golden, but my measures still see-why cheaper-but-noisier outcome measures in large simple trials can be more cost-effective than gold standards.

Publisher: BioMed Central Country of Publication: England NLM ID: 101263253 Publication Model: Electronic Cited Medium: Internet ISSN: 1745-6215 (Electronic) Linking ISSN: 17456215 NLM ISO Abbreviation: Trials Subsets: MEDLINE

Original Publication: [London] : BioMed Central, 2006-

Cost-Benefit Analysis* ; Research Design*/standards ; Clinical Trials as Topic*/economics ; Clinical Trials as Topic*/standards; Humans ; Surveys and Questionnaires ; Reproducibility of Results ; Sample Size ; Treatment Outcome ; Models, Economic ; Endpoint Determination

Objective: To assess the cost-effectiveness of using cheaper-but-noisier outcome measures, such as a short questionnaire, for large simple clinical trials.
Background: To detect associations reliably, trials must avoid bias and random error. To reduce random error, we can increase the size of the trial and increase the accuracy of the outcome measurement process. However, with fixed resources, there is a trade-off between the number of participants a trial can enrol and the amount of information that can be collected on each participant during data collection.
Methods: To consider the effect on measurement error of using outcome scales with varying numbers of categories, we define and calculate the variance from categorisation that would be expected from using a category midpoint; define the analytic conditions under which such a measure is cost-effective; use meta-regression to estimate the impact of participant burden, defined as questionnaire length, on response rates; and develop an interactive web-app to allow researchers to explore the cost-effectiveness of using such a measure under plausible assumptions.
Results: An outcome scale with only a few categories greatly reduced the variance of non-measurement. For example, a scale with five categories reduced the variance of non-measurement by 96% for a uniform distribution. We show that a simple measure will be more cost-effective than a gold-standard measure if the relative increase in variance due to using it is less than the relative increase in cost from the gold standard, assuming it does not introduce bias in the measurement. We found an inverse power law relationship between participant burden and response rates such that a doubling the burden on participants reduces the response rate by around one third. Finally, we created an interactive web-app ( https://benjiwoolf.shinyapps.io/cheapbutnoisymeasures/ ) to allow exploration of when using a cheap-but-noisy measure will be more cost-effective using realistic parameters.
Conclusion: Cheaper-but-noisier questionnaires containing just a few questions can be a cost-effective way of maximising power. However, their use requires a judgement on the trade-off between the potential increase in risk of information bias and the reduction in the potential of selection bias due to the expected higher response rates.
(© 2024. The Author(s).)

BMJ. 2013 Feb 28;346:f1317. (PMID: 23449675)
Lancet. 2005 Jun 4-10;365(9475):1957-9. (PMID: 15936423)
Science. 1946 Jun 7;103(2684):677-80. (PMID: 20984256)
J Epidemiol Community Health. 1999 Jan;53(1):46-50. (PMID: 10326053)
Eur Respir J. 2017 May 11;49(5):. (PMID: 28495688)
Hum Mol Genet. 2014 Sep 15;23(R1):R89-98. (PMID: 25064373)
Int J Epidemiol. 2012 Oct;41(5):1383-93. (PMID: 23045203)
Epidemiol Rev. 2022 Jan 14;43(1):94-105. (PMID: 34664648)
Cochrane Database Syst Rev. 2023 Nov 30;11:MR000008. (PMID: 38032037)
Evid Based Ment Health. 2019 Nov;22(4):153-160. (PMID: 31563865)
Qual Health Care. 1994 Mar;3(1):11-6. (PMID: 10171955)
J Am Geriatr Soc. 1994 Sep;42(9):1006-8. (PMID: 8064088)
J Neurotrauma. 2002 Sep;19(9):999-1005. (PMID: 12482113)
Stat Med. 1987 Apr-May;6(3):233-44. (PMID: 3616281)
J Am Med Dir Assoc. 2010 Jun;11(5):356-64. (PMID: 20511103)
Lancet. 1997 May 31;349(9065):1569-81. (PMID: 9174558)
Disabil Rehabil. 2002 May 10;24(7):371-7. (PMID: 12022787)
Eur Heart J. 2014 Mar;35(9):549-51. (PMID: 24408889)
J Neurol Neurosurg Psychiatry. 2000 Oct;69(4):487-93. (PMID: 10990509)
Int J Epidemiol. 2020 Feb 1;49(1):338-347. (PMID: 31821469)
Nat Rev Genet. 2009 Dec;10(12):872-8. (PMID: 19859063)
Stat Med. 1984 Oct-Dec;3(4):409-22. (PMID: 6528136)

Keywords: Loss to follow-up; Measurement error; Outcome assessment; Questionnaire length; Questionnaires; Response rate; Sampling error

Date Created: 20240811 Date Completed: 20240811 Latest Revision: 20240814

20240814

PMC11318131

10.1186/s13063-024-08374-5

39128997