Read more: Evidation Collaborates with Merck on Digital Monitoring Study on Alzheimer’s Disease

Machine learning algorithms designed to characterize, monitor, and intervene on human health (ML4H) are expected to perform safely and reliably when operating at scale, potentially outside strict human supervision. This requirement warrants a stricter attention to issues of reproducibility than other fields of machine learning. In this work, we conduct a systematic evaluation of over 100 recently published ML4H research papers along several dimensions related to reproducibility. We find that the field of ML4H compares poorly to more established machine learning fields, particularly concerning data and code accessibility. Finally, drawing from success in other fields of science, we propose recommendations to data providers, academic publishers, and the ML4H research community in order to promote reproducible research moving forward.

Figure 1
Fraction of papers satisfying certain conditions by ML field (see ICLR 2019 link for full paper and methods):

Source: ICLR 2019

Further reading

Aug 08 2019

Chan R, Jankovic F, Marinsek N, Foschini L, Kourtis L, Signorini A, Pugh M, Shen J, Yaari R, Maljkovic V, Sunga M, Hee Song H, Joon Jung H, Tseng B, Trister A

Source: KDD 2019

Jun 06 2019

Bakker JP, Goldsack JC, Clarke M, Coravos A, Geoghegan C, Godfrey A, Heasley MG, Karlin DR, Manta C, Peterson B, Ramirez E, Sheth N, Bruno A, Bullis E, Wareham K, Zimmerman N, Forrest A, Wood WA

Source: NPJ Digital Medicine