Performance of four computer-coded verbal autopsy methods for cause of death assignment compared with physician coding on 24,000 deaths in low- and middle-income countries
Centre for Global Heath Research, St. Michael's Hospital, Dalla Lana School of Public Health, University of Toronto, Toronto ON, Canada; Center for the Promotion of Research Involving Innovative Statistical Methodology, Steinhardt School of Culture, Education and Human Development, New York University, New York NY, United States; WHO Collaborating Centre for Verbal Autopsy, Umeå Centre for Global Health Research, Umeå University, Umeå, Sweden; Umeå Centre for Global Health Research, Division of Epidemiology and Global Health, Department of Public Health and Clinical Medicine, Umeå University, Umeå, Sweden; Medical Research Council, Wits Univ. Rural Public Health and Health Transitions Research Unit, School of Public Health, Faculty of Health Sciences, University of the Witwatersrand, Johannesburg, South Africa; International Network for the Demographic Evaluation of Populations, Their Health (INDEPTH) Network, Accra, Ghana; International Centre for Diarrhoeal Disease Research, Bangladesh (ICDDR, B), Dhaka, Bangladesh; International Institute for Population Sciences, Mumbai, Maharashtra, India; School of Public Health, Post Graduate Institute of Medical Research and Education, Chandigarh, India
Background: Physician-coded verbal autopsy (PCVA) is the most widely used method to determine causes of death (CODs) in countries where medical certification of death is uncommon. Computer-coded verbal autopsy (CCVA) methods have been proposed as a faster and cheaper alternative to PCVA, though they have not been widely compared to PCVA or to each other.Methods: We compared the performance of open-source random forest, open-source tariff method, InterVA-4, and the King-Lu method to PCVA on five datasets comprising over 24,000 verbal autopsies from low- and middle-income countries. Metrics to assess performance were positive predictive value and partial chance-corrected concordance at the individual level, and cause-specific mortality fraction accuracy and cause-specific mortality fraction error at the population level.Results: The positive predictive value for the most probable COD predicted by the four CCVA methods averaged about 43% to 44% across the datasets. The average positive predictive value improved for the top three most probable CODs, with greater improvements for open-source random forest (69%) and open-source tariff method (68%) than for InterVA-4 (62%). The average partial chance-corrected concordance for the most probable COD predicted by the open-source random forest, open-source tariff method and InterVA-4 were 41%, 40% and 41%, respectively, with better results for the top three most probable CODs. Performance generally improved with larger datasets. At the population level, the King-Lu method had the highest average cause-specific mortality fraction accuracy across all five datasets (91%), followed by InterVA-4 (72% across three datasets), open-source random forest (71%) and open-source tariff method (54%).Conclusions: On an individual level, no single method was able to replicate the physician assignment of COD more than about half the time. At the population level, the King-Lu method was the best method to estimate cause-specific mortality fractions, though it does not assign individual CODs. Future testing should focus on combining different computer-coded verbal autopsy tools, paired with PCVA strengths. This includes using open-source tools applied to larger and varied datasets (especially those including a random sample of deaths drawn from the population), so as to establish the performance for age- and sex-specific CODs. © 2014 Desai et al.; licensee BioMed Central Ltd.
adolescent; adult; aged; article; autopsy; cause of death; child; computer coded verbal autopsy; computer program; controlled study; death certificate; human; infant; intermethod comparison; interVA 4 method; king lu method; lowest income group; measurement accuracy; middle aged; mortality; newborn; physician; physician coded verbal autopsy; predictive value; preschool child; procedures, parameters and devices; random forest; school child; Tariff method; task performance; very elderly; young adult; Automatic Data Processing; Autopsy; Cause of Death; Databases, Factual; Humans; Physician's Role; Poverty