Automated Generation of Accurate & Fluent Medical X-ray Reports

Hoang Nguyen, Dong Nie, Taivanbat Badamdorj, Yujie Liu, Yingying Zhu, Jason Truong, Li Cheng


Abstract
Our paper aims to automate the generation of medical reports from chest X-ray image inputs, a critical yet time-consuming task for radiologists. Existing medical report generation efforts emphasize producing human-readable reports, yet the generated text may not be well aligned to the clinical facts. Our generated medical reports, on the other hand, are fluent and, more importantly, clinically accurate. This is achieved by our fully differentiable and end-to-end paradigm that contains three complementary modules: taking the chest X-ray images and clinical history document of patients as inputs, our classification module produces an internal checklist of disease-related topics, referred to as enriched disease embedding; the embedding representation is then passed to our transformer-based generator, to produce the medical report; meanwhile, our generator also creates a weighted embedding representation, which is fed to our interpreter to ensure consistency with respect to disease-related topics. Empirical evaluations demonstrate very promising results achieved by our approach on commonly-used metrics concerning language fluency and clinical accuracy. Moreover, noticeable performance gains are consistently observed when additional input information is available, such as the clinical document and extra scans from different views.
Anthology ID:
2021.emnlp-main.288
Volume:
Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing
Month:
November
Year:
2021
Address:
Online and Punta Cana, Dominican Republic
Editors:
Marie-Francine Moens, Xuanjing Huang, Lucia Specia, Scott Wen-tau Yih
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
3552–3569
Language:
URL:
https://aclanthology.org/2021.emnlp-main.288
DOI:
10.18653/v1/2021.emnlp-main.288
Bibkey:
Cite (ACL):
Hoang Nguyen, Dong Nie, Taivanbat Badamdorj, Yujie Liu, Yingying Zhu, Jason Truong, and Li Cheng. 2021. Automated Generation of Accurate & Fluent Medical X-ray Reports. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 3552–3569, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Cite (Informal):
Automated Generation of Accurate & Fluent Medical X-ray Reports (Nguyen et al., EMNLP 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.emnlp-main.288.pdf
Video:
 https://aclanthology.org/2021.emnlp-main.288.mp4
Code
 ginobilinie/xray_report_generation
Data
CheXpert