Skip to yearly menu bar Skip to main content


Poster

DART: Disease-aware Image-Text Alignment and Self-correcting Re-alignment for Trustworthy Radiology Report Generation

Sang-Jun Park · Keun-Soo Heo · Dong-Hee Shin · Young-Han Son · Ji-Hye Oh · Tae-Eui Kam

[ ] [ Paper PDF ]
[ Poster
Sat 14 Jun 8:30 a.m. PDT — 10:30 a.m. PDT

Abstract:

The automatic generation of radiology reports has emerged as a promising solution to reduce a time-consuming task and accurately capture critical disease-relevant findings in X-ray images. Previous approaches for radiology report generation have shown impressive performance. However, there remains significant potential to improve accuracy by ensuring that retrieved reports contain disease-relevant findings similar to those in the X-ray images and by refining generated reports. In this study, we propose a Disease-aware image-text Alignment and self-correcting Re-alignment for Trustworthy radiology report generation (DART) framework. In the first stage, we generate initial reports based on image-to-text retrieval with disease-matching, embedding both images and texts in a shared embedding space through contrastive learning. This approach ensures the retrieval of reports with similar disease-relevant findings that closely align with the input X-ray images. In the second stage, we further enhance the initial reports by introducing a self-correction module that re-aligns them with the X-ray images. Our proposed framework achieves state-of-the-art results on the MIMIC-CXR and IU X-ray benchmarks, surpassing previous approaches in both report generation and disease classification, thereby enhancing the trustworthiness of radiology reports.

Chat is not available.