ItpCtrl-AI

ItpCtrl-AI: End-to-End Interpretable and Controllable Artificial Intelligence by Modeling Radiologists' Intentions

Publication

Published in Artificial Intelligence in Medicine
Volume 160, February 2025, 103054
DOI: https://doi.org/10.1016/j.artmed.2024.103054

Authors

• Trong-Thang Pham
• Jacob Brecheisen
• Carol C. Wu
• Hien Nguyen
• Zhigang Deng
• Donald Adjeroh
• Gianfranco Doretto
• Arabinda Choudhary
• Ngan Le

Highlights

• Novel controllable interpretable method for chest X-ray diagnosis
• Generation of gaze heatmaps to aid in diagnosis with user control capabilities
• Introduction of Diagnosed-Gaze++ dataset linking medical findings with eye gaze data
• Experimental validation of finding identification and gaze pattern reproduction

Abstract

Using Deep Learning in computer-aided diagnosis systems has been of great interest due to its impressive performance in the general domain and medical domain. However, a notable challenge is the lack of explainability of many advanced models, which poses risks in critical applications such as diagnosing findings in CXR. To address this problem, we propose ItpCtrl-AI, a novel end-to-end interpretable and controllable framework that mirrors the decision-making process of the radiologist.

Our framework:

• Emulates radiologists' eye gaze patterns
• Determines focal areas and pixel significance
• Generates attention heatmaps representing radiologist attention
• Extracts attended visual information for diagnosis
• Provides user control through directional input
• Ensures interpretability via eye gaze heatmap visualization

Dataset

The Diagnosed-Gaze++ dataset aligns medical findings with eye gaze data.

For access to the full dataset, please contact: [email protected]

Citation

@article{pham2025itpctrl,
  title={ItpCtrl-AI: End-to-end interpretable and controllable artificial intelligence by modeling radiologists' intentions},
  author={Pham, Trong-Thang and Brecheisen, Jacob and Wu, Carol C and Nguyen, Hien and Deng, Zhigang and Adjeroh, Donald and Doretto, Gianfranco and Choudhary, Arabinda and Le, Ngan},
  journal={Artificial Intelligence in Medicine},
  volume={160},
  pages={103054},
  year={2025},
  publisher={Elsevier},
  doi={10.1016/j.artmed.2024.103054}
}

License

MIT License

Copyright (c) 2024 AICV Lab, University of Arkansas