방사선의학 웹진

바로가기  >

Abstract
Target volume contouring for radiation therapy is considered significantly more challenging than the normal organ segmentation tasks as it necessitates the utilization of both image and text-based clinical information. Inspired by the recent advancement of large language models (LLMs) that can facilitate the integration of the textural information and images, here we present an LLM-driven multimodal artificial intelligence (AI), namely LLMSeg, that utilizes the clinical information and is applicable to the challenging task of 3-dimensional context-aware target volume delineation for radiation oncology. We validate our proposed LLMSeg within the context of breast cancer radiotherapy using external validation and data-insufficient environments, which attributes highly conducive to real-world applications. We demonstrate that the proposed multimodal LLMSeg exhibits markedly improved performance compared to conventional unimodal AI models, particularly exhibiting robust generalization performance and data-efficiency.

Affiliations

Yujin Oh # 1, Sangjoon Park # 2 3, Hwa Kyung Byun 4, Yeona Cho 5, Ik Jae Lee 2, Jin Sung Kim 6 7, Jong Chul Ye 8
1Department of Radiology, Massachusetts General Hospital (MGH) and Harvard Medical School, Boston, MA, USA.
2Department of Radiation Oncology, Yonsei University College of Medicine, Seoul, South Korea.
3Institute for Innovation in Digital Healthcare, Yonsei University, Seoul, South Korea.
4Department of Radiation Oncology, Yongin Severance Hospital, Yongin, Gyeonggi-do, South Korea.
5Department of Radiation Oncology, Gangnam Severance Hospital, Seoul, South Korea.
6Department of Radiation Oncology, Yonsei University College of Medicine, Seoul, South Korea. jinsung@yuhs.ac.
7Oncosoft Inc., Seoul, South Korea. jinsung@yuhs.ac.
8Kim Jaechul Graduate School of AI, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, South Korea. jong.ye@kaist.ac.kr.
#Contributed equally.