CPKD: channel-wise and pixel-wise relational knowledge distillation for medical image segmentation

Ziyun Xiong; Xiangchun Yu

doi:10.1117/12.3035260

19 July 2024 CPKD: channel-wise and pixel-wise relational knowledge distillation for medical image segmentation

Ziyun Xiong, Xiangchun Yu

Proceedings Volume 13213, International Conference on Image Processing and Artificial Intelligence (ICIPAl 2024); 1321311 (2024) https://doi.org/10.1117/12.3035260
Event: International Conference on Image Processing and Artificial Intelligence (ICIPAl2024), 2024, Suzhou, China

Abstract

Deep neural networks have demonstrated remarkable performance in medical image segmentation tasks. Nevertheless, their efficacy often comes at the cost of high model complexity and sluggish inference speeds. To address these limitations, we explore a new channel-level and pixel-level relational knowledge distillation (CPKD) method to efficiently transfer the abundant relational knowledge from the teacher network to the light-weight student network. Concretely, CPKD first captures the inter-class and intra-class semantic information of the teacher network at each channel and pixel, and then distills it to the student network in the form of knowledge. Channel-wise relational distillation utilizes the correlation between channels, while pixel-wise relational distillation utilizes the correlation between pixels. These two forms of relational knowledge complement each other in the learning process, which enables the student network to better simulate the behavior of the teacher network. Extensive experimental results on the medical image segmentation task show that CPKD can significantly improve the performance of the student network compared to other knowledge distillation methods, is highly sensitive to edge information, especially in HD95 metrics our method is far superior to the comparison methods, and has smaller model complexity at the meantime.

(2024) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Ziyun Xiong and Xiangchun Yu "CPKD: channel-wise and pixel-wise relational knowledge distillation for medical image segmentation", Proc. SPIE 13213, International Conference on Image Processing and Artificial Intelligence (ICIPAl 2024), 1321311 (19 July 2024); https://doi.org/10.1117/12.3035260

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available