|
Neoadjuvant chemotherapy (NAC) followed by surgery is the standard treatment for breast cancer patients with pathologically confirmed lymph node (LN) metastasis, which enables 40-50% of patients to achieve axillary pathologic complete remission (pCR). However, owing to the lack of accurate methods for predicting the status of axillary LN (ALN) after NAC in clinical practice, axillary lymph node dissection (ALND) remains the conventional treatment approach following NAC, resulting in unnecessary overtreatment for patients who achieve pCR. B-mode ultrasound (BUS) images can provide morphological and echoic information about tissues, whereas shear wave elastography (SWE) images can reflect tissue heterogeneity, both of which are crucial for the diagnosis of ALN status. Therefore, it is necessary to exploit multimodal information fully to improve diagnostic accuracy. However, redundant information between BUS and SWE images poses a challenge in extracting modality-specific features and integrating them appropriately. We propose a multimodal deep learning network to predict the possibility of residual ALN metastasis in patients with biopsy-proven node-positive breast cancer after NAC by using BUS and SWE. Additionally, we incorporate clinicopathologic characteristics and conventional ultrasound features of ALN as supplementary information. To reduce the redundancy between different modalities and extract modality-specific features, we apply a multimodal orthogonal loss function to aid modal training. After extracting the modality-specific features, we introduce keyless attention for each modality to select the specific features related to our prediction task and fuse them in the prediction model. This attention mechanism calculates the individual attention scores based on the features from each modality independently, rather than considering the similarity between modalities. The experimental dataset used in this study is derived from the Sun Yat-sen University Cancer Center, and contains 236 cases with pathological examination results used as the ground truth. Our proposed network achieved an area under the receiver operating characteristics curve (AUC) of 0.9466 in the test cohort, which shows great potential for predicting ALN status after NAC. Because medical imaging plays an increasingly vital role in clinical diagnosis, our proposed method has the potential to serve as a valuable tool for effectively enhancing the accuracy of physician diagnosis.
|
