One Novel Transfer Learning-based CLIP Model Combined With Self-attention Mechanism for Differentiating the Tumor-stroma Ratio in Pancreatic Ductal Adenocarcinoma: a Multi-center Retrospective Cohort Study

The proposed novel Contrastive Language-Image Pretraining-Adapter (CLIP-adapter) model, leveraging transfer learning, framing CLIP and a self-attention mechanism for predicting TSR in PDAC, in order to exhibit high performance in distinguishing low and high TSR PDAC in the test cohort. We speculated the CLIP-adapter model outperformed single-phase models, specifically CLIP models based on arterial or venous phase images alone. The addition of a feature fusion module could enhance the model's differentiation capacity, emphasizing its superiority over single-phase models. Besides, the model we designed utilized both image and text information during network training, instead of focusing on images only. This underscores the importance of comprehensive assessment in PDAC imaging evaluation, with the potential to contribute to risk stratification and personalized treatment planning.