利用TCGA-BRCA 公共数据集中乳腺癌患者的数字病理全切片图像(WSI) 数据和相对应的病理报告文本信息，搭建乳腺癌预后相关的预测模型，以期为临床决策提供可靠的参考。

本研究使用癌症基因组图谱（TCGA）中乳腺癌数据集（TCGA-BRCA），获得702例具有完整全切片病理图像（WSI）、对应病理报告文本和生存数据的样本。针对WSI高分辨率特性，采用形态学方法对组织区域进行分割，切割成图像块，构建多实例特征包；病理报告借助大语言模型（LLM）进行解析与结构化，提取关键的语义特征。在此基础上构建基于Transformer的多模态生存分析框架：图像分支通过改进的多实例学习和注意力机制兼顾全局与局部特征；文本分支将病理报告编码为语义嵌入。通过交叉注意力机制实现文本与图像特征的深度融合，最后经过全连接层输出为患者级生存风险分数，采用离散时间生存分析的负对数似然损失进行端到端训练。

建立基于数字病理全切片图像（WSI）的单模态生存预测模型，采用多实例学习框架输出患者的复发风险评分与生存概率。通过5折交叉验证，模型的C指数达0.687，其性能显著优于以往文献报道的结果。进一步融合临床病理文本信息后，多模态模型的预测性能进一步提升：C指数提高至0.698，标准差由±0.046降至±0.022，降幅约50%，模型的稳定性与可靠性显著增强。

本研究通过开发并验证一种新型多模态融合模型，可为乳腺癌患者的生存预测提供更为精准、可靠的解决方案。

To construct a high-accuracy, interpretable prognostic prediction model for breast cancer by integrating whole slide images (WSIs) and corresponding pathology report text from patients in the TCGA-BRCA public dataset, thereby providing a reliable auxiliary decision-making tool for clinical practice.

This study utilized The Cancer Genome Atlas Breast Cancer dataset (TCGA-BRCA). A total of 702 samples with complete WSIs, matched pathology reports, and survival data were obtained through stringent filtering. To handle the high resolution of WSIs, morphological methods were employed to segment tissue regions and partition them into image patches, constructing multi-instance feature bags. Pathology reports were parsed and structured using a large language model (LLM) to extract key semantic features. For model construction, a Transformer-based multimodal survival analysis framework was proposed: the image branch aggregated global and local features via improved multi-instance learning with an attention mechanism; the text branch encoded pathology reports into semantic embeddings. Deep fusion of textual and imaging features was achieved through a cross-attention mechanism. Finally, the output from the fully connected layer is a patient-level survival risk score, and the model is trained end-to-end using negative log-likelihood loss based on discrete-time survival analysis.

This study first developed a unimodal survival prediction model based solely on digital pathology whole-slide images (WSIs), employing a multiple instance learning framework to generate patients' recurrence risk scores and survival probabilities. Through 5-fold cross-validation, the model achieved a C-index of 0.687, significantly outperforming previously reported results in the literature. Further integration of clinical and pathological text information further improved the performance of the multimodal model: the C-index increased to 0.698, and the standard deviation decreased from ±0.046 to ±0.022—a reduction of approximately 50%—indicating a significant enhancement in model stability and reliability.

By developing and validating a novel multimodal fusion model, this study provides a more accurate and reliable solution for survival prediction in breast cancer patients.