论文标题
跨模式变压器gan：大脑结构功能的深层融合框架，用于阿尔茨海默氏病
Cross-Modal Transformer GAN: A Brain Structure-Function Deep Fusing Framework for Alzheimer's Disease
论文作者
论文摘要
不同类型的神经影像数据的跨模式融合显示了预测阿尔茨海默氏病（AD）进展的巨大希望。但是，在神经影像中应用的大多数现有方法无法有效地融合来自多模式神经图像的功能和结构信息。在这项工作中，提出了一种新型的跨模式变压器生成对抗网络（CT-GAN），以融合静止状态功能磁共振成像（RS-FMRI）中包含的功能信息，并包含在扩散张量张量成像（DTI）中包含的结构信息。开发的双重注意机制可以有效地匹配功能信息，并最大程度地提高从RS-FMRI和DTI提取互补信息的能力。通过捕获结构特征和功能特征之间的深度互补信息，提出的CT-GAN可以检测到与AD相关的大脑连接性，该连通性可以用作AD的生物标志物。实验结果表明，所提出的模型不仅可以提高分类性能，还可以有效地检测与广告相关的大脑连接性。
Cross-modal fusion of different types of neuroimaging data has shown great promise for predicting the progression of Alzheimer's Disease(AD). However, most existing methods applied in neuroimaging can not efficiently fuse the functional and structural information from multi-modal neuroimages. In this work, a novel cross-modal transformer generative adversarial network(CT-GAN) is proposed to fuse functional information contained in resting-state functional magnetic resonance imaging (rs-fMRI) and structural information contained in Diffusion Tensor Imaging (DTI). The developed bi-attention mechanism can match functional information to structural information efficiently and maximize the capability of extracting complementary information from rs-fMRI and DTI. By capturing the deep complementary information between structural features and functional features, the proposed CT-GAN can detect the AD-related brain connectivity, which could be used as a bio-marker of AD. Experimental results show that the proposed model can not only improve classification performance but also detect the AD-related brain connectivity effectively.
