论文标题
多目标优化确定何时，如何融合深层网络：预测COVID-19结果的应用
Multi-objective optimization determines when, which and how to fuse deep networks: an application to predict COVID-19 outcomes
论文作者
论文摘要
Covid-19的大流行造成了数百万例和死亡，而与AI相关的科学界参与了医学图像中的COVID-19迹象后，现在一直在指导努力开发可以预测疾病进展的方法。这项任务本质上是多模式的，最近在公开可用的Aiforcovid数据集上取得的基线结果表明，胸部X射线扫描和临床信息对于识别有严重结果风险的患者很有用。尽管深度学习在几个医学领域表现出卓越的表现，但在大多数情况下，它仅考虑单峰数据。在这方面，何时，如何融合不同的方式是多模式深度学习的开放挑战。为了解决这三个问题，我们提出了一种新颖的方法，以优化多模式端到端模型的设置。它利用了帕累托多目标优化，可融合性能指标和多个候选单形式神经网络的多样性评分。我们在Aiforcovid数据集上测试我们的方法，取得最新的结果，不仅表现优于基线性能，而且对外部验证也很强。此外，利用XAI算法，我们找出了模式之间的层次结构，并提取了功能内模式的重要性，从而丰富了对模型预测的信任。
The COVID-19 pandemic has caused millions of cases and deaths and the AI-related scientific community, after being involved with detecting COVID-19 signs in medical images, has been now directing the efforts towards the development of methods that can predict the progression of the disease. This task is multimodal by its very nature and, recently, baseline results achieved on the publicly available AIforCOVID dataset have shown that chest X-ray scans and clinical information are useful to identify patients at risk of severe outcomes. While deep learning has shown superior performance in several medical fields, in most of the cases it considers unimodal data only. In this respect, when, which and how to fuse the different modalities is an open challenge in multimodal deep learning. To cope with these three questions here we present a novel approach optimizing the setup of a multimodal end-to-end model. It exploits Pareto multi-objective optimization working with a performance metric and the diversity score of multiple candidate unimodal neural networks to be fused. We test our method on the AIforCOVID dataset, attaining state-of-the-art results, not only outperforming the baseline performance but also being robust to external validation. Moreover, exploiting XAI algorithms we figure out a hierarchy among the modalities and we extract the features' intra-modality importance, enriching the trust on the predictions made by the model.
