论文标题
在多模式分析中分析模态鲁棒性
Analyzing Modality Robustness in Multimodal Sentiment Analysis
论文作者
论文摘要
构建强大的多模型模型对于在野外实现可靠的部署至关重要。尽管它很重要，但对识别和改善多模式分析（MSA）模型的鲁棒性的关注减少了。在这项工作中，我们希望通过（i）在训练有素的多模式模型中提出简单的诊断检查模式鲁棒性。使用这些检查，我们发现MSA模型对单个模式高度敏感，这会在其稳健性中产生问题。 （ii）我们分析了众所周知的强大培训策略，以减轻这些问题。至关重要的是，我们观察到可以实现鲁棒性，而不会损害原始性能。我们希望我们在五个模型和两个基准数据集和提议的程序中进行广泛的研究表现，这将使鲁棒性成为MSA研究中不可或缺的组成部分。我们的诊断检查和强大的培训解决方案易于实现，并在https：// github上获得。 com/necrare-lab/msa-obustness。
Building robust multimodal models are crucial for achieving reliable deployment in the wild. Despite its importance, less attention has been paid to identifying and improving the robustness of Multimodal Sentiment Analysis (MSA) models. In this work, we hope to address that by (i) Proposing simple diagnostic checks for modality robustness in a trained multimodal model. Using these checks, we find MSA models to be highly sensitive to a single modality, which creates issues in their robustness; (ii) We analyze well-known robust training strategies to alleviate the issues. Critically, we observe that robustness can be achieved without compromising on the original performance. We hope our extensive study-performed across five models and two benchmark datasets-and proposed procedures would make robustness an integral component in MSA research. Our diagnostic checks and robust training solutions are simple to implement and available at https://github. com/declare-lab/MSA-Robustness.
