论文标题
野外的两际流听觉影响分析
Two-Stream Aural-Visual Affect Analysis in the Wild
论文作者
论文摘要
人类情感识别是自然人类相互作用的重要组成部分。但是，当前的方法仍处于起步阶段，尤其是对于野外数据。在这项工作中，我们将提交介绍给2020年野外情感行为分析（ABAW）。我们提出了一个两流听觉分析模型，以识别视频中的情感行为。音频和图像流首先分别处理并馈入卷积神经网络。而不是将经常性体系结构应用于时间分析，而仅使用时间卷积。此外，该模型可以访问面部对齐期间提取的其他功能。在培训时，我们利用不同情绪表征之间的相关性来提高性能。我们的模型在具有挑战性的AFF-WILD2数据库上取得了令人鼓舞的结果。
Human affect recognition is an essential part of natural human-computer interaction. However, current methods are still in their infancy, especially for in-the-wild data. In this work, we introduce our submission to the Affective Behavior Analysis in-the-wild (ABAW) 2020 competition. We propose a two-stream aural-visual analysis model to recognize affective behavior from videos. Audio and image streams are first processed separately and fed into a convolutional neural network. Instead of applying recurrent architectures for temporal analysis we only use temporal convolutions. Furthermore, the model is given access to additional features extracted during face-alignment. At training time, we exploit correlations between different emotion representations to improve performance. Our model achieves promising results on the challenging Aff-Wild2 database.
