论文标题
表情符号：使用Frege原理的上下文感知的多模式情感识别
EmotiCon: Context-Aware Multimodal Emotion Recognition using Frege's Principle
论文作者
论文摘要
我们提出表情符号，这是一种基于学习的算法，可从视频和图像中感知到感知的人类情感识别。由弗雷格（Frege）的情境原则的促进，我们的方法结合了情感识别的上下文的三种解释。我们的第一个解释是基于使用多种方式（例如面部和步态）进行情感识别。对于第二个解释，我们从输入图像中收集语义上下文，并使用基于自发的CNN编码此信息。最后，我们使用深度图来建模与社会互动相关的第三个解释和代理之间的邻近性。我们通过对基准数据集的情感实验来证明网络的效率。我们报告了26个类别的平均精度（AP）分数为35.48，比先前方法的提高了7-8。我们还介绍了一个新的数据集GroupWalk，该数据集是在多个人行走的现实世界中捕获的视频集合。我们在GroupWalk上的4个类别报告AP为65.83，这也是对先前方法的改进。
We present EmotiCon, a learning-based algorithm for context-aware perceived human emotion recognition from videos and images. Motivated by Frege's Context Principle from psychology, our approach combines three interpretations of context for emotion recognition. Our first interpretation is based on using multiple modalities(e.g. faces and gaits) for emotion recognition. For the second interpretation, we gather semantic context from the input image and use a self-attention-based CNN to encode this information. Finally, we use depth maps to model the third interpretation related to socio-dynamic interactions and proximity among agents. We demonstrate the efficiency of our network through experiments on EMOTIC, a benchmark dataset. We report an Average Precision (AP) score of 35.48 across 26 classes, which is an improvement of 7-8 over prior methods. We also introduce a new dataset, GroupWalk, which is a collection of videos captured in multiple real-world settings of people walking. We report an AP of 65.83 across 4 categories on GroupWalk, which is also an improvement over prior methods.
