论文标题
Arabsign：连续阿拉伯语手语识别的多模式数据集和基准
ArabSign: A Multi-modality Dataset and Benchmark for Continuous Arabic Sign Language Recognition
论文作者
论文摘要
近年来，手语识别引起了研究人员的兴趣。尽管已经为欧洲和亚洲标志性语言识别提出了许多方法，但已经尝试开发阿拉伯语手语（ARSL）的类似系统的尝试非常有限。这可以部分归因于句子级别缺乏数据集。在本文中，我们旨在通过提出ArabSign（连续的ARSL数据集）做出重大贡献。提出的数据集由6个签名者执行的9,335个样本组成。记录句子的总时间约为10小时，平均句子的长度为3.1个符号。使用Kinect V2摄像机记录ArabSign数据集，该摄像头提供了每句话同时记录的三种类型的信息（颜色，深度和骨架关节点）。此外，我们根据ARSL和阿拉伯语结构提供数据集的注释，这些结构可以帮助研究ARSL的语言特征。为了基准该数据集，我们为连续ARSL识别提出了一个编码器模型。该模型已在提出的数据集上进行了评估，并且获得的结果表明，编码器模型的平均单词误差率（WER）优于注意机制为0.50，而注意机制则优于0.50。数据和代码可在github.com/hamzah-luqman/arabsign上获得
Sign language recognition has attracted the interest of researchers in recent years. While numerous approaches have been proposed for European and Asian sign languages recognition, very limited attempts have been made to develop similar systems for the Arabic sign language (ArSL). This can be attributed partly to the lack of a dataset at the sentence level. In this paper, we aim to make a significant contribution by proposing ArabSign, a continuous ArSL dataset. The proposed dataset consists of 9,335 samples performed by 6 signers. The total time of the recorded sentences is around 10 hours and the average sentence's length is 3.1 signs. ArabSign dataset was recorded using a Kinect V2 camera that provides three types of information (color, depth, and skeleton joint points) recorded simultaneously for each sentence. In addition, we provide the annotation of the dataset according to ArSL and Arabic language structures that can help in studying the linguistic characteristics of ArSL. To benchmark this dataset, we propose an encoder-decoder model for Continuous ArSL recognition. The model has been evaluated on the proposed dataset, and the obtained results show that the encoder-decoder model outperformed the attention mechanism with an average word error rate (WER) of 0.50 compared with 0.62 with the attention mechanism. The data and code are available at github.com/Hamzah-Luqman/ArabSign
