Yohanes Satria Nugroho, Chuan-Kai Yang and Yuan-Cheng Lai, National Taiwan University of Science and Technology, Taiwan

Sign Language Recognition is a variant of Action Recognition that consists of more detailed features, such as hand shapes and movements. Researchers have been trying to apply computer-based methods to tackle this task throughout the years. However, the methods proposed are constrained by hardware limitations, thus limiting them from being applied in real-life situations. In this research, we explore the possibilities of creating a lightweight Sign Language Recognition model so that it can be applied in real-life situations. We explore two different approaches. First, we extract keypoints and use a simple LSTM model to do the recognition and get 75% of Top-1 Validation Accuracy. We used the lightweight MoViNet A0 model for the second one and achieved 71% of Top-1 Test accuracy. Although these models achieved worse results than the state-of-the-art I3D, their complexity in terms of FLOPs is far better.

Sign Language Recognition, Lightweight Model, Keypoints Estimation