Abstract:
To address the issue of insufficient hand action recognition accuracy caused by dynamic illumination interference, complex backgrounds, hand movement, and partial occlusion in intelligent industrial scenarios, this paper proposes a fused architecture termed MSS-YOLO-MPHands. The scheme first improves YOLOv8 by constructing the MSS-YOLO model to optimize the skeletal keypoint detection performance of MediaPipe Hands. Meanwhile, the MobileNetV4-UIB module is adopted to reconstruct the original C2f structure, substantially enhancing the model's robustness against common occlusion and illumination variations in industrial scenes. To further boost detection performance, the STA (Spatial-Temporal Attention) mechanism and the SPConv module are introduced into the backbone network. Finally, a Bi-LSTM-based temporal modeling method is incorporated to build an end-to-end action recognition framework. Experimental results demonstrate that the MSS-YOLO-MPHands model achieves a PCK (Percentage of Correct Keypoints) of 93.85% and an average hand action recognition accuracy of 92.5%, validating the effectiveness and feasibility of the proposed scheme in complex industrial environments.