Abstract:
Aiming at the problems of high power consumption and difficulty in low-power deployment of traditional convolutional neural network (CNN)-based pose estimation methods when processing high-frequency event data, a pose estimation method based on a hybrid spiking neural network (SNN) is proposed. A fast-slow dual-branch network architecture, termed FLPNet, is designed, in which a low-frequency lightweight artificial neural network (ANN) branch extracts features from event frames, while a high-frequency SNN branch is continuously driven by a cross-timestep state mechanism to evolve membrane potentials, thereby enhancing the model's ability to perceive and respond to temporal features. To address the discrepancy in temporal modeling resolution between the ANN and SNN branches, periodic state injection and an event-density-based weighted fusion strategy are introduced to achieve stable firing control within short timesteps and to suppress spurious and missed firing caused by abrupt membrane potential drops. Experimental results on the DHP19 dataset show that the proposed method achieves a mean per joint position error (MPJPE) of 5.98 cm with an average power consumption of 0.27 W, achieving about twice the energy efficiency of the U-Net+SNN hybrid model proposed by Aydin et al., with only about 7% accuracy loss, while exhibiting faster convergence and higher prediction stability in the early stages of training and inference. Generalization experiments on the H3.6m and THU-EACT-50 datasets further verify the effectiveness of the proposed architecture on non-event datasets and event-based action datasets. The proposed method significantly reduces power consumption while maintaining high pose estimation accuracy, providing an effective technical approach for neuromorphic vision and low-power human pose estimation.