• 基于混合脉冲神经网络的姿态估计研究

    Research on pose estimation based on hybrid spiking neural networks

    • 针对传统卷积神经网络姿态估计方法在处理高频事件数据时功耗较高、难以适应低功耗部署的问题,提出一种基于混合脉冲神经网络的姿态估计方法。该方法设计快慢双分支网络架构FLPNet,以低频轻量级人工神经网络分支提取事件帧特征,并通过跨时间步状态机制持续驱动高频脉冲神经网络分支的膜电位演化,增强模型对时间序列特征的感知与响应能力;针对人工神经网络与脉冲神经网络在时间建模分辨率上的不一致问题,引入周期性状态注入与基于事件密度的加权融合策略,实现短时间步下的稳定发放控制,抑制因膜电位骤降导致的误发放与漏发放现象。在DHP19数据集上的实验结果表明:所提方法平均关节位置误差达到5.98 cm,平均功耗为0.27 W,较Aydin等提出的U-Net+SNN混合模型具有约2倍的能耗优势,精度损失仅约7%,同时在训练与推理初期表现出更快的收敛速度和更高的预测稳定性。在H3.6m与THU-EACT-50数据集上的泛化实验进一步验证了该架构在非事件数据集与事件动作数据集上的有效性。该方法在保持较高姿态估计精度的同时显著降低功耗,可为神经形态视觉与低功耗人体姿态估计提供有效技术途径。

       

      Abstract: Aiming at the problems of high power consumption and difficulty in low-power deployment of traditional convolutional neural network (CNN)-based pose estimation methods when processing high-frequency event data, a pose estimation method based on a hybrid spiking neural network (SNN) is proposed. A fast-slow dual-branch network architecture, termed FLPNet, is designed, in which a low-frequency lightweight artificial neural network (ANN) branch extracts features from event frames, while a high-frequency SNN branch is continuously driven by a cross-timestep state mechanism to evolve membrane potentials, thereby enhancing the model's ability to perceive and respond to temporal features. To address the discrepancy in temporal modeling resolution between the ANN and SNN branches, periodic state injection and an event-density-based weighted fusion strategy are introduced to achieve stable firing control within short timesteps and to suppress spurious and missed firing caused by abrupt membrane potential drops. Experimental results on the DHP19 dataset show that the proposed method achieves a mean per joint position error (MPJPE) of 5.98 cm with an average power consumption of 0.27 W, achieving about twice the energy efficiency of the U-Net+SNN hybrid model proposed by Aydin et al., with only about 7% accuracy loss, while exhibiting faster convergence and higher prediction stability in the early stages of training and inference. Generalization experiments on the H3.6m and THU-EACT-50 datasets further verify the effectiveness of the proposed architecture on non-event datasets and event-based action datasets. The proposed method significantly reduces power consumption while maintaining high pose estimation accuracy, providing an effective technical approach for neuromorphic vision and low-power human pose estimation.

       

    /

    返回文章
    返回