• 基于稀疏反馈的无梯度脉冲神经网络训练算法

    Gradient-free spiking neural network training algorithm based on sparse feedback

    • 脉冲神经网络(Spiking Neural Network, SNN)作为一种模拟生物神经系统运作机制的神经网络模型,在低功耗计算与高效硬件部署方面展现出显著潜力。为实现SNN的片上学习,需将其训练过程有效映射到硬件平台。目前主流方法为替代梯度训练算法,虽在软件上可取得良好性能,但依赖于基于链式法则的反向传播过程,在硬件实现中存在梯度计算复杂、资源消耗大的挑战。为此,本文提出一种基于稀疏直接反馈的无梯度SNN训练方法。该方法采用直接反馈对齐(Direct Feedback Alignment, DFA)传递输出误差,绕过反向传播中的链式求导,有效降低了计算复杂度与硬件资源需求。为进一步优化硬件实现,通过在反馈矩阵中引入稀疏结构,显著降低了数据传输量,并以非线性函数替代脉冲神经元的可微近似导数,避免显式梯度计算,使得算法在具有生物学合理性的同时保持良好的网络性能。在MNIST数据集上的实验表明,相较于传统DFA方法,所提算法在减少权重更新次数的同时,提升了网络训练精度。本研究为SNN的片上学习提供了一种高效的训练实现方案,有助于推动SNN在低功耗硬件平台上的实际应用。

       

      Abstract: As a neural network model that simulates the operational mechanisms of biological neural systems, Spiking Neural Networks (SNNs) demonstrate significant potential for low-power computing and efficient hardware deployment. To enable on-chip learning for SNNs, it is essential to effectively map their training process onto hardware platforms. The current mainstream approach, the surrogate gradient training algorithm, can achieve satisfactory performance in software. However, it relies on the chain rule-based backpropagation process, which presents challenges in hardware implementation, including complex gradient computation and high resource consumption. To address this issue, this paper proposes a gradient-free SNN training method based on sparse direct feedback. This method employs Direct Feedback Alignment (DFA) to propagate output errors, thereby bypassing the chain rule differentiation in backpropagation and effectively reducing computational complexity and hardware resource requirements. To further optimize hardware implementation, a sparse structure is introduced into the feedback matrix, significantly reducing data transmission volume. Additionally, nonlinear functions are used to replace the differentiable approximations of spiking neuron derivatives, eliminating explicit gradient calculations. This approach ensures both biological plausibility and strong network performance. Experimental results on the MNIST dataset show that compared to the conventional DFA method, the proposed algorithm improves network training accuracy while reducing the number of weight updates. This study provides an efficient training implementation solution for on-chip learning of SNNs, contributing to the practical application of SNNs on low-power hardware platforms.

       

    /

    返回文章
    返回