Current Issue Cover
面向图像识别的多层脉冲神经网络学习算法综述

李雅馨1,2, 申江荣3, 徐齐1,2(1.大连理工大学人工智能学院, 大连 116024;2.人工智能与数字经济广东省实验室, 广州 510335;3.浙江大学计算机科学与技术学院, 杭州 310058)

摘 要
相较于第1代和第2代神经网络,第3代神经网络的脉冲神经网络是一种更加接近于生物神经网络的模型,因此更具有生物可解释性和低功耗性。基于脉冲神经元模型,脉冲神经网络可以通过脉冲信号的形式模拟生物信号在神经网络中的传播,通过脉冲神经元的膜电位变化来发放脉冲序列,脉冲序列通过时空联合表达不仅传递了空间信息还传递了时间信息。当前面向模式识别任务的脉冲神经网络模型性能还不及深度学习,其中一个重要原因在于脉冲神经网络的学习方法不成熟,深度学习中神经网络的人工神经元是基于实数形式的输出,这使得其可以使用全局性的反向传播算法对深度神经网络的参数进行训练,脉冲序列是二值性的离散输出,这直接导致对脉冲神经网络的训练存在一定困难,如何对脉冲神经网络进行高效训练是一个具有挑战的研究问题。本文首先总结了脉冲神经网络研究领域中的相关学习算法,然后对其中主要的方法:直接监督学习、无监督学习的算法以及ANN2SNN的转换算法进行分析介绍,并对其中代表性的工作进行对比分析,最后基于对当前主流方法的总结,对未来更高效、更仿生的脉冲神经网络参数学习方法进行展望。
关键词
A summary of image recognition-relevant multi-layer spiking neural networks learning algorithms

Li Yaxin1,2, Shen Jiangrong3, Xu Qi1,2(1.School of Artificial Intelligence, Dalian University of Technology, Dalian 116024, China;2.Guangdong Artificial Intelligence and Digital Economy Laboratory, Guangzhou 510335, China;3.College of Computer Science and Technology, Zhejiang University, Hangzhou 310058, China)

Abstract
To understand the structure of human brain further, Wolfgang Mass summarizes that the structures, training methods and some other crucial parts of spiking neural networks (SNNs) systematically, which are known as the third-generation of artificial neural networks. There are hundreds of millions of neurons and synaptic structures in related to human brain, but the requirement of energy is quite small. The SNNs has its advantages of biological interpretability and lower power consumption in comparison with the first and the second generation artificial neural networks (ANNs). Its neurons simulate the internal dynamics of biological neurons, and the weight-balanced simulates the construction, enhancement and inhibition rules of biological synapses. The SNN is mainly composed of such commonly-used spiking neuron models in relevant to Hodgkin Huxley (HH) model, leaky integrate-and-fire (LIF) model, and spiking response (SRM) model. The difference of ion concentration-inner and the biological neuron-outer can activate the potential of the cell membrane. To improve an action potential, channel-based ions move in and out of the neuron membrane in the neuron membrane when a neuron is stimulated. Spiking neuron model is a mathematical model that simulates the action potential process of biological neuron. A spiking neuron receives neurons-derived spiking stimulation in the upper layer. It will fire spikes to outreach a spiking train. The SNNs is focused on the transmission from spiking trains to information-targeted, which can simulate the propagation of biological signals in the biological neural network. The spiking trains can convey spatiotemporal information. However, current performance of SNNs-based pattern recognition tasks is still challenged to its immature deep learning methods. The artificial neurons of the neural network are based on the output in the form of real numbers, which makes it possible to use the global back-propagation algorithm to train the parameters of the deep neural network. But the spiking train is a sort of binary discrete output, which is still a challenging issue for SNN-based training. First, to clarify its current situation, our summary is focused on recent SNN-based learning algorithms. Then, to analyze pros and cons of popular works, the three main algorithms are introduced:1) supervised learning, 2) unsupervised learning, and 3) ANN-SNN conversion. The unsupervised learning algorithm is mainly based on the mechanism of spike timing dependent plasticity (STDP). The biological synapses-interconnected is enhanced or inhibited according to the relative timing of the firing of presynaptic neurons and postsynaptic neurons. Unsupervised learning methods have stronger biological interpretability, which can use the local optimization method to balance the synaptic weights, but this method is challenged for its complicated and large-scale network structures. Therefore, drawing on the advantages of ANN's easy calculation, some supervised algorithms have emerged like gradient-based training method and ANN2SNN method. The gradient-based learning algorithm is mainly concerned of the training idea of back-propagation (BP), which can balance the weight in terms of the error between the output value and the target value. This challenge is to be resolved in accordance with non-differentiable nature of discrete spikes. More methods of the BP-error have been proposed like gradient surrogate method. This gradient-based training method is focused on leveraging the training advantages of ANN and SNN. The training of SNN is interpretable biologically and easy to be computed. The ANN2SNN method can be used to convert the ANN weights-trained to SNN. This method can be used to realize the continuous activation values in the ANN into spiking trains. To reduce the conversion loss of ANN and SNN, this method is fine-tuned and converted according to neuron dynamics. This training method feature is indirect that it can apply SNN to complex network structures. The method of weight transfer can avoid direct training of SNN, which can apply SNN to complicated network structures. The ANN has been widely used in the field of image recognition. To extract more image features, ANN can be mainly used for consistent functions. SNN has its features of interpretability-biological and power consumption-lower, which can show its high performance in image recognition tasks. Finally, future SNNs-bionic learning methods are predicted in terms of some popular domain methods.
Keywords

订阅号|日报