面向类脑计算的脉冲神经网络研究
摘 要
随着深度学习在训练成本、泛化能力、可解释性以及可靠性等方面的不足日益突出,类脑计算已成为下一代人工智能的研究热点。脉冲神经网络能更好地模拟生物神经元的信息传递方式,且具有计算能力强、功耗低等特点,在模拟人脑学习、记忆、推理、判断和决策等复杂信息方面具有重要的潜力。本文对脉冲神经网络从以下几个方面进行总结:首先阐述脉冲神经网络的基本结构和工作原理;在结构优化方面,从脉冲神经网络的编码方式、脉冲神经元改进、拓扑结构、训练算法以及结合其他算法这5个方面进行总结;在训练算法方面,从基于反向传播方法、基于脉冲时序依赖可塑性规则方法、人工神经网络转脉冲神经网络和其他学习算法这4个方面进行总结;针对脉冲神经网络的不足与发展,从监督学习和无监督学习两方面剖析;最后,将脉冲神经网络应用到类脑计算和仿生任务中。本文对脉冲神经网络的基本原理、编码方式、网络结构和训练算法进行了系统归纳,对脉冲神经网络的研究发展具有一定的积极意义。
关键词
Research on spiking neural networks for brain-inspired computing
Huo Bingqiang1, Gao Yanzhao2, Qi Xiaofeng2(1.Tianjin Binhai Information Technology Innovation Center, Tianjin 300450, China;2.Information Technology Research Institute, Information Engineering University, Zhengzhou 450000, China) Abstract
Human brain-like computing is focused on for next-generation artificial intelligence (AI) recently. Spiking neural network (SNN) has its potentials to simulate human brain-like complicated information in relevant to its learning, memory, reasoning, judgment, and decision-making. The higher computational and lower power-consumed SNN can be used to simulate the neurons-biological information transfer mode effectively. It is summarized as mentioned below:first, the brain-biological and event-driven SNN is regarded as a new generation of artificial neural network, which consists of input layer, coding mode, spiking neurons, synaptic weights, learning rules and output layer. SNN has its complication-biological and brain-like mechanism in terms of its spatio-temperal information. Second, its structural optimization is related to 5 aspects and highlighted in terms of 1) coding technique, 2) spiking neuron enhancement, 3) topology, 4) algorithm-trained, and 5) algorithm-coordinated. For the encoding method:1) grouping, 2) bursting, 3) frequency, and 4) temporal; For the spike neurons-related models:1) the Hodgkin-Huxley (H-H), 2) the integrated and fire (IF), 3) leaky integrate and fire (LIF), 4) spike response model (SRM), and 5) the Izhikevich between conventional models-relevant. In addition, the improved models for spike neurons are summarized on 8 aspects:for network topology:SNN has 3 conventional structures:1) feed-forward, 2) recurrent, and 3) hybird on the basis of 5 SNN-based popular topology models. For algorithm-trained, it is summarized from 4 aspects for these methods:1) back propagation (BP), 2) spike-timing dependent plasticity (STDP), 3) artificial neural network-to-spiking neural network(ANN-to-SNN),and 4) learning algorithms-relevant; Third, the pros and cons of SNN are segmented from the supervised and unsupervised learning both. Supervised learning:since spiking neurons are nonlinear and inconsistent, the error function between the actual output spike sequence and the desired output spike sequence of the neuron is challenged to meet the requirement of differentiability-consistent, resulting in differentiable transfer function and suppress itsback propagation. Unsupervised learning:the Hebb's rule and STDP algorithm are taken as the key elment in the field of brain-like computing, and it evolves into a variety of learning algorithms. This method is more consistent with the information transmission mode of neurons-biological. But, it is infeasible to build up a large-scale deep network model. Finally, SNNs are applied to brain-like computing and bionic tasks. A systematic SNN-based overview is clarified in the context of its 1) mechanisms, 2) encoding techniques, 3) network structure, and 4) algorithms-trained. Furthermore, to optimize the effectiveness of human brain-like computing model, we predict that the third generation of artificial intelligence is feasible to be realized through information transmission and multi-brain functional simulation and super-large-scale neural network topological and connection-strengthened information memorizing.
Keywords
brain-inspired computing spiking neural network(SNN) deep learning network structure training algorithm
|