论文标题
替代梯度尖峰神经网络作为大型词汇连续语音识别的编码器
Surrogate Gradient Spiking Neural Networks as Encoders for Large Vocabulary Continuous Speech Recognition
论文作者
论文摘要
与传统的人工神经元相比，产生致密和实现的反应，生物学启发的尖峰神经元传递稀疏和二元信息，这也可以导致节能实施。最近的研究表明，使用替代梯度方法可以像标准复发性神经网络一样训练尖峰神经网络。他们在语音命令识别任务上显示出令人鼓舞的结果。使用相同的技术，我们表明它们可扩展到大型词汇连续的语音识别，在这些词汇中，它们能够替换编码器中的LSTM，而只有少量的性能损失。这表明它们可能适用于更多涉及的顺序到序列任务。此外，与它们反复出现的非尖刺对应物相比，它们在不需要使用门的情况下表现出强大的爆炸梯度问题。
Compared to conventional artificial neurons that produce dense and real-valued responses, biologically-inspired spiking neurons transmit sparse and binary information, which can also lead to energy-efficient implementations. Recent research has shown that spiking neural networks can be trained like standard recurrent neural networks using the surrogate gradient method. They have shown promising results on speech command recognition tasks. Using the same technique, we show that they are scalable to large vocabulary continuous speech recognition, where they are capable of replacing LSTMs in the encoder with only minor loss of performance. This suggests that they may be applicable to more involved sequence-to-sequence tasks. Moreover, in contrast to their recurrent non-spiking counterparts, they show robustness to exploding gradient problems without the need to use gates.
