论文标题
利用语音处理的插图来利用基于Riemannian梯度下降的低级张量深度神经网络
Exploiting Low-Rank Tensor-Train Deep Neural Networks Based on Riemannian Gradient Descent With Illustrations of Speech Processing
论文作者
论文摘要
这项工作着重于设计低复杂性混合张量网络，通过考虑模型复杂性和实际性能之间的权衡。首先，我们利用低级张量训练的深神经网络（TT-DNN）来构建端到端深度学习管道，即LR-TT-DNN。其次，将LR-TT-DNN与卷积神经网络（CNN）相结合的混合模型（表示为CNN+（LR-TT-DNN）），以提高性能。我们利用Riemannian梯度下降来确定与小型TT级别相关的TT-DNN。此外，CNN+（LR-TT-DNN）由底部的卷积层组成，用于特征提取，顶部的几个TT层以解决回归和分类问题。我们分别评估了语音增强和口语命令识别任务的LR-TT-DNN和CNN+（LR-TT-DNN）模型。我们的经验证据表明，具有较少模型参数的LR-TT-DNN和CNN+（LR-TT-DNN）模型可以优于TT-DNN和CNN+（TT-DNN）对应物。
This work focuses on designing low complexity hybrid tensor networks by considering trade-offs between the model complexity and practical performance. Firstly, we exploit a low-rank tensor-train deep neural network (TT-DNN) to build an end-to-end deep learning pipeline, namely LR-TT-DNN. Secondly, a hybrid model combining LR-TT-DNN with a convolutional neural network (CNN), which is denoted as CNN+(LR-TT-DNN), is set up to boost the performance. Instead of randomly assigning large TT-ranks for TT-DNN, we leverage Riemannian gradient descent to determine a TT-DNN associated with small TT-ranks. Furthermore, CNN+(LR-TT-DNN) consists of convolutional layers at the bottom for feature extraction and several TT layers at the top to solve regression and classification problems. We separately assess the LR-TT-DNN and CNN+(LR-TT-DNN) models on speech enhancement and spoken command recognition tasks. Our empirical evidence demonstrates that the LR-TT-DNN and CNN+(LR-TT-DNN) models with fewer model parameters can outperform the TT-DNN and CNN+(TT-DNN) counterparts.
