论文标题
ALRELU：泄漏的Relu激活功能的另一种方法以提高神经网络性能
ALReLU: A different approach on Leaky ReLU activation function to improve Neural Networks Performance
论文作者
论文摘要
尽管尚未解决“垂死的归因问题”，但经典的relu激活函数（AF）已广泛应用于深神经网络（DNN），特别是卷积神经网络（CNN）进行图像分类。 RELU的常见梯度问题在学院和行业的应用中构成了挑战。最新的改进方法仅仅提出了AF的变化，例如泄漏的Relu（LRELU），同时将解决方案保持在相同的未解决梯度问题中，朝着相似的方向朝着相似的方向朝着相似的方向发展。在本文中，提出了LRELU的一种绝对泄漏的Relu（Alrelu）AF，作为一种解决基于NN的基于NN的算法的替代方法，用于解决监督学习的常见“垂死relu问题”。实验结果表明，通过使用LRELU的小负梯度的绝对值，与LRELU和RELU相比，在五个不同数据集中的文本数据和表格数据分类任务上，与LRELU和RELU相比有了显着改善。
Despite the unresolved 'dying ReLU problem', the classical ReLU activation function (AF) has been extensively applied in Deep Neural Networks (DNN), in particular Convolutional Neural Networks (CNN), for image classification. The common gradient issues of ReLU pose challenges in applications on academy and industry sectors. Recent approaches for improvements are in a similar direction by just proposing variations of the AF, such as Leaky ReLU (LReLU), while maintaining the solution within the same unresolved gradient problems. In this paper, the Absolute Leaky ReLU (ALReLU) AF, a variation of LReLU, is proposed, as an alternative method to resolve the common 'dying ReLU problem' on NN-based algorithms for supervised learning. The experimental results demonstrate that by using the absolute values of LReLU's small negative gradient, has a significant improvement in comparison with LReLU and ReLU, on image classification of diseases such as COVID-19, text and tabular data classification tasks on five different datasets.
