论文标题
从头开始学习卷积
Towards Learning Convolutions from Scratch
论文作者
论文摘要
卷积是计算机视觉中使用的架构最重要的组成部分之一。随着机器学习朝着减少专家偏见和从数据中学习的发展，下一步似乎是从头开始学习类似卷积的结构。但是，这证明这是难以捉摸的。例如，当前的最新体系结构搜索算法使用卷积作为现有模块之一，而不是从数据中学习。为了了解引起卷积的归纳偏见，我们将最小描述长度作为指导原则，并表明在某些情况下，它确实可以表明建筑的性能。 To find architectures with small description length, we propose $β$-LASSO, a simple variant of LASSO algorithm that, when applied on fully-connected networks for image classification tasks, learns architectures with local connections and achieves state-of-the-art accuracies for training fully-connected nets on CIFAR-10 (85.19%), CIFAR-100 (59.56%) and SVHN （94.07％）弥合了完全连接和卷积网之间的差距。
Convolution is one of the most essential components of architectures used in computer vision. As machine learning moves towards reducing the expert bias and learning it from data, a natural next step seems to be learning convolution-like structures from scratch. This, however, has proven elusive. For example, current state-of-the-art architecture search algorithms use convolution as one of the existing modules rather than learning it from data. In an attempt to understand the inductive bias that gives rise to convolutions, we investigate minimum description length as a guiding principle and show that in some settings, it can indeed be indicative of the performance of architectures. To find architectures with small description length, we propose $β$-LASSO, a simple variant of LASSO algorithm that, when applied on fully-connected networks for image classification tasks, learns architectures with local connections and achieves state-of-the-art accuracies for training fully-connected nets on CIFAR-10 (85.19%), CIFAR-100 (59.56%) and SVHN (94.07%) bridging the gap between fully-connected and convolutional nets.
