论文标题
通过随机行走神经代表和自我划分的神经特征空间的涂鸦语义分割
Scribble-Supervised Semantic Segmentation by Random Walk on Neural Representation and Self-Supervision on Neural Eigenspace
论文作者
论文摘要
涂鸦监督的语义细分最近因其有希望的表现而没有高质量注释引起了人们的关注。已经提出了许多方法。通常，他们处理此问题，可以从另一个相关任务引入标记良好的数据集，以使用图形模型转到迭代精致和后处理，或者操纵涂鸦标签。这项工作旨在直接通过涂鸦标签监督的语义细分，而无需辅助信息和其他中间操作。具体而言，我们通过自我划分对神经特征空间的随机行走和一致性对神经表示扩散，这迫使神经网络在整个数据集中产生密集且一致的预测。网络中嵌入的随机步行将计算一个概率过渡矩阵，神经表示形式扩散为均匀。此外，鉴于概率过渡矩阵，我们在其本征空间上应用了自我诉讼，以使图像的主要部分保持一致性。除了比较常见的涂鸦数据集外，我们还对经过修改的数据集进行了实验，这些数据集随机收缩甚至将涂鸦放在图像对象上。结果证明了该方法的优越性，甚至与某些全标签监督的方法相媲美。该代码和数据集可在https://github.com/panzhiyi/rw-ss上找到。
Scribble-supervised semantic segmentation has gained much attention recently for its promising performance without high-quality annotations. Many approaches have been proposed. Typically, they handle this problem to either introduce a well-labeled dataset from another related task, turn to iterative refinement and post-processing with the graphical model, or manipulate the scribble label. This work aims to achieve semantic segmentation supervised by scribble label directly without auxiliary information and other intermediate manipulation. Specifically, we impose diffusion on neural representation by random walk and consistency on neural eigenspace by self-supervision, which forces the neural network to produce dense and consistent predictions over the whole dataset. The random walk embedded in the network will compute a probabilistic transition matrix, with which the neural representation diffused to be uniform. Moreover, given the probabilistic transition matrix, we apply the self-supervision on its eigenspace for consistency in the image's main parts. In addition to comparing the common scribble dataset, we also conduct experiments on the modified datasets that randomly shrink and even drop the scribbles on image objects. The results demonstrate the superiority of the proposed method and are even comparable to some full-label supervised ones. The code and datasets are available at https://github.com/panzhiyi/RW-SS.
