论文标题
利用半监督时间动作细分的动作亲和力和连续性
Leveraging Action Affinity and Continuity for Semi-supervised Temporal Action Segmentation
论文作者
论文摘要
我们为时间动作细分任务提供了半监督的学习方法。该任务的目的是在长时间的未修剪程序视频中暂时检测和细分操作，其中只有一小部分视频被密集地标记，并且没有标记的大量视频。为此，我们为未标记的数据提出了两个新的损失函数：动作亲和力损失和动作连续性损失。动作亲和力损失通过施加从标记的集合引起的动作先验来指导未标记的样品学习。动作连续性损失强制执行动作的时间连续性，这也提供了框架分类的监督。此外，我们提出了一种自适应边界平滑（ABS）方法，以建立更粗的动作边界，以实现更健壮和可靠的学习。在三个基准上评估了拟议的损失函数和ABS。结果表明，它们以较低的标记数据（5％和10％）的数据显着改善了动作分割性能，并获得了与50％标记数据的全面监督相当的结果。此外，当将ABS整合到完全监督的学习中时，ABS成功地提高了性能。
We present a semi-supervised learning approach to the temporal action segmentation task. The goal of the task is to temporally detect and segment actions in long, untrimmed procedural videos, where only a small set of videos are densely labelled, and a large collection of videos are unlabelled. To this end, we propose two novel loss functions for the unlabelled data: an action affinity loss and an action continuity loss. The action affinity loss guides the unlabelled samples learning by imposing the action priors induced from the labelled set. Action continuity loss enforces the temporal continuity of actions, which also provides frame-wise classification supervision. In addition, we propose an Adaptive Boundary Smoothing (ABS) approach to build coarser action boundaries for more robust and reliable learning. The proposed loss functions and ABS were evaluated on three benchmarks. Results show that they significantly improved action segmentation performance with a low amount (5% and 10%) of labelled data and achieved comparable results to full supervision with 50% labelled data. Furthermore, ABS succeeded in boosting performance when integrated into fully-supervised learning.
