论文标题
TCAM：在弱标记的无约束视频中的物体定位的时间类激活图
TCAM: Temporal Class Activation Maps for Object Localization in Weakly-Labeled Unconstrained Videos
论文作者
论文摘要
弱监督的视频对象本地化（WSVOL）允许仅使用全局视频标签（例如对象类）在视频中找到对象。最先进的方法依赖于多个独立阶段，其中最初的时空建议是使用视觉和运动提示生成的，然后确定和完善了突出的对象。本地化是通过在一个或多个视频上解决优化问题来完成的，并且视频标签通常用于视频聚类。这需要每次视频模型或每类制造代价高昂的推断。此外，由于无监督的运动方法（如光流）或视频标签是从优化中丢弃的，因此本地化区域不是必需的判别。在本文中，我们利用成功的类激活映射（CAM）方法，该方法是基于静止图像而设计的。引入了一种新的时间凸轮（TCAM）方法，以训练一种判别深度学习（DL）模型，以利用称为CAM-temporal Max Max Pooling（CAM-TMP）的聚集机制在视频中利用时空信息，而不是连续的凸轮。特别是，从验证的CNN分类器生成的CAM中收集了感兴趣区域（ROI）的激活（ROI），以构建Pseudo-Labels构建用于训练DL模型的伪标签。此外，使用全局无监督的尺寸约束和诸如CRF之类的局部约束来产生更准确的凸轮。对单个独立帧的推断允许并行处理框架夹和实时定位。在两个挑战性的YouTube-Objects数据集中进行了无限制视频的广泛实验，表明CAM方法（在独立框架上训练）可以产生不错的定位精度。我们提出的TCAM方法在WSVOL准确性方面达到了新的艺术品，并且视觉结果表明它可以适用于后续任务，例如视觉对象跟踪和检测。代码公开可用。
Weakly supervised video object localization (WSVOL) allows locating object in videos using only global video tags such as object class. State-of-art methods rely on multiple independent stages, where initial spatio-temporal proposals are generated using visual and motion cues, then prominent objects are identified and refined. Localization is done by solving an optimization problem over one or more videos, and video tags are typically used for video clustering. This requires a model per-video or per-class making for costly inference. Moreover, localized regions are not necessary discriminant because of unsupervised motion methods like optical flow, or because video tags are discarded from optimization. In this paper, we leverage the successful class activation mapping (CAM) methods, designed for WSOL based on still images. A new Temporal CAM (TCAM) method is introduced to train a discriminant deep learning (DL) model to exploit spatio-temporal information in videos, using an aggregation mechanism, called CAM-Temporal Max Pooling (CAM-TMP), over consecutive CAMs. In particular, activations of regions of interest (ROIs) are collected from CAMs produced by a pretrained CNN classifier to build pixel-wise pseudo-labels for training the DL model. In addition, a global unsupervised size constraint, and local constraint such as CRF are used to yield more accurate CAMs. Inference over single independent frames allows parallel processing of a clip of frames, and real-time localization. Extensive experiments on two challenging YouTube-Objects datasets for unconstrained videos, indicate that CAM methods (trained on independent frames) can yield decent localization accuracy. Our proposed TCAM method achieves a new state-of-art in WSVOL accuracy, and visual results suggest that it can be adapted for subsequent tasks like visual object tracking and detection. Code is publicly available.
