论文标题
自动合成到现实的概括
Automated Synthetic-to-Real Generalization
论文作者
论文摘要
经过合成图像训练的模型通常面临着对真实数据的泛化。作为一个惯例，这些模型通常用ImageNet预训练的表示初始化。然而，尽管有共同的实践利用了这种知识来维持概括能力，但很少讨论成像网知识的作用。一个例子是对早期停止和层次学习率进行仔细的手工调节，这证明可以改善合成到现实的概括，但也很费力和启发式。在这项工作中，我们明确鼓励经过合成训练的模型通过ImageNet预训练的模型维护相似的表示，并提出了\ textIt {学习到optimize（L2O）}策略，以自动化层次学习率的选择。我们证明，所提出的框架可以显着改善合成到现实的概括性能，而无需看到和培训真实数据，同时也使下游任务（例如域适应）受益。代码可在以下网址获得：https：//github.com/nvlabs/asg。
Models trained on synthetic images often face degraded generalization to real data. As a convention, these models are often initialized with ImageNet pre-trained representation. Yet the role of ImageNet knowledge is seldom discussed despite common practices that leverage this knowledge to maintain the generalization ability. An example is the careful hand-tuning of early stopping and layer-wise learning rates, which is shown to improve synthetic-to-real generalization but is also laborious and heuristic. In this work, we explicitly encourage the synthetically trained model to maintain similar representations with the ImageNet pre-trained model, and propose a \textit{learning-to-optimize (L2O)} strategy to automate the selection of layer-wise learning rates. We demonstrate that the proposed framework can significantly improve the synthetic-to-real generalization performance without seeing and training on real data, while also benefiting downstream tasks such as domain adaptation. Code is available at: https://github.com/NVlabs/ASG.
