论文标题
超越现状：对音频字幕的进步和挑战的当代调查
Beyond the Status Quo: A Contemporary Survey of Advances and Challenges in Audio Captioning
论文作者
论文摘要
自动音频字幕（AAC）是模仿人类感知的任务，以及创新的音频处理和自然语言处理，在过去几年中一直在监督了很多进步。 AAC需要识别诸如环境，声音事件和声音事件之间的时间关系之类的内容，并用流利的句子描述这些元素。当前，基于编码器的深度学习框架是解决此问题的标准方法。大量作品提出了新型的网络体系结构和培训方案，包括额外的指导，强化学习，音频文本自我监督学习以及多样或可控的字幕。探索了有效的数据增强技术，尤其是基于大型语言模型。基准数据集和面向AAC的评估指标也加速了该领域的改进。本文将自己作为一项全面的调查，涵盖了AAC及其相关任务，现有的深度学习技术，数据集和AAC评估指标之间的比较，并提供了见解，以指导潜在的未来研究指导。
Automated audio captioning (AAC), a task that mimics human perception as well as innovatively links audio processing and natural language processing, has overseen much progress over the last few years. AAC requires recognizing contents such as the environment, sound events and the temporal relationships between sound events and describing these elements with a fluent sentence. Currently, an encoder-decoder-based deep learning framework is the standard approach to tackle this problem. Plenty of works have proposed novel network architectures and training schemes, including extra guidance, reinforcement learning, audio-text self-supervised learning and diverse or controllable captioning. Effective data augmentation techniques, especially based on large language models are explored. Benchmark datasets and AAC-oriented evaluation metrics also accelerate the improvement of this field. This paper situates itself as a comprehensive survey covering the comparison between AAC and its related tasks, the existing deep learning techniques, datasets, and the evaluation metrics in AAC, with insights provided to guide potential future research directions.
