论文标题
明天建造：评估文本分类器的时间持久性
Building for Tomorrow: Assessing the Temporal Persistence of Text Classifiers
论文作者
论文摘要
由于数据的变化，文本分类模型的性能往往会随着时间的流逝而下降，这限制了预验证模型的寿命。因此，预测模型随着时间的推移能力的能力可以帮助设计模型，这些模型可以在更长的时间内有效使用。在本文中，我们对问题进行了详尽的讨论，为任务建立评估设置。从实际角度来看，我们通过评估各种语言模型和分类算法随着时间的推移持续存在的能力，以及数据集特性如何帮助预测不同模型的时间稳定性，从而研究了这个问题。我们在跨越6到19年的三个数据集上执行纵向分类实验，并涉及各种任务和类型的数据。通过将纵向数据集划分为几年，我们通过在过去和将来跨越彼此不同的数据进行培训和测试，通过培训和测试进行全面的实验。这可以逐步研究训练和测试集对分类性能之间的时间差距的影响，并衡量随着时间的推移持久性的程度。
Performance of text classification models tends to drop over time due to changes in data, which limits the lifetime of a pretrained model. Therefore an ability to predict a model's ability to persist over time can help design models that can be effectively used over a longer period of time. In this paper, we provide a thorough discussion into the problem, establish an evaluation setup for the task. We look at this problem from a practical perspective by assessing the ability of a wide range of language models and classification algorithms to persist over time, as well as how dataset characteristics can help predict the temporal stability of different models. We perform longitudinal classification experiments on three datasets spanning between 6 and 19 years, and involving diverse tasks and types of data. By splitting the longitudinal datasets into years, we perform a comprehensive set of experiments by training and testing across data that are different numbers of years apart from each other, both in the past and in the future. This enables a gradual investigation into the impact of the temporal gap between training and test sets on the classification performance, as well as measuring the extent of the persistence over time.
