论文标题
部分可观测时空混沌系统的无模型预测
MolE: a molecular foundation model for drug discovery
论文作者
论文摘要
准确预测基于化学结构的特性的模型是药物发现中的有价值的工具。但是，对于许多物业，公共和私人培训集通常很小，并且模型很难在培训数据之外概述。最近，大型语言模型通过在大型未标记的数据集上使用自我监督的预处理解决了这个问题，然后在较小的标签数据集上进行微调。在本文中，我们报告了mole，这是一种分子基础模型，该模型适应了Deberta架构，用于分子图和两步预处理策略。预处理的第一步是一种专注于学习化学结构的自学方法，第二步是一种学习生物学信息的大量多任务方法。我们表明，经过验证的痣可以在治疗数据共享中包含的22项ADMET任务中有9个实现了最先进的结果。
Models that accurately predict properties based on chemical structure are valuable tools in drug discovery. However, for many properties, public and private training sets are typically small, and it is difficult for the models to generalize well outside of the training data. Recently, large language models have addressed this problem by using self-supervised pretraining on large unlabeled datasets, followed by fine-tuning on smaller, labeled datasets. In this paper, we report MolE, a molecular foundation model that adapts the DeBERTa architecture to be used on molecular graphs together with a two-step pretraining strategy. The first step of pretraining is a self-supervised approach focused on learning chemical structures, and the second step is a massive multi-task approach to learn biological information. We show that fine-tuning pretrained MolE achieves state-of-the-art results on 9 of the 22 ADMET tasks included in the Therapeutic Data Commons.
