论文标题
MEMONET：通过CTR预测有效地记住所有Cross功能的表示形式
MemoNet: Memorizing All Cross Features' Representations Efficiently via Multi-Hash Codebook Network for CTR Prediction
论文作者
论文摘要
自然语言处理（NLP）中的新发现表明，强大的记忆能力为大型语言模型（LLM）的成功做出了很大贡献。这激发了我们明确地将独立的内存机制带入CTR排名模型中，以学习和记住交叉功能的表示。在本文中，我们提出了多锤代码书网络（HCNET），作为有效学习和记忆CTR任务中跨特征表示的内存机制。 HCNET使用多锤代码簿作为主存储位置，整个内存过程包括三个阶段：多锤地址，内存恢复和功能缩小。我们还提出了一种名为Memonet的新CTR模型，该模型将HCNET与DNN骨架结合在一起。三个公共数据集和在线测试的广泛实验结果表明，Memonet比最先进的方法达到了卓越的性能。此外，MEMONET在NLP中显示了大语言模型的扩展定律，这意味着我们可以扩大HCNet中代码书的大小，以可持续地获得性能提高。我们的工作证明了学习和记忆交叉特征表示的重要性和可行性，这阐明了新的有希望的研究方向。
New findings in natural language processing (NLP) demonstrate that the strong memorization capability contributes a lot to the success of Large Language Models (LLM). This inspires us to explicitly bring an independent memory mechanism into CTR ranking model to learn and memorize cross features' representations. In this paper, we propose multi-Hash Codebook NETwork (HCNet) as the memory mechanism for efficiently learning and memorizing representations of cross features in CTR tasks. HCNet uses a multi-hash codebook as the main memory place and the whole memory procedure consists of three phases: multi-hash addressing, memory restoring, and feature shrinking. We also propose a new CTR model named MemoNet which combines HCNet with a DNN backbone. Extensive experimental results on three public datasets and online test show that MemoNet reaches superior performance over state-of-the-art approaches. Besides, MemoNet shows scaling law of large language model in NLP, which means we can enlarge the size of the codebook in HCNet to sustainably obtain performance gains. Our work demonstrates the importance and feasibility of learning and memorizing representations of cross features, which sheds light on a new promising research direction.
