论文标题
非单击是无关紧要的？倾向比评分作为校正
Non-Clicks Mean Irrelevant? Propensity Ratio Scoring As a Correction
论文作者
论文摘要
无偏学习的最新进展（LTR）依靠反向倾向评分（IP）来消除隐式反馈中的偏见。尽管从理论上讲，通过将点击文档视为相关引入的偏见时，IPS忽略了（隐含地）将未粘贴的偏差视为无关的偏见。在这项工作中，我们首先严格地证明，这种点击数据的使用导致相关文档之间不必要的成对比较，从而防止无偏的排名者优化。根据证明，我们得出了一种简单而有道理的新加权方案，称为倾向比评分（PRS），该方案在点击和非单击方面提供处理。除了在点击中纠正偏见外，PR还避免了相关的文档比较LTR培训中的比较，并享有较低的可变性。我们广泛的经验评估证实，PR可确保从一组LTR基准测试以及Gmail Search的真实世界大规模数据中更有效地利用点击数据并提高了两个合成数据的性能。
Recent advances in unbiased learning to rank (LTR) count on Inverse Propensity Scoring (IPS) to eliminate bias in implicit feedback. Though theoretically sound in correcting the bias introduced by treating clicked documents as relevant, IPS ignores the bias caused by (implicitly) treating non-clicked ones as irrelevant. In this work, we first rigorously prove that such use of click data leads to unnecessary pairwise comparisons between relevant documents, which prevent unbiased ranker optimization. Based on the proof, we derive a simple yet well justified new weighting scheme, called Propensity Ratio Scoring (PRS), which provides treatments on both clicks and non-clicks. Besides correcting the bias in clicks, PRS avoids relevant-relevant document comparisons in LTR training and enjoys a lower variability. Our extensive empirical evaluations confirm that PRS ensures a more effective use of click data and improved performance in both synthetic data from a set of LTR benchmarks, as well as in the real-world large-scale data from GMail search.
