论文标题
走向类似人类的开放域聊天机器人
Towards a Human-like Open-Domain Chatbot
论文作者
论文摘要
我们介绍了Meena，这是一种多转弯域聊天机器人，经过培训的端到端，该数据已挖掘并从公共领域社交媒体对话中过滤。这个2.6b参数神经网络经过训练，以最大程度地减少接下来的令牌的困惑。我们还提出了一个称为“敏感性和特异性平均值”（SSA）的人类评估指标，该指标捕获了类似人类的多转交谈的关键要素。我们的实验显示了困惑性与SSA之间的密切相关性。 SSA的最佳困惑端到端训练的MEENA得分很高（多转变评估的72％）表明，如果我们能够更好地优化困惑，则有可能达到86％的人类SSA。此外，MEENA的完整版本（具有过滤机制和调谐解码）得分为79％，绝对SSA比我们评估的现有聊天机器人高23％。
We present Meena, a multi-turn open-domain chatbot trained end-to-end on data mined and filtered from public domain social media conversations. This 2.6B parameter neural network is simply trained to minimize perplexity of the next token. We also propose a human evaluation metric called Sensibleness and Specificity Average (SSA), which captures key elements of a human-like multi-turn conversation. Our experiments show strong correlation between perplexity and SSA. The fact that the best perplexity end-to-end trained Meena scores high on SSA (72% on multi-turn evaluation) suggests that a human-level SSA of 86% is potentially within reach if we can better optimize perplexity. Additionally, the full version of Meena (with a filtering mechanism and tuned decoding) scores 79% SSA, 23% higher in absolute SSA than the existing chatbots we evaluated.
