论文标题
在上下文中从语言中推断出奖励
Inferring Rewards from Language in Context
论文作者
论文摘要
在经典的指导下，诸如“我想要捷蓝航空”之类的语言映射到动作（例如，选择该航班）。但是，语言还传达了有关用户基础奖励功能的信息（例如，对JetBlue的一般偏好），该功能可以允许模型在新环境中执行理想的操作。我们提出了一个模型，该模型务实地从语言中汲取了回报：关于说话者如何选择话语的推理，不仅是为了引起所需的行动，还可以揭示有关其偏好的信息。在使用自然语言的新的交互式飞行预订任务中，我们的模型更准确地渗透了奖励，并预测了在看不见的环境中的最佳动作，与过去的工作相比，将首先将语言映射到动作（以下说明），然后将动作映射到奖励（逆增强学习）。
In classic instruction following, language like "I'd like the JetBlue flight" maps to actions (e.g., selecting that flight). However, language also conveys information about a user's underlying reward function (e.g., a general preference for JetBlue), which can allow a model to carry out desirable actions in new contexts. We present a model that infers rewards from language pragmatically: reasoning about how speakers choose utterances not only to elicit desired actions, but also to reveal information about their preferences. On a new interactive flight-booking task with natural language, our model more accurately infers rewards and predicts optimal actions in unseen environments, in comparison to past work that first maps language to actions (instruction following) and then maps actions to rewards (inverse reinforcement learning).
