We present a novel bilateral negotiation model that allows a self-interested agent to learn how to negotiate over multiple issues in the presence of user preference uncertainty. The model relies upon interpretable strategy templates representing the tactics the agent should employ during the negotiation and learns template parameters to maximize the average utility received over multiple negotiations, thus resulting in optimal bid acceptance and generation. Our model also uses deep reinforcement learning to evaluate threshold utility values, for those tactics that require them, thereby deriving optimal utilities for every environment state. To handle user preference uncertainty, the model relies on a stochastic search to find user model that best agrees with a given partial preference profile. Multi-objective optimization and multi-criteria decision-making methods are applied at negotiation time to generate Pareto-optimal outcomes thereby increasing the number of successful (win-win) negotiations. Rigorous experimental evaluations show that the agent employing our model outperforms the winning agents of the 10th Automated Negotiating Agents Competition (ANAC'19) in terms of individual as well as social-welfare utilities.

中文翻译：

---

针对多个问题的双边代理谈判的可学习策略

我们提出了一种新颖的双边协商模型，该模型允许自利代理学习如何在存在用户偏好不确定性的情况下就多个问题进行协商。该模型依赖于代表代理在谈判期间应采用的策略的可解释策略模板，并学习模板参数以最大化在多次谈判中收到的平均效用，从而实现最佳投标接受和生成。我们的模型还使用深度强化学习来评估阈值效用值，用于需要它们的策略，从而为每个环境状态推导出最佳效用。为了处理用户偏好的不确定性，该模型依靠随机搜索来找到最符合给定部分偏好配置文件的用户模型。在谈判时应用多目标优化和多标准决策方法来产生帕累托最优结果，从而增加成功（双赢）谈判的数量。严格的实验评估表明，在个人和社会福利效用方面，采用我们模型的代理的表现优于第 10 届自动谈判代理竞赛 (ANAC'19) 的获胜代理。