We present a novel negotiation model that allows an agent to learn how to negotiate during concurrent bilateral negotiations in unknown and dynamic e-markets. The agent uses an actor-critic architecture with model-free reinforcement learning to learn a strategy expressed as a deep neural network. We pre-train the strategy by supervision from synthetic market data, thereby decreasing the exploration time required for learning during negotiation. As a result, we can build automated agents for concurrent negotiations that can adapt to different e-market settings without the need to be pre-programmed. Our experimental evaluation shows that our deep reinforcement learning based agents outperform two existing well-known negotiation strategies in one-to-many concurrent bilateral negotiations for a range of e-market settings.

我们提出了一种新颖的谈判模型，该模型允许代理学习如何在未知和动态电子市场中同时进行的双边谈判中进行谈判。该代理使用具有无模型强化学习的 actor-critic 架构来学习表示为深度神经网络的策略。我们通过对合成市场数据的监督来预训练策略，从而减少谈判过程中学习所需的探索时间。因此，我们可以为并发谈判构建自动化代理，该代理可以适应不同的电子市场设置，而无需预先编程。我们的实验评估表明，我们基于深度强化学习的代理在一系列电子市场设置的一对多并发双边谈判中优于两种现有的知名谈判策略。