In online advertising, auto-bidding has become an essential tool for advertisers to optimize their preferred ad performance metrics by simply expressing the high-level campaign objectives and constraints. Previous works consider the design of auto-bidding agents from the single-agent view without modeling the mutual influence between agents. In this paper, we instead consider this problem from the perspective of a distributed multi-agent system, and propose a general Multi-Agent reinforcement learning framework for Auto-Bidding, namely MAAB, to learn the auto-bidding strategies. First, we investigate the competition and cooperation relation among auto-bidding agents, and propose temperature-regularized credit assignment for establishing a mixed cooperative-competitive paradigm. By carefully making a competition and cooperation trade-off among the agents, we can reach an equilibrium state that guarantees not only individual advertiser's utility but also the system performance (social welfare). Second, due to the observed collusion behaviors of bidding low prices underlying the cooperation, we further propose bar agents to set a personalized bidding bar for each agent, and then to alleviate the degradation of revenue. Third, to deploy MAAB to the large-scale advertising system with millions of advertisers, we propose a mean-field approach. By grouping advertisers with the same objective as a mean auto-bidding agent, the interactions among advertisers are greatly simplified, making it practical to train MAAB efficiently. Extensive experiments on the offline industrial dataset and Alibaba advertising platform demonstrate that our approach outperforms several baseline methods in terms of social welfare and guarantees the ad platform's revenue.

中文翻译：

---

一种用于在线广告自动竞价的合作竞争多代理框架

在在线广告中，自动竞价已成为广告商通过简单表达高级活动目标和限制来优化其首选广告绩效指标的重要工具。以前的工作从单一代理的角度考虑自动投标代理的设计，而没有对代理之间的相互影响进行建模。在本文中，我们从分布式多智能体系统的角度考虑这个问题，并提出了一个通用的自动竞价多智能体强化学习框架，即 MAAB，来学习自动竞价策略。首先，我们研究了自动投标代理之间的竞争与合作关系，并提出了温度调节信用分配，以建立混合合作竞争范式。通过谨慎地在代理之间进行竞争和合作权衡，我们可以达到一个平衡状态，不仅保证个人广告商的效用，而且保证系统性能（社会福利）。其次，由于观察到合作背后竞价低价的勾结行为，我们进一步建议酒吧代理为每个代理设置个性化的投标栏，从而缓解收入下降。第三，为了将 MAAB 部署到具有数百万广告商的大规模广告系统，我们提出了平均场方法。通过将具有与平均自动投标代理相同目标的广告客户分组，大大简化了广告客户之间的交互，使得有效训练 MAAB 变得可行。