The widespread use of experimental benchmarks in AI research has created competition and collaboration dynamics that are still poorly understood. Here we provide an innovative methodology to explore these dynamics and analyse the way different entrants in these challenges, from academia to tech giants, behave and react depending on their own or others’ achievements. We perform an analysis of 25 popular benchmarks in AI from Papers With Code, with around 2,000 result entries overall, connected with their underlying research papers. We identify links between researchers and institutions (that is, communities) beyond the standard co-authorship relations, and we explore a series of hypotheses about their behaviour as well as some aggregated results in terms of activity, performance jumps and efficiency. We characterize the dynamics of research communities at different levels of abstraction, including organization, affiliation, trajectories, results and activity. We find that hybrid, multi-institution and persevering communities are more likely to improve state-of-the-art performance, which becomes a watershed for many community members. Although the results cannot be extrapolated beyond our selection of popular machine learning benchmarks, the methodology can be extended to other areas of artificial intelligence or robotics, and combined with bibliometric studies.

人工智能研究中实验基准的广泛使用创造了竞争和协作动态，但人们仍然知之甚少。在这里，我们提供了一种创新的方法来探索这些动态，并分析从学术界到科技巨头的不同参与者在这些挑战中的行为和反应方式，这取决于他们自己或他人的成就。我们对 Papers With Code 中的 25 个流行的 AI 基准进行了分析，总共有大约 2,000 个结果条目，与其基础研究论文相关联。我们确定了超出标准合着关系的研究人员和机构（即社区）之间的联系，我们探索了一系列关于他们行为的假设，以及在活动、绩效跳跃和效率方面的一些汇总结果。我们在不同抽象层次上描述研究社区的动态，包括组织、隶属关系、轨迹、结果和活动。我们发现混合、多机构和坚持不懈的社区更有可能提高最先进的性能，这成为许多社区成员的分水岭。尽管结果无法推断超出我们选择的流行机器学习基准，但该方法可以扩展到人工智能或机器人技术的其他领域，并与文献计量研究相结合。这成为许多社区成员的分水岭。尽管结果无法推断超出我们选择的流行机器学习基准，但该方法可以扩展到人工智能或机器人技术的其他领域，并与文献计量研究相结合。这成为许多社区成员的分水岭。尽管结果无法推断超出我们选择的流行机器学习基准，但该方法可以扩展到人工智能或机器人技术的其他领域，并与文献计量研究相结合。