Story point estimation is a task to estimate the overall effort required to fully implement a product backlog item. Various estimation approaches (e.g., Planning Poker, Analogy, and expert judgment) are widely-used, yet they are still inaccurate and may be subjective, leading to ineffective sprint planning. Recent work proposed Deep-SE, a deep learning-based Agile story point estimation approach, yet it is still inaccurate, not transferable to other projects, and not interpretable. In this paper, we propose GPT2SP, a Transformer-based Agile Story Point Estimation approach. Our GPT2SP employs a GPT-2 pre-trained language model with a GPT-2 Transformer-based architecture, allowing our GPT2SP models to better capture the relationship among words while considering the context surrounding a given word and its position in the sequence and be transferable to other projects, while being interpretable. Through an extensive evaluation on 23,313 issues that span across 16 open-source software projects with 10 existing baseline approaches for within- and cross-project scenarios, our results show that our GPT2SP approach achieves a median MAE of 1.16, which is (1) 34%-57% more accurate than existing baseline approaches for within-project estimations; (2) 39%-49% more accurate than existing baseline approaches for cross-project estimations. The ablation study also shows that the GPT-2 architecture used in our approach substantially improves Deep-SE by 6%-47%, highlighting the significant advancement of the AI for Agile story point estimation. Finally, we develop a proof-of-concept tool to help practitioners better understand the most important words that contributed to the story point estimation of the given issue with the best supporting examples from past estimates. Our survey study with 16 Agile practitioners shows that the story point estimation task is perceived as an extremely challenging task. In addition, our AI-based story point estimation with explanations is perceived as more useful and trustworthy than without explanations, highlighting the practical need of our Explainable AI-based story point estimation approach.

中文翻译：

---

GPT2SP：基于 Transformer 的敏捷故事点估计方法


故事点估计是一项估计完全实现产品积压项目所需的总体工作量的任务。各种估计方法（例如，规划扑克、类比和专家判断）被广泛使用，但它们仍然不准确并且可能是主观的，导致冲刺计划无效。最近的工作提出了 Deep-SE，一种基于深度学习的敏捷故事点估计方法，但它仍然不准确，无法转移到其他项目，并且不可解释。在本文中，我们提出了 GPT2SP，一种基于 Transformer 的敏捷故事点估计方法。我们的 GPT2SP 采用 GPT-2 预训练语言模型和基于 GPT-2 Transformer 的架构，使我们的 GPT2SP 模型能够更好地捕获单词之间的关系，同时考虑给定单词周围的上下文及其在序列中的位置，并且可转移到其他项目，同时可以解释。通过对跨 16 个开源软件项目的 23,313 个问题以及针对项目内和跨项目场景的 10 种现有基线方法进行广泛评估，我们的结果表明，我们的 GPT2SP 方法的 MAE 中位数为 1.16，即 (1) 34比项目内估算的现有基线方法准确 %-57%； (2) 比现有的跨项目估算基线方法准确39%-49%。消融研究还表明，我们的方法中使用的 GPT-2 架构将 Deep-SE 显着提高了 6%-47%，凸显了人工智能在敏捷故事点估计方面的显着进步。最后，我们开发了一个概念验证工具，以帮助从业者更好地理解对给定问题的故事点估计做出贡献的最重要的单词，以及过去估计中的最佳支持示例。 我们对 16 名敏捷实践者的调查研究表明，故事点估计任务被认为是一项极具挑战性的任务。此外，我们基于人工智能的带有解释的故事点估计被认为比没有解释更有用和值得信赖，这凸显了我们基于可解释的基于人工智能的故事点估计方法的实际需要。