Context

The computational linguistics community has developed tools, called wikifiers, to identify links to Wikipedia articles from free-form text. Software engineering research can leverage wikifiers to add semantic information to software artifacts. However, no empirically-grounded basis exists to choose an effective wikifier and to configure it for the software domain, on which wikifiers were not specifically trained.

Objective

We conducted a study to guide the selection of a wikifier and its configuration for applications in the software domain, and to measure what performance can be expected of wikifiers.

Method

We applied six wikifiers, with multiple configurations, to a sample of 500 Stack Overflow posts. We manually annotated the 41 124 articles identified by the wikifiers as correct or not to compare their precision and recall.

Results

Each wikifier, in turn, achieved the highest precision, between 13% and 82%, for different thresholds of recall, from 60% to 5%. However, filtering the wikifiers’ output with a whitelist can considerably improve the precision above 79% for recall up to 30%, and above 47% for recall up to 60%.

Conclusions

Results reported in each wikifier’s original article cannot be generalized to software-specific documents. Given that no wikifier performs universally better than all others, we provide empirically grounded insights to select a wikifier for different scenarios, and suggest ways to further improve their performance for the software domain.

中文翻译：

---

增强软件工件

语境

计算语言学社区已经开发了称为Wikifiers的工具，用于从自由格式文本中标识到Wikipedia文章的链接。软件工程研究可以利用Wikifier将语义信息添加到软件工件中。但是，不存在基于经验的依据来选择有效的增强器并将其配置用于软件领域，而Wikifier并未经过专门的训练。

客观的

我们进行了一项研究，以指导在软件领域中为应用程序选择Wikifier及其配置，并衡量Wikifier的预期性能。

方法

我们将六个具有多种配置的wikifier应用于500个Stack Overflow帖子的样本。我们手动注释了wikifiers认为正确或不正确的41 124篇文章，以比较其准确性和召回率。

结果

相应地，对于不同的召回阈值（从60％到5％），每个wikifier都达到了最高的精度，介于13％和82％之间。但是，用白名单过滤Wikifier的输出可以大大提高精度，召回率高达30％时高于79％，召回率高达60％时高于47％。

结论

每个wikifier原始文章中报告的结果均不能推广到特定于软件的文档。鉴于没有任何wikifier的性能普遍优于其他所有，我们提供了基于经验的见解，为不同情况选择wikifier，并提出了进一步改善其在软件领域性能的方法。