In this paper, we re-implement a recent work published in Usenix Security 2018: "Acquistional Rule Based Engine for Discovering Internet-of-Things Devices". The paper introduced an NLP-based engine for automatically identifying the type, vendor, and product of IoT devices given banner data as input. We report on our efforts to reproduce the original implementation of the engine, documenting ambiguities around implementation and evaluation details that we encountered, as well as how we addressed them in our work. We evaluate our implementation on two ground truth datasets, finding that it fails to achieve the accuracy reported by the original authors. Our findings highlight the importance of recent community efforts towards a culture of reproducibility by presenting an example of how ambiguities in a research paper combined with lack of access to the original datasets can significantly affect a faithful re-implementation and evaluation.

中文翻译：

---

使用应用层横幅数据自动识别物联网设备

在本文中，我们重新实现了最近发表在 Usenix Security 2018 上的工作：“基于获取规则的引擎发现物联网设备”。该论文介绍了一种基于 NLP 的引擎，用于在输入横幅数据的情况下自动识别物联网设备的类型、供应商和产品。我们报告了我们为重现引擎的原始实现所做的努力，记录了我们遇到的实现和评估细节的歧义，以及我们如何在工作中解决这些问题。我们在两个基本事实数据集上评估我们的实施，发现它未能达到原作者报告的准确性。