Online reputation systems serve as core building blocks in various Internet services such as E-commerce (e.g., eBay) and crowdsourcing (e.g., oDesk). The flaws and deficiencies of real-world online reputation systems have been reported extensively. Users who are frustrated about the system will eventually abandon such service. However, there is no systematic and formal studies which examine such deficiencies. This paper presents the first attempt, which develops a novel data analytical framework to uncover online reputation system deficiencies from data. We develop two novel measures to quantify the efficiency of online reputation systems: (1) ramp up time of a new service provider, (2) long term profit gains for a service provider. We present a new data analytical framework to evaluate these two measures from data. We show that inherent preferences or personal biases in expressing feedbacks (or ratings) cause the computational infeasibility in evaluating the ramp up time and the long term profit gains from data. We develop two computationally efficient randomized algorithms with theoretical performance guarantees to address this computational challenge. We apply our methodology to analyze real-life datasets (from eBay, Google Helpouts, Amazon and TripAdvisor). We extensively validate our model and we uncover the deficiencies of online reputation systems. Our experimental results uncovers insights on why Google Helpouts was eventually shut down in April 2015 and why eBay is losing some sellers heavily.

中文翻译：

---

挖掘在线声誉系统的缺陷：方法、实验和启示

在线声誉系统是电子商务（例如 eBay）和众包（例如 oDesk）等各种互联网服务的核心构建块。现实世界在线声誉系统的缺陷和缺陷已被广泛报道。对系统感到沮丧的用户最终会放弃此类服务。然而，没有系统和正式的研究来检查这些缺陷。本文提出了第一次尝试，它开发了一种新颖的数据分析框架，以从数据中发现在线声誉系统的缺陷。我们开发了两种新方法来量化在线声誉系统的效率：（1）新服务提供商的加速时间，（2）服务提供商的长期利润收益。我们提出了一个新的数据分析框架来从数据中评估这两个度量。我们表明，在表达反馈（或评级）时固有的偏好或个人偏见导致在评估加速时间和从数据中获得的长期利润收益时的计算不可行性。我们开发了两种具有理论性能保证的计算高效的随机算法来解决这一计算挑战。我们应用我们的方法来分析现实生活中的数据集（来自 eBay、Google Helpouts、Amazon 和 TripAdvisor）。我们广泛验证了我们的模型，并发现了在线声誉系统的缺陷。我们的实验结果揭示了为什么 Google Helpouts 最终在 2015 年 4 月被关闭以及为什么 eBay 会严重失去一些卖家。