Computational social science uses computational and statistical methods in order to evaluate social interaction. The public availability of data sets is thus a necessary precondition for reliable and replicable research. These data allow researchers to benchmark the computational methods they develop, test the generalizability of their findings, and build confidence in their results. When social media data are concerned, data sharing is often restricted for legal or privacy reasons, which makes the comparison of methods and the replicability of research results infeasible. Social media analytics research, consequently, faces an integrity crisis. How is it possible to create trust in computational or statistical analyses, when they cannot be validated by third parties? In this work, we explore this well-known, yet little discussed, problem for social media analytics. We investigate how this problem can be solved by looking at related computational research areas. Moreover, we propose and implement a prototype to address the problem in the form of a new evaluation framework that enables the comparison of algorithms without the need to exchange data directly, while maintaining flexibility for the algorithm design.

计算社会科学使用计算和统计方法来评估社会互动。因此，数据集的公开可用性是进行可靠且可复制的研究的必要先决条件。这些数据使研究人员可以对他们开发的计算方法进行基准测试，测试其发现的一般性并建立对结果的信心。当涉及社交媒体数据时，出于法律或隐私原因，数据共享常常受到限制，这使得方法的比较和研究结果的可复制性变得不可行。因此，社交媒体分析研究面临诚信危机。当第三方无法验证它们时，如何建立对计算或统计分析的信任？在这项工作中，我们探索了这个众所周知但很少讨论的内容，社交媒体分析的问题。我们通过查看相关的计算研究领域来研究如何解决此问题。此外，我们提出并实现了一个原型，以一种新的评估框架的形式来解决该问题，该评估框架使算法的比较无需直接交换数据即可进行，同时保持了算法设计的灵活性。