ABSTRACT

While traditional computer-mediated communication happened through transparent, passive, and neutral channels, today’s communication channels are obscure, proactive, and distorted. Social algorithms, guided by a socio-technological codependency, often bias communication, usually in pursuit of some third-party goal of commercial or political nature. We propose a method to derive several summary measures to tests for transformational accuracy when transforming input into an output. Since dynamical flexibility of social algorithms prevents anticipating their behavior, we study these black boxes as if we study human behavior, through controlled experiments. We conceptualize them as noisy communication channels and evaluate their throughput with the same information theoretic measures engineers had originally used to minimize communicative distortion (i.e., mutual information). We use repeated experiments to reverse-engineer algorithmic behavior and test for its statistical significance. We apply the method to three artificial intelligence algorithms: a neural net from IBM’s Watson, and to the recommender engines of YouTube and Twitter.

摘要

传统的计算机介导的通信是通过透明，被动和中立的渠道发生的，而如今的通信渠道却是模糊，主动和扭曲的。在社会技术相互依存的指导下，社会算法通常会偏向交流，通常是为了追求某些具有商业或政治性质的第三方目标。我们提出了一种方法，用于在将输入转换为输出时推导多种汇总度量来测试转换精度。由于社交算法的动态灵活性会阻止人们预测其行为，因此我们通过受控实验来研究这些黑匣子，就像研究人类行为一样。我们将它们概念化为嘈杂的通信渠道，并使用工程师最初用于最大程度地减少通信失真（即互信息）的信息理论方法来评估它们的吞吐量。我们使用重复实验对算法行为进行逆向工程并测试其统计意义。我们将该方法应用于三种人工智能算法：来自IBM Watson的神经网络，以及YouTube和Twitter的推荐引擎。