Drawing reliable inferences from data involves many, sometimes arbitrary, decisions across phases of data collection, wrangling, and modeling. As different choices can lead to diverging conclusions, understanding how researchers make analytic decisions is important for supporting robust and replicable analysis. In this study, we pore over nine published research studies and conduct semi-structured interviews with their authors. We observe that researchers often base their decisions on methodological or theoretical concerns, but subject to constraints arising from the data, expertise, or perceived interpretability. We confirm that researchers may experiment with choices in search of desirable results, but also identify other reasons why researchers explore alternatives yet omit findings. In concert with our interviews, we also contribute visualizations for communicating decision processes throughout an analysis. Based on our results, we identify design opportunities for strengthening end-to-end analysis, for instance via tracking and meta-analysis of multiple decision paths.

中文翻译：

---

探索的路径、省略的路径、模糊的路径：端到端数据分析中的决策点和选择性报告

从数据中得出可靠的推论涉及跨数据收集、整理和建模阶段的许多决策，有时是任意决策。由于不同的选择会导致不同的结论，因此了解研究人员如何做出分析决策对于支持稳健且可复制的分析非常重要。在这项研究中，我们仔细研究了九项已发表的研究，并对其作者进行了半结构化访谈。我们观察到，研究人员经常根据方法论或理论问题做出决定，但会受到数据、专业知识或感知可解释性的限制。我们确认研究人员可能会尝试各种选择来寻找理想的结果，但也会确定研究人员探索替代方案但忽略发现的其他原因。配合我们的采访，我们还提供可视化，用于在整个分析过程中交流决策过程。根据我们的结果，我们确定了加强端到端分析的设计机会，例如通过多个决策路径的跟踪和元分析。