当前位置: X-MOL 学术arXiv.cs.CL › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
Adjusting for Confounders with Text: Challenges and an Empirical Evaluation Framework for Causal Inference
arXiv - CS - Computation and Language Pub Date : 2020-09-21 , DOI: arxiv-2009.09961
Galen Weld, Peter West, Maria Glenski, David Arbour, Ryan Rossi, Tim Althoff

Leveraging text, such as social media posts, for causal inferences requires the use of NLP models to 'learn' and adjust for confounders, which could otherwise impart bias. However, evaluating such models is challenging, as ground truth is almost never available. We demonstrate the need for empirical evaluation frameworks for causal inference in natural language by showing that existing, commonly used models regularly disagree with one another on real world tasks. We contribute the first such framework, generalizing several challenges across these real world tasks. Using this framework, we evaluate a large set of commonly used causal inference models based on propensity scores and identify their strengths and weaknesses to inform future improvements. We make all tasks, data, and models public to inform applications and encourage additional research.

中文翻译:

用文本调整混杂因素:挑战和因果推理的实证评估框架

利用社交媒体帖子等文本进行因果推断需要使用 NLP 模型来“学习”并调整混杂因素,否则可能会产生偏见。然而,评估此类模型具有挑战性,因为基本事实几乎永远无法获得。我们通过展示现有的、常用的模型在现实世界的任务中经常彼此不一致,证明了对自然语言因果推理的经验评估框架的必要性。我们贡献了第一个这样的框架,概括了这些现实世界任务中的几个挑战。使用这个框架,我们根据倾向得分评估了大量常用的因果推理模型,并确定它们的优势和劣势,以告知未来的改进。我们将所有任务、数据、
更新日期:2020-09-22
down
wechat
bug