Natural language and visualization are being increasingly deployed together for supporting data analysis in different ways, from multimodal interaction to enriched data summaries and insights. Yet, researchers still lack systematic knowledge on how viewers verbalize their interpretations of visualizations, and how they interpret verbalizations of visualizations in such contexts. We describe two studies aimed at identifying characteristics of data and charts that are relevant in such tasks. The first study asks participants to verbalize what they see in scatterplots that depict various levels of correlations. The second study then asks participants to choose visualizations that match a given verbal description of correlation. We extract key concepts from responses, organize them in a taxonomy and analyze the categorized responses. We observe that participants use a wide range of vocabulary across all scatterplots, but particular concepts are preferred for higher levels of correlation. A comparison between the studies reveals the ambiguity of some of the concepts. We discuss how the results could inform the design of multimodal representations aligned with the data and analytical tasks, and present a research roadmap to deepen the understanding about visualizations and natural language.

中文翻译：

---

估计相关性的词语：研究散点图的语言化


自然语言和可视化越来越多地一起部署，以不同的方式支持数据分析，从多模式交互到丰富的数据摘要和见解。然而，研究人员仍然缺乏关于观众如何用语言表达他们对可视化的解释，以及他们如何在这种背景下解释可视化的语言化的系统知识。我们描述了两项旨在识别与此类任务相关的数据和图表特征的研究。第一项研究要求参与者用语言描述他们在描述不同级别相关性的散点图中看到的内容。然后，第二项研究要求参与者选择与给定的相关性口头描述相匹配的可视化效果。我们从回复中提取关键概念，将其组织成分类法并分析分类的回复。我们观察到参与者在所有散点图中使用了广泛的词汇，但为了更高级别的相关性，更倾向于使用特定的概念。研究之间的比较揭示了一些概念的模糊性。我们讨论结果如何为与数据和分析任务相一致的多模态表示设计提供信息，并提出研究路线图以加深对可视化和自然语言的理解。