Abstract

Best practices in statistics and data science courses include the use of real and relevant data as well as teaching the entire data science cycle starting with importing data. A rich source of real and current data is the web, where data are often presented and stored in a structure that needs some wrangling and transforming before they can be ready for analysis. The web is a resource students naturally turn to for finding data for data analysis projects, but without formal instruction on how to get that data into a structured format, they often resort to copy-pasting or manual entry into a spreadsheet, which are both time consuming and error-prone. Teaching web scraping provides an opportunity to bring such data into the curriculum in an effective and efficient way. In this article, we explain how web scraping works and how it can be implemented in a pedagogically sound and technically executable way at various levels of statistics and data science curricula. We provide classroom activities where we connect this modern computing technique with traditional statistical topics. Finally, we share the opportunities web scraping brings to the classrooms as well as the challenges to instructors and tips for avoiding them.

摘要

统计和数据科学课程的最佳实践包括使用实际数据和相关数据，以及从导入数据开始的整个数据科学周期的教学。网络是真实和当前数据的丰富来源，通常在其中将数据呈现和存储在一个结构中，该结构需要进行一些整理和转换才能准备好进行分析。网络是学生自然而然地用来查找数据分析项目数据的资源，但是由于没有关于如何将数据转换为结构化格式的正式指导，他们常常求助于复制粘贴或手动输入电子表格的工作，这两者都是时间耗时且容易出错。网上抓取教学提供了一种以有效方式将此类数据引入课程的机会。在本文中，我们将解释在各种统计和数据科学课程中，网络抓取是如何工作的，以及如何以教学上合理且技术上可执行的方式实施的。我们提供课堂活动，将这种现代计算技术与传统的统计主题联系起来。最后，我们分享了网络抓取为课堂带来的机遇，以及对教师的挑战和避免方法。