当前位置: X-MOL 学术Sci. Comput. Program. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
A debugging approach for live Big Data applications
Science of Computer Programming ( IF 1.5 ) Pub Date : 2020-04-03 , DOI: 10.1016/j.scico.2020.102460
Matteo Marra , Guillermo Polito , Elisa Gonzalez Boix

Many frameworks exist for programmers to develop and deploy Big Data applications such as Hadoop Map/Reduce and Apache Spark. However, very little debugging support is currently provided in those frameworks. When an error occurs, developers are lost in trying to understand what has happened from the information provided in log files. Recently, new solutions allow developers to record & replay the application execution, but replaying is not always affordable when hours of computation need to be re-executed. In this paper, we present an online approach that allows developers to debug Big Data applications in isolation by moving the debugging session to an external process when a halting point is reached. We introduce IDRAMR, our prototype implementation in Pharo. IDRAMR centralizes the debugging of parallel applications by introducing novel debugging concepts, such as composite debugging events, and the ability to dynamically update both the code of the debugged application and the same configuration of the running framework. We validate our approach by debugging both application and configuration failures for two driving scenarios. The scenarios are implemented and executed using Port, our Map/Reduce framework for Pharo, also introduced in this paper.



中文翻译:

实时大数据应用程序的调试方法

程序员可以使用许多框架来开发和部署大数据应用程序,例如Hadoop Map / Reduce和Apache Spark。但是,这些框架中目前很少提供调试支持。当发生错误时,开发人员将无法从日志文件中提供的信息中了解发生了什么。最近,新的解决方案允许开发人员记录和重播应用程序的执行,但是当需要重新执行数小时的计算时,重播并不总是可以承受的。在本文中,我们提出了一种在线方法,通过该方法,开发人员可以在达到暂停点时将调试会话移至外部进程,从而独立地调试大数据应用程序。我们介绍IDRA MR,这是我们在Pharo中的原型实现。IDRA先生通过引入新颖的调试概念(例如复合调试事件)以及动态更新已调试应用程序的代码和运行框架的相同配置的功能,可以集中化并行应用程序的调试。我们通过调试两种驾驶场景的应用程序和配置故障来验证我们的方法。这些场景是使用Port(我们在Pharo中的Map / Reduce框架)来实现和执行的,本文也对此进行了介绍。

更新日期:2020-04-03
down
wechat
bug