Error messages in relational database management systems: A comparison of effectiveness, usefulness, and user confidence☆

https://doi.org/10.1016/j.jss.2021.111034Get rights and content
Under a Creative Commons license
open access

Highlights

  • •

    Database management systems have different error messages.

  • •

    Some systems have more effective error messages than others.

  • •

    Perceived usefulness of error messages might not be reflected on error fixing.

  • •

    SQL compilers, despite their age, are a fresh seedbed for usability studies.

Abstract

The database and the database management system (DBMS) are two of the main components of any information system. Structured Query Language (SQL) is the most popular query language for retrieving data from the database, as well as for many other data management tasks. During system development and maintenance, software developers use a considerable amount of time to interpret compiler error messages. The quality of these error messages has been demonstrated to affect software development effectiveness, and correctly formulating queries and fixing them when needed is an important task for many software developers. In this study, we set out to investigate how participants (N=152) experienced the qualities of error messages of four popular DBMSs in terms of error message effectiveness, perceived usefulness for finding and fixing errors, and error recovery confidence. Our results show differences between the DBMSs by three of the four metrics, and indicate a discrepancy between objective effectiveness and subjective usefulness. The results suggest that although error messages have perceived differences in terms of usefulness for finding and fixing errors, these differences may not necessarily result in differences in query fixing success rates.

Keywords

Structured query language (SQL)
Compiler
Error message
Database management system

Cited by (0)

Toni Taipalus is a researcher and teacher at the Faculty of Information Technology, University of Jyväskylä, Finland. His research has been published in journals such as Journal of Systems and Software and ACM Transactions on Computing Education. His current research interests include query languages, database management systems, computing education, and data analytics.

Hilkka Grahn is a researcher and teacher at the Faculty of Information Technology, University of Jyväskylä, Finland. Her research has been published in journals such as International Journal of Human–Computer Studies and Accident Analysis & Prevention. Her current research interests include human–computer interaction, driver distraction, visual attention, and human factors.

Hadi Ghanbari is a postdoctoral researcher at the Department of Information and Service Management, Aalto University School of Business, Finland. Simultaneously, he is a visiting research fellow at FinEst Twins Smart City Center of Excellence, TalTech, Estonia. He mostly conducts empirical research on sociotechnical aspects of information systems development and digital innovation, with a focus on Technical Debt and information security.

☆

Editor: Kelly Blincoe.

© 2021 The Author(s). Published by Elsevier Inc.