Abstract
Optimizing maintenance practices is a continuous process that must take into account the evolving state of the equipment, resources, workers, and more. To help streamline this process, facilities need a concise procedure for identifying critical tasks and assets that have major impact on the performance of maintenance activities. This work provides a process for making data investigations more effective by discovering influential equipment, actions, and other environmental factors from tacit knowledge within maintenance documents and reports. Traditional application of text analysis focuses on prediction and modeling of system state directly. Variation in domain data, quality, and managerial expectations prevent the creation of a generic method to do this with real industrial data. Instead, text analysis techniques can be applied to discover key factors within a system, which function as indicators for further, in-depth analysis. These factors can point investigators where to find good or bad behaviors, but do not explicitly perform any anomaly detection. This paper details an adaptable procedure tailored to maintenance and industrial settings for determining important named entities within natural language documents. The procedure in this paper utilizes natural language processing techniques to extract these terms or concepts from maintenance work orders and measure their influence on Key Performance Indicators (KPIs) as defined by managers and decision makers. We present a case study to demonstrate the developed workflow (algorithmic procedure) to identify terms associated with concepts or systems which have strong relationships with a selected KPI, such as time or cost. This proof of concept uses the length of time a Maintenance Work Order (MWO) remains open from creation to completion as the relevant performance indicator. By identifying tasks, assets, and environments that have significant relevance to KPIs, planners and decision makers can more easily direct investigations to identify problem areas within a facility, better allocate resources, and guide more effective analysis for both monitoring and improving a facility. The output of the analysis workflow presented in this paper is not intended as a direct indicator of good or bad practices and assets, but instead is intended to be used to help direct and improve the effectiveness of investigations determining those. This workflow provides a preparatory investigation that both conditions the data, helps guide investigators into more productive and effective investigations of the latent information contained in human generated work logs, specifically the natural language recorded in MWOs. When this information preparing and gathering procedure is used in conjunction with other tacit knowledge or analysis tools it gives a more full picture of the efficiency and effectiveness of maintenance strategies. When properly applied, this methodology can identify pain points, highlight anomalous patterns, or verify expected outcomes of a facility’s maintenance strategy.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bg, I. (2007). Bayesian networks. In F. Ruggeri, R. S. Kennett, & F. W. Faltin (Eds.), Encyclopedia of statistics in quality and reliability. Hoboken: Wiley.
Bokinsky, H., McKenzie, A., Bayoumi, A., McCaslin, R., Patterson, A., Matthews, M., et al. (2013). Application of natural language processing techniques to marine v–22 maintenance data for populating a cbm-oriented database. AHS Airworthiness, CBM, and HUMS Specialists Meeting (pp. 463–472).
Borovicka, T., Jirina, M., Jr., Kordik, P., & Jirina, M. (2012). Selecting representative data sets. Advances in Data Mining Knowledge Discovery and Applications, 12, 43–70.
Brundage, M.P., Morris, K., Sexton, T., Moccozet, S., & Hoffman, M. (2018). Developing maintenance key performance indicators from maintenance work order data. In ASME 2018 13th International Manufacturing Science and Engineering Conference, American Society of Mechanical Engineers, pp V003t02a027–v003t02a027
Buckland, M., & Gey, F. (1994). The relationship between recall and precision. Journal of the American Society for Information Science, 45(1), 12–19.
Carvalho, T. P., Soares, F. A., Vita, R., Francisco, Rd. P., Basto, J. P., & Alcalá, S. G. (2019). A systematic literature review of machine learning methods applied to predictive maintenance. Computers & Industrial Engineering. https://doi.org/10.1016/j.cie.2019.106024.
Cox, D. R. (1972). Regression models and life-tables. Journal of the Royal Statistical Society Series B (Methodological), 34(2), 187–220.
Devlin, J., Chang, M.W., Lee, K., & Toutanova, K. (2019). Bert: Pre-training of deep bidirectional transformers for language understanding arXiv:1810.04805.
Fahrmeir, L., Kneib, T., Lang, S., & Marx, B. (2013). Regression models. In: Regression, vol 1, Springer, pp 21–72.
Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., & Pedreschi, D. (2018). A survey of methods for explaining black box models. ACM Computing Surveys (CSUR), 51(5), 1–42.
Harris, D., & Harris, S. (2012). Digital design and computer architecture (2nd ed.). San Francisco: Morgan Kaufmann.
Hodkiewicz, M., & Ho, M. T. W. (2016). Cleaning historical maintenance work order data for reliability analysis. Journal of Quality in Maintenance Engineering, 22(2), 146–163.
Iso. (2014). Automation systems and integration – key performance indicators (kpis) for manufacturing operations management – part 1: Overview, concepts and terminology. Tech. Rep. Iso22400-1, International Organization for Standardization.
Jin, X., Weiss, B. A., Siegel, D., & Lee, J. (2016). Present status and future growth of advanced maintenance technology and strategy in us manufacturing. International journal of prognostics and health management. https://doi.org/10.1051/mfreview/2016005.
Li, L., Wang, Y., & Lin, K. Y. (2021). Preventive maintenance scheduling optimization based on opportunistic production-maintenance synchronization. Journal of Intelligent Manufacturing, 32(2), 545–558.
Lovins, J. B. (1968). Development of a stemming algorithm. Mech Translat & Comp Linguistics, 11(1–2), 22–31.
Lukens, S., Naik, M., Saetia, K., & Hu, X. (2019). Best practices framework for improving maintenance data quality to enable asset performance analytics. In Annual Conference of the PHM Society, vol 11.
Meseroll, R.J., Kirkos, C.J., & Shannon, R.A. (2007). Data mining navy flight and maintenance data to affect repair. In: 2007 IEEE Autotestcon, pp 476–481, https://doi.org/10.1109/AUTEST.2007.4374256.
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient estimation of word representations in vector space. arXiv:1301.3781.
Mukherjee, S., & Chakraborty, A. (2007). Automated fault tree generation: Bridging reliability with text mining. In 2007 Annual Reliability and Maintainability Symposium, pp 83–88.
Nalepa, J., & Kawulok, M. (2019). Selecting training sets for support vector machines: A review. Artificial Intelligence Review, 52(2), 857–900. https://doi.org/10.1007/s10462-017-9611-1.
Nembrini, S., König, I. R., & Wright, M. N. (2018). The revival of the gini importance? Bioinformatics, 34(21), 3711–3718. https://doi.org/10.1093/bioinformatics/bty373.
Oliphant, T., Peterson, P., & Jones, E. (2013). Scipy. http://github.com/scipy/scipy/.
Sakib, N., & Wuest, T. (2018). Challenges and opportunities of condition-based predictive maintenance: A review. Procedia CIRP, 78, 267–272.
Saltelli, A. (2002). Sensitivity analysis for importance assessment. Risk Analysis, 22(3), 579–590. https://doi.org/10.1111/0272-4332.00040.
Savolainen, J., & Urbani, M. (2021). Maintenance optimization for a multi-unit system with digital twin simulation. Journal of Intelligent Manufacturing. https://doi.org/10.1007/s10845-021-01740-z.
Seale, M., Hines, A., Nabholz, G., Ruvinsky, A., Eslinger, O., Rigoni, N., Vega-Maisonet, L. (2019). Approaches for using machine learning algorithms with large label sets for rotorcraft maintenance. In 2019 IEEE Aerospace Conference, pp 1–8, https://doi.org/10.1109/AERO.2019.8742027.
Sexton, T., & Brundage, M. (2019). Nestor: A tool for natural language annotation of short texts.
Sexton, T., Brundage, M.P., Hoffman, M., & Morris, K.C. (2017). Hybrid datafication of maintenance logs from ai-assisted human tags. In: Big Data (Big Data), 2017 IEEE International Conference on, Ieee, pp 1769–1777.
Sexton, T., Hodkiewicz, M., & Brundage, M. (2019). Categorization errors for data entry in maintenance work-orders. In Proceedings of the Annual Conference of the PHM Society, Scottsdale, AZ, https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=928437.
Sexton, T., Hodkiewicz, M., Brundage, M.P., & Smoker, T. (2018). Benchmarking for keyword extraction methodologies in maintenance work orders. In Proceedings of the Annual Conference of the PHM Society, 10, https://doi.org/10.36001/phmconf.2018.v10i1.541.
Sharp, M. (2019). Observations on developing reliability information utilization in a manufacturing environment with case study: Robotic arm manipulators. The International Journal of Advanced Manufacturing Technology, 102, 3243–3264. https://doi.org/10.1007/s00170-018-03263-z.
Sharp, M., Sexton, T., & Brundage, M. P. (2017). Toward semi-autonomous information. In H. Lödding, R. Riedel, K. D. Thoben, G. von Cieminski, & D. Kiritsis (Eds.), Advances in Production Management Systems (pp. 425–432). The Path to Intelligent: Collaborative and Sustainable Manufacturing, Springer International Publishing, Cham.
Sipos, R., Fradkin, D., Moerchen, F., & Wang, Z. (2014). Log-based predictive maintenance. In Proceedings of the 20th ACM SIGKDD international conference on knowledge discovery and data mining, Acm, pp 1867–1876.
SMRP. (2017). Society of Maintenance and Reliability Professionals (SMRP) Best Practices (5th ed.). Standard: Society for Maintenance and Reliability Professionals, Atlanta, GA.
Swanson, L. (2001). Linking maintenance strategies to performance. International Journal of Production Economics, 70(3), 237–244. https://doi.org/10.1016/S0925-5273(00)00067-0.
Szpytko, J., & Duarte, Y. S. (2020). A digital twins concept model for integrated maintenance: A case study for crane operation. Journal of Intelligent Manufacturing. https://doi.org/10.1007/s10845-020-01689-5.
Thomas, D. S., & Weiss, B. A. (2020). Economics of manufacturing machinery maintenance: A survey and analysis of us costs and benefits.https://doi.org/10.6028/NIST.AMS.100-34.
Zhang, Y., Jin, R., & Zhou, Z. H. (2010). Understanding bag-of-words model: A statistical framework. International Journal of Machine Learning and Cybernetics, 1(1–4), 43–52.
Zhong, R. Y., Xu, X., Klotz, E., & Newman, S. T. (2017). Intelligent manufacturing in the context of industry 4.0: A review. Engineering, 3(5), 616–630. https://doi.org/10.1016/J.ENG.2017.05.015.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Ethical approval
The use of any products described in this paper does not imply recommendation or endorsement by the National Institute of Standards and Technology, nor does it imply that products are necessarily the best available for the purpose.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This material is declared a work of the U.S. Government and is not subject to copyright protection in the United States.
Approved for public release; distribution is unlimited.
Rights and permissions
About this article
Cite this article
Navinchandran, M., Sharp, M.E., Brundage, M.P. et al. Discovering critical KPI factors from natural language in maintenance work orders. J Intell Manuf 33, 1859–1877 (2022). https://doi.org/10.1007/s10845-021-01772-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10845-021-01772-5