Reinforcement Learning-Based Fleet Dispatching for Greenhouse Gas Emission Reduction in Open-Pit Mining Operations

https://doi.org/10.1016/j.resconrec.2022.106664Get rights and content

Abstract

In typical mining operations, more than half of the direct greenhouse gas (GHG) emissions come from haulage fuel consumption. Smarter truck fleet dispatching is a feasible and manageable solution to reduce direct emissions with existing equipment. Conventional scheduling-based and human-led dispatching solutions often cause lower efficiency that wastes resources and elevates emissions. In this study, a simulated environment is developed to enable testing smarter real-time dispatching systems, Q-learning as a model-free reinforcement learning algorithm is used to improve fleet productivity, decrease waiting time and, consequently, reduce GHG emissions. The proposed algorithm trains the fleet to make better decisions based on payload, traffic, queueing, and maintenance conditions. Results show that this solution can reduce GHG emissions from haulage fuel consumption by over 30% while achieving the same production levels as compared to fixed scheduling. The proposed solution also shows advantages in handling operational randomness and balancing fleet size, productivity, and emissions.

Introduction

Greenhouse gas (GHG) emissions from primary mineral and metal production account for approximately 10% of global energy-related anthropogenic GHG emissions (Azadi et al., 2020; Yokoi et al., 2022). As more governments commit to net-zero emissions by 2030 or 2050 (van Soest et al., 2021), most major mining companies also target significant GHG reduction and carbon neutrality by 2030 or 2050 (e.g., BHP, Newmont, Vale, Anglo American, Glencore). Given these pressing climate targets, the mining industry needs effective emission mitigation strategies and techniques (Azadi et al., 2020; Jiskani et al., 2021; Liu et al., 2021; Shao et al., 2016). One of the primary sources of GHG emissions in open-pit mining operations is the fuel consumed by haul trucks, which accounts for more than 30% of total energy use in surface mines (Siami-Irdemoosa & Dindarloo, 2015). Some heavy trucks can burn more than 250 liters of diesel per hour. A significant portion of the GHG emissions can be avoided if smarter dispatching systems are used to reduce operation mistakes, idling, and queue waiting time for mining trucks (Koryagin & Voronov, 2017; Moradi Afrapoli & Askari-Nasab, 2019).

Queueing and dispatching errors cause significant inefficiency in large mining fleets, and over 40% of the lost time can be eliminated with better dispatching (Koryagin & Voronov, 2017). In their waiting mode, trucks can consume over 10 liters of diesel per hour (Caterpillar Inc., 2010), which generates additional GHG emissions. Most conventional scheduling solutions are planned for a certain scenario where the solution cannot respond in real-time to disruptions such as truck failures and shovel delays, which make these schedules sub-optimal. From a methodology perspective, real time mining fleet management is a research gap. Therefore, our motivation is to explore smarter real-time dispatching solutions to improve productivity and reduce emissions from mining fleets.

Intelligent fleet management is a critical component in the mining sector to mitigate their direct GHG emissions. Studies have suggested that productivity could be significantly improved with an autonomous fleet using intelligent dispatching strategies (Koryagin & Voronov, 2017), which could substantially reduce operating costs and GHG emissions per unit production. Historically, mining fleet dispatching algorithms were developed based on linear programming and queuing theory to maximize production (Li, 1990). More recently, Mixed Integer Linear Programming (MILP) approaches that minimize the delays and costs have become commonly used (Chang et al., 2015; Ta et al., 2013; Topal & Ramazan, 2010; Upadhyay & Askari-Nasab, 2016). Goal programming is another type of algorithm with large versatility, as users can set different goals, and the algorithm finds solutions to minimize deviations from these goals (Temeng et al., 1998). A more recent approach is called Mixed Integer Linear Goal Programming (MILGP), which can reduce deviations from the crushers' desired head grade and tonnage while meeting the production and cost goals (Upadhyay & Askari-Nasab, 2019). Stochastic factors and nonlinearity are also accounted for in some algorithms for various purposes (Moradi Afrapoli & Askari-Nasab, 2019; Ta et al., 2013). However, almost all existing dispatching approaches do not include a direct emission reduction goal. They have limitations in dealing with operational randomness and uncertainties such as ore grade, shovel dig rate, and loading time. They also require substantial work and knowledge to re-optimize when the scenarios or production targets change. Therefore, an easily-customizable and emission-oriented solution is needed to help the mining industry transition to smarter and greener fleet management.

Artificial intelligence (AI) algorithms, particularly reinforcement learning (RL), represent one of the most promising solutions for intelligent fleet management in open-pit mining operations to address the abovementioned issues. RL is a subset of machine learning that trains an agent to make a sequence of correct decisions in a dynamic environment through feedback from current actions and experiences from previous actions. Researchers have successfully used RL algorithms to increase production and deal with unplanned fleet size changes (Zhang et al., 2020); reduce congestion for electric vehicle charging coordination (Tuchnitz et al., 2021); to improve the performance of ambulance dispatch (Liu et al., 2020), and to maximize profit in ride-sharing services (Lin et al., 2018; Wang et al., 2018). Recent studies have also shown that the RL algorithm can help mitigate the expected global warming impact of pavement infrastructure (Renard et al., 2021). However, few studies have adopted this idea in the mining industry, and existing studies have not extensively evaluated the environmental benefits of AI-based smart fleet management systems in mining operations.

This study aims to fill these gaps by quantifying and evaluating the emission reduction potential of an AI-powered intelligent dispatching system in open-pit mining operations using truck-shovel simulations combined with real-time estimates of fleet GHG emissions from haulage fuel consumption. The original contributions of this paper include (1) the set-up of a new open-pit mining haulage simulation environment that allows testing various dispatching approaches, (2) the integration of a real-time GHG emission calculation component into truck-shovel simulations, (3) the application of Q-learning as a proof-of-concept smart dispatching system in this environment to improve productivity and to reduce emissions. We highlight the improvement in efficiency and reduction in emissions after adopting this smart fleet dispatching system, which shows promising applications in mining operations to combat climate change.

Section snippets

Dynamic Fleet Dispatching Problem

Dynamic fleet dispatching in an open-pit mine has two goals: (1) assign active trucks to the right shovels to maximize productivity (Zhang & Xia, 2015), and (2) minimize delays and costs for the fleet while satisfying the demand (Lin et al., 2018). Therefore, the objective for the fleet is to deliver as many materials (ore or waste) from shovels to the correct dumpsites as possible, while maintaining the lowest possible waiting and idle times in the fleet by taking the shortest routes,

Fleet Performance

Fleet performance, including productivity, operational mistakes and queue time, are improved by the RL algorithm. Productivity is measured by the number of correct material deliveries (i.e., ore to mill and waste to dump). Fig.s 3A and 3B compare the delivery performance of the RL approach (AI) and the fixed schedule approach (FS). The AI outperformed the FS for all fleet sizes. The overall production of the fleet (total correct delivery of ore and waste) is improved by 55% (from 235kt to

Emission Reduction Potential of AI Dispatching

Climate change mitigation policies are strengthening across all major industries, and recent studies have emphasized the importance of GHG emissions quantification and transparency in the mining sector (Azadi et al., 2020). Given the pressing climate targets, emission quantification and reduction are expected to become increasingly important for mining companies in the form of taxes and obligations. However, existing literature has disproportionately focused on optimizing the productions and

Conclusions

This study quantifies the fossil-fuel-related GHG emissions from mining fleets and evaluated the emission reduction and production performance of a novel AI-powered fleet dispatching solution. Algorithms are developed using reinforcement learning to manage haul truck fleets at mineral production sites in real time. We draw the following conclusions based on truck-shovel simulations and comparisons with conventional fixed schedule solutions:

  • RL-based dispatching could reduce the GHG emissions per

Declaration of Competing Interest

The authors declare no competing interests.

Acknowledgements

We acknowledge the support from Compute Canada for providing high-performance computing. Qian Zhang would like to thank the support from the Research Initiation Grant provided by Queen's University in Canada and the Discovery Grant (2022) from the Natural Sciences and Engineering Research Council of Canada (NSERC).

References (39)

  • S. Ulrich et al.

    Greenhouse gas emissions and production cost footprints in Australian gold mines

    Journal of Cleaner Production

    (2020)
  • L. Zhang et al.

    An Integer Programming Approach for Truck-Shovel Dispatching Problem in Open-Pit Mines

    Energy Procedia

    (2015)
  • National Greenhouse and Energy Reporting (Measurement) Determination 2008

    (2008)
  • M. Azadi et al.

    Transparency on greenhouse gas emissions from mining to enable climate change mitigation

    Nature Geoscience

    (2020)
  • Caterpillar Performance Handbook

    (2010)
  • Y. Chang et al.

    Modelling and Optimizing an Open-Pit Truck Scheduling Problem

    Discrete Dynamics in Nature and Society

    (2015)
  • G.V. de la Cruz et al.

    Pre-training with non-expert human demonstration for deep reinforcement learning

    The Knowledge Engineering Review

    (2019)
  • Eggleston, S., Buendia, L., Miwa, K., Ngara, T., & Tanabe, K. (2006). 2006 IPCC guidelines for national greenhouse gas...
  • B. Forsman et al.

    Truck dispatch computer simulation in Aitik open pit mine

    International Journal of Surface Mining, Reclamation and Environment

    (1993)
  • Cited by (16)

    View all citing articles on Scopus
    View full text