One of the most crucial challenges faced by the Li-ion battery community concerns the search for the minimum time charging without damaging the cells. This goal can be achieved by solving a large-scale constrained optimal control problem, which relies on accurate electrochemical models. However, these models are limited by their high computational cost, as well as identifiability and observability issues. As an alternative, simple output-feedback algorithms can be employed, but their performance strictly depends on trial and error tuning. Moreover, particular techniques have to be adopted to handle safety constraints. With the aim of overcoming these limitations, we propose an optimal-charging procedure based on deep reinforcement learning. In particular, we focus on a policy gradient method to cope with continuous sets of states and actions. First, we assume full state measurements from the Doyle–Fuller–Newman (DFN) model, which is projected to a lower dimensional feature space via the principal component analysis. Subsequently, this assumption is removed, and only output measurements are considered as the agent observations. Finally, we show the adaptability of the proposed policy to changes in the environment’s parameters. The results are compared with other methodologies presented in the literature, such as the reference governor and the proportional–integral–derivative approach.

中文翻译：

---

用于锂离子电池快速充电的深度强化学习框架


锂离子电池社区面临的最关键挑战之一是寻求不损坏电池的最短充电时间。这一目标可以通过解决大规模约束最优控制问题来实现，该问题依赖于精确的电化学模型。然而，这些模型受到高计算成本以及可识别性和可观测性问题的限制。作为替代方案，可以采用简单的输出反馈算法，但其性能严格取决于试错调整。此外，必须采用特殊技术来处理安全约束。为了克服这些限制，我们提出了一种基于深度强化学习的最佳充电程序。特别是，我们专注于策略梯度方法来应对连续的状态和动作集。首先，我们假设来自 Doyle–Fuller–Newman (DFN) 模型的全状态测量，该模型通过主成分分析投影到较低维的特征空间。随后，该假设被删除，并且仅输出测量被视为代理观察。最后，我们展示了所提出的策略对环境参数变化的适应性。将结果与文献中提出的其他方法进行比较，例如参考调节​​器和比例积分微分方法。