A major goal of materials design is to find material structures with desired properties and in a second step to find a processing path to reach one of these structures. In this paper, we propose and investigate a deep reinforcement learning approach for the optimization of processing paths. The goal is to find optimal processing paths in the material structure space that lead to target-structures, which have been identified beforehand to result in desired material properties. There exists a target set containing one or multiple different structures, bearing the desired properties. Our proposed methods can find an optimal path from a start structure to a single target structure, or optimize the processing paths to one of the equivalent target-structures in the set. In the latter case, the algorithm learns during processing to simultaneously identify the best reachable target structure and the optimal path to it. The proposed methods belong to the family of model-free deep reinforcement learning algorithms. They are guided by structure representations as features of the process state and by a reward signal, which is formulated based on a distance function in the structure space. Model-free reinforcement learning algorithms learn through trial and error while interacting with the process. Thereby, they are not restricted to information from a priori sampled processing data and are able to adapt to the specific process. The optimization itself is model-free and does not require any prior knowledge about the process itself. We instantiate and evaluate the proposed methods by optimizing paths of a generic metal forming process. We show the ability of both methods to find processing paths leading close to target structures and the ability of the extended method to identify target-structures that can be reached effectively and efficiently and to focus on these targets for sample efficient processing path optimization.

材料设计的一个主要目标是找到具有所需特性的材料结构，然后在第二步中找到达到这些结构之一的加工路径。在本文中，我们提出并研究了一种用于优化处理路径的深度强化学习方法。目标是在材料结构空间中找到导致目标结构的最佳处理路径，这些目标结构已预先确定以产生所需的材料特性。存在一个包含一个或多个不同结构的目标集，具有所需的特性。我们提出的方法可以找到从起始结构到单个目标结构的最佳路径，或优化到集合中等效目标结构之一的处理路径。在后一种情况下，该算法在处理过程中学习以同时识别最佳可达目标结构和最佳路径。所提出的方法属于无模型深度强化学习算法系列。它们由作为过程状态特征的结构表示和基于结构空间中的距离函数制定的奖励信号引导。无模型强化学习算法在与过程交互的同时通过反复试验来学习。因此，它们不限于来自先验采样处理数据的信息，并且能够适应特定过程。优化本身是无模型的，不需要任何关于过程本身的先验知识。我们通过优化通用金属成形过程的路径来实例化和评估所提出的方法。