This research gauges the ability of deep reinforcement learning (DRL) techniques to assist the control of conjugate heat transfer systems governed by the coupled Navier–Stokes and heat equations. It uses a novel, “degenerate” version of the proximal policy optimization (PPO) algorithm, intended for situations where the optimal policy to be learnt by a neural network does not depend on state, as is notably the case in optimization and open-loop control problems. The numerical reward fed to the neural network is computed with an in-house stabilized finite elements environment combining variational multi-scale (VMS) modeling of the governing equations, immerse volume method, and multi-component anisotropic mesh adaptation. Several test cases of natural and forced convection in two and three dimensions are used as testbed for developing the methodology. The approach successfully alleviates the natural convection induced enhancement of heat transfer in a two-dimensional, differentially heated square cavity controlled by piece-wise constant fluctuations of the sidewall temperature. It also proves capable of improving the homogeneity of temperature across the surface of two and three-dimensional hot workpieces under impingement cooling. Various cases are tackled, in which the position of multiple cold air injectors is optimized relative to a fixed workpiece position. The flexibility of the numerical framework makes it tractable to solve also the inverse problem, i.e., to optimize the workpiece position relative to a fixed injector distribution. The obtained results showcase the potential of the method for black-box optimization of practically meaningful computational fluid dynamics (CFD) conjugate heat transfer systems. More significantly, they stress how DRL can reveal unanticipated solutions or parameter relations (as the optimal workpiece position under symmetrical actuation turns to be offset from the symmetry axis), in addition to being a tool for optimizing searches in large parameter spaces.

这项研究评估了深度强化学习（DRL）技术的能力，以协助控制由Navier–Stokes和热方程耦合控制的共轭传热系统。它使用了一种新颖的“简并”版本的近端策略优化（PPO）算法，适用于神经网络要学习的最优策略不依赖状态的情况，特别是在优化和开环情况下控制问题。在内部稳定的有限元环境中，结合控制方程的变分多尺度（VMS）建模，沉浸体积法和多分量各向异性网格自适应，可以计算出馈入神经网络的数值报酬。使用二维和三维自然对流的几个测试案例作为开发该方法的测试平台。该方法成功地缓解了自然对流引起的二维受热方形腔中传热的增强，该二维腔由侧壁温度的逐段恒定波动控制。它也被证明能够在冲击冷却下改善二维和三维热工件表面温度的均匀性。解决了各种情况，其中相对于固定的工件位置优化了多个冷空气喷射器的位置。数字框架的灵活性使其易于解决反问题，即相对于固定的喷射器分布优化工件位置。获得的结果表明，该方法可用于对实际有意义的计算流体动力学（CFD）共轭传热系统进行黑盒优化。更重要的是，他们强调DRL不仅可以优化大型参数空间中的搜索，还可以揭示意外的解决方案或参数关系（因为对称驱动下的最佳工件位置会偏离对称轴）。