In network-on-chip (NoC) designs, the intellectual property (IP) mapping problem is a critical issue and is usually solved by heuristic searches. However, heuristic searches suffer from the problem of easily falling into the local optimum. To tackle this problem, this article proposes a reinforcement learning-based framework (RLF), which enhances the performance of heuristic searches through the neural network-based probability model. Within this framework, first, a neural network-based probability model for IP mapping is built and trained by reinforcement learning instead of supervised learning to overcome the difficulty of obtaining a high-quality labeled training set. Second, based on the pretrained probability model, the model-based heuristic uses the probability model to generate the initial population and then employs heuristic searches to find the optimal solution. Two model-based heuristics, i.e., the message passing neural network-pointer network-based genetic algorithm (MPN-GA) and the message passing neural network-pointer network-based PSMAP (MPN-PSMAP), are proposed as specific instances. Simulation results show that the MPN-GA reduces the communication cost by an average of 9.32% than the genetic algorithm (GA). The MPN-PSMAP achieves an average reduction in the communication cost of 8.37% than the PSMAP. Finally, two extensions are given as examples to show the good extensibility of this framework.

中文翻译：

---

用于解决 IP 映射问题的基于强化学习的框架

在片上网络 (NoC) 设计中，知识产权 (IP) 映射问题是一个关键问题，通常通过启发式搜索来解决。然而，启发式搜索存在容易陷入局部最优的问题。为了解决这个问题，本文提出了一个基于强化学习的框架（RLF），它通过基于神经网络的概率模型来增强启发式搜索的性能。在这个框架内，首先，通过强化学习代替监督学习，建立和训练基于神经网络的 IP 映射概率模型，以克服获得高质量标记训练集的困难。二、基于预训练的概率模型，基于模型的启发式算法使用概率模型生成初始种群，然后使用启发式搜索找到最优解。两种基于模型的启发式算法，即消息传递神经网络-基于指针网络的遗传算法（MPN-GA）和消息传递神经网络-基于指针网络的PSMAP（MPN-PSMAP），作为具体实例被提出。仿真结果表明，MPN-GA比遗传算法(GA)平均降低了9.32%的通信成本。MPN-PSMAP 实现了比 PSMAP 平均降低 8.37% 的通信成本。最后，给出了两个扩展作为例子，展示了该框架良好的扩展性。提出了基于消息传递神经网络指针网络的遗传算法（MPN-GA）和基于消息传递神经网络指针网络的PSMAP（MPN-PSMAP）作为具体实例。仿真结果表明，MPN-GA比遗传算法(GA)平均降低了9.32%的通信成本。MPN-PSMAP 实现了比 PSMAP 平均降低 8.37% 的通信成本。最后，给出了两个扩展作为例子，展示了该框架良好的可扩展性。提出了基于消息传递神经网络指针网络的遗传算法（MPN-GA）和基于消息传递神经网络指针网络的PSMAP（MPN-PSMAP）作为具体实例。仿真结果表明，MPN-GA比遗传算法(GA)平均降低了9.32%的通信成本。MPN-PSMAP 实现了比 PSMAP 平均降低 8.37% 的通信成本。最后，给出了两个扩展作为例子，展示了该框架良好的扩展性。