Reinforcement learning is an efficient, widely used machine learning technique that performs well when the state and action spaces have a reasonable size. This is rarely the case regarding control-related problems, as for instance controlling traffic signals. Here, the state space can be very large. In order to deal with the curse of dimensionality, a rough discretization of such space can be employed. However, this is effective just up to a certain point. A way to mitigate this is to use techniques that generalize the state space such as function approximation. In this paper, a linear function approximation is used. Specifically, SARSA(λ) with Fourier basis features is implemented to control traffic signals in the agent-based transport simulation MATSim. The results are compared not only to trivial controllers such as fixed-time, but also to state-of-the-art rule-based adaptive methods. It is concluded that SARSA(λ) with Fourier basis features is able to outperform such methods, especially in scenarios with varying traffic demands or unexpected events.

中文翻译：

---

强化学习与基于规则的自适应交通信号控制：交通信号控制的傅立叶线性函数逼近

强化学习是一种有效且广泛使用的机器学习技术，当状态空间和动作空间的大小合理时，它会表现良好。对于与控制有关的问题，例如控制交通信号灯，很少出现这种情况。在这里，状态空间可能非常大。为了应对维数的诅咒，可以采用这种空间的粗略离散化。但是，这仅在一定程度上有效。减轻这种情况的一种方法是使用概括状态空间的技术，例如函数逼近。在本文中，使用线性函数近似。具体而言，具有傅立叶基础特征的SARSA（λ）实现为在基于代理的传输模拟MATSim中控制交通信号。不仅将结果与诸如固定时间，而且还涉及基于规则的最新自适应方法。结论是具有傅立叶基础特征的SARSA（λ）能够胜过此类方法，尤其是在流量需求变化或突发事件的情况下。