Flooding in coastal cities is increasing due to climate change and sea-level rise, stressing the traditional stormwater systems these communities rely on. Automated real-time control (RTC) of these systems can improve performance, and creating control policies for smart stormwater systems is an active area of study. This research explores reinforcement learning (RL) to create control policies to mitigate flood risk. RL is trained using a model of hypothetical urban catchments with a tidal boundary and two retention ponds with controllable valves. RL’s performance is compared to the passive system, a model predictive control (MPC) strategy, and a rule-based control strategy (RBC). RL learns to proactively manage pond levels using current and forecast conditions and reduced flooding by 32% over the passive system. Compared to the MPC approach using a physics-based model and genetic algorithm, RL achieved nearly the same flood reduction, just 3% less than MPC, with a significant 88× speedup in runtime. Compared to RBC, RL was able to quickly learn similar control strategies and reduced flooding by an additional 19%. This research demonstrates that RL can effectively control a simple system and offers a computationally efficient method that could scale to RTC of more complex stormwater systems.

中文翻译：

---

使用实时雨水基础设施控制和强化学习减轻沿海城市集水区的洪水

由于气候变化和海平面上升，沿海城市的洪水正在增加，给这些社区依赖的传统雨水系统带来压力。这些系统的自动实时控制 (RTC) 可以提高性能，为智能雨水系统创建控制策略是一个活跃的研究领域。本研究探索强化学习 (RL) 以制定控制政策以减轻洪水风险。RL 使用具有潮汐边界和两个带有可控阀门的蓄水池的假设城市集水区模型进行训练。RL 的性能与被动系统、模型预测控制 (MPC) 策略和基于规则的控制策略 (RBC) 进行比较。RL 学会了使用当前和预测条件主动管理池塘水位，并通过被动系统将洪水减少 32%。与使用基于物理模型和遗传算法的 MPC 方法相比，RL 实现了几乎相同的洪水减少，仅比 MPC 少 3%，运行时显着提高了 88 倍。与 RBC 相比，RL 能够快速学习类似的控制策略，并将洪水减少了 19%。这项研究表明，RL 可以有效地控制一个简单的系统，并提供一种计算效率高的方法，可以扩展到更复杂的雨水系统的 RTC。