论文标题
通过深厚的增强学习有效的水库管理
Efficient Reservoir Management through Deep Reinforcement Learning
论文作者
论文摘要
大坝通过流动调节和上游下游连接的破坏影响下游河流动力学。但是，由于无法响应上游下游系统的复杂和不确定的动态和储层的各种用法，因此目前的大坝操作远非令人满意的。更进一步的是，不令人满意的大坝操作可能会导致下游地区的洪水。因此，我们利用强化学习（RL）方法来计算这项工作中有效的大坝操作指南。具体而言，我们构建了具有真实数据的离线模拟器和上游流入的不同数学模型，即最低方形（GLS）和动态线性模型（DLM），然后使用模拟器来训练包括DDPG，TD3和SAC在内的最先进的RL RL算法。实验表明，具有DLM的模拟器可以有效地对上游的流入动力学建模，而由RL算法训练的大坝操作策略显着优于人类生成的策略。
Dams impact downstream river dynamics through flow regulation and disruption of upstream-downstream linkages. However, current dam operation is far from satisfactory due to the inability to respond the complicated and uncertain dynamics of the upstream-downstream system and various usages of the reservoir. Even further, the unsatisfactory dam operation can cause floods in downstream areas. Therefore, we leverage reinforcement learning (RL) methods to compute efficient dam operation guidelines in this work. Specifically, we build offline simulators with real data and different mathematical models for the upstream inflow, i.e., generalized least square (GLS) and dynamic linear model (DLM), then use the simulator to train the state-of-the-art RL algorithms, including DDPG, TD3 and SAC. Experiments show that the simulator with DLM can efficiently model the inflow dynamics in the upstream and the dam operation policies trained by RL algorithms significantly outperform the human-generated policy.
