论文标题
深厚的增强学习依靠V2X信息的排练控制
Deep Reinforcement Learning Aided Platoon Control Relying on V2X Information
论文作者
论文摘要
研究了车辆到全能（V2X）通信对排控制性能的影响。排控制本质上是一个顺序的随机决策问题（SSDP），可以通过深度加强学习（DRL）来解决该排列车的控制限制和不确定性。在这种情况下，研究了V2X通信对基于DRL的排基管控制器的价值，重点是在系统状态中包括外源信息的增益，以减少不确定性的外源信息和由于数量的诅咒而导致的性能侵蚀。我们的目标是找到应在车辆之间共享的特定信息集，以建造最合适的状态空间。通过考虑“足够的”信息，通过在不同信息拓扑（IFT）下对排控制的SSDP模型进行了控制。此外，还建立了定理以比较其最佳策略的性能。为了确定是否应传输一条信息以改善基于DRL的控制策略，我们通过得出过渡模型的条件KL差异来量化其值。在传输方面，更高的优先级具有更高的优先级，因为在状态空间中包括较高的状态维度的负面影响的可能性更高。最后，提供了模拟结果以说明理论分析。
The impact of Vehicle-to-Everything (V2X) communications on platoon control performance is investigated. Platoon control is essentially a sequential stochastic decision problem (SSDP), which can be solved by Deep Reinforcement Learning (DRL) to deal with both the control constraints and uncertainty in the platoon leading vehicle's behavior. In this context, the value of V2X communications for DRL-based platoon controllers is studied with an emphasis on the tradeoff between the gain of including exogenous information in the system state for reducing uncertainty and the performance erosion due to the curse-of-dimensionality. Our objective is to find the specific set of information that should be shared among the vehicles for the construction of the most appropriate state space. SSDP models are conceived for platoon control under different information topologies (IFT) by taking into account `just sufficient' information. Furthermore, theorems are established for comparing the performance of their optimal policies. In order to determine whether a piece of information should or should not be transmitted for improving the DRL-based control policy, we quantify its value by deriving the conditional KL divergence of the transition models. More meritorious information is given higher priority in transmission, since including it in the state space has a higher probability in offsetting the negative effect of having higher state dimensions. Finally, simulation results are provided to illustrate the theoretical analysis.
