论文标题
在混合自主流量中的合作决策：框架，调查和挑战
Graph Reinforcement Learning Application to Co-operative Decision-Making in Mixed Autonomy Traffic: Framework, Survey, and Challenges
论文作者
论文摘要
连接和自动化车辆（CAV）的正确运行对于未来智能运输系统的安全性和效率至关重要。同时，过渡到完全自主驾驶需要长时间的混合自主流量，包括骑士和人类驱动的车辆。因此，为骑士的协作决策对于产生适当的驾驶行为至关重要，以提高混合自主流量的安全性和效率。近年来，深入的强化学习（DRL）已被广泛用于解决决策问题。但是，现有的基于DRL的方法主要集中于解决单个CAV的决策。在混合自主流量中使用现有的基于DRL的方法不能准确代表车辆和模型动态交通环境的相互影响。为了解决这些缺点，本文提出了一种图形增强学习（GRL）方法，用于在混合自主流量中对骑士的多代理决策。首先，设计了一个通用和模块化的GRL框架。然后，提出了对DRL和GRL方法的系统综述，重点介绍了最近的研究中解决的问题。此外，基于设计的框架进一步提出了关于不同GRL方法的比较研究，以验证GRL方法的有效性。结果表明，与DRL方法相比，GRL方法可以很好地优化混合自主流量中骑士的多代理决策的性能。最后，总结了挑战和未来的研究方向。这项研究可以提供有价值的研究参考，以解决混合自主流量中骑士的多代理决策问题，并可以促进基于GRL的方法在智能运输系统中实施。我们工作的源代码可以在https://github.com/jacklinkk/graph_cavs上找到。
Proper functioning of connected and automated vehicles (CAVs) is crucial for the safety and efficiency of future intelligent transport systems. Meanwhile, transitioning to fully autonomous driving requires a long period of mixed autonomy traffic, including both CAVs and human-driven vehicles. Thus, collaboration decision-making for CAVs is essential to generate appropriate driving behaviors to enhance the safety and efficiency of mixed autonomy traffic. In recent years, deep reinforcement learning (DRL) has been widely used in solving decision-making problems. However, the existing DRL-based methods have been mainly focused on solving the decision-making of a single CAV. Using the existing DRL-based methods in mixed autonomy traffic cannot accurately represent the mutual effects of vehicles and model dynamic traffic environments. To address these shortcomings, this article proposes a graph reinforcement learning (GRL) approach for multi-agent decision-making of CAVs in mixed autonomy traffic. First, a generic and modular GRL framework is designed. Then, a systematic review of DRL and GRL methods is presented, focusing on the problems addressed in recent research. Moreover, a comparative study on different GRL methods is further proposed based on the designed framework to verify the effectiveness of GRL methods. Results show that the GRL methods can well optimize the performance of multi-agent decision-making for CAVs in mixed autonomy traffic compared to the DRL methods. Finally, challenges and future research directions are summarized. This study can provide a valuable research reference for solving the multi-agent decision-making problems of CAVs in mixed autonomy traffic and can promote the implementation of GRL-based methods into intelligent transportation systems. The source code of our work can be found at https://github.com/Jacklinkk/Graph_CAVs.
