论文标题
VMA：集体机器人学习的矢量化多代理模拟器
VMAS: A Vectorized Multi-Agent Simulator for Collective Robot Learning
论文作者
论文摘要
虽然许多多机器人协调问题可以通过精确算法来最佳解决，但在机器人的数量中，解决方案通常无法扩展。多代理增强学习（MARL）正在在机器人社区中越来越关注，这是解决此类问题的一种有希望的解决方案。然而，我们仍然缺乏使我们能够快速有效地找到大规模集体学习任务的解决方案的工具。在这项工作中，我们介绍了矢量化的多代理模拟器（VMA）。 VMA是一个旨在有效的Marl基准测试的开源框架。它由用Pytorch编写的矢量化2D物理引擎和一组十二个具有挑战性的多机器人方案组成。可以通过简单的模块化接口实现其他方案。我们演示了矢量化如何在没有增加复杂性的情况下对加速硬件进行并行模拟。在将VMA与OpenAI MPE进行比较时，我们显示了MPE的执行时间如何在模拟数量中线性增加，而VMA可以在10s以下执行30,000个并行模拟，证明更快的速度超过100倍。使用VMA的RLLIB接口，我们使用基于各种近端策略优化（PPO）的MARL算法对多机器人方案进行基准测试。 VMA的场景以正交方式证明了最先进的MARL算法。 VMA框架可在https://github.com/proroklab/dectorizedmultiagentsimulator上获得。 VMA场景和实验的视频可在https://youtu.be/aadryfiesay上找到。
While many multi-robot coordination problems can be solved optimally by exact algorithms, solutions are often not scalable in the number of robots. Multi-Agent Reinforcement Learning (MARL) is gaining increasing attention in the robotics community as a promising solution to tackle such problems. Nevertheless, we still lack the tools that allow us to quickly and efficiently find solutions to large-scale collective learning tasks. In this work, we introduce the Vectorized Multi-Agent Simulator (VMAS). VMAS is an open-source framework designed for efficient MARL benchmarking. It is comprised of a vectorized 2D physics engine written in PyTorch and a set of twelve challenging multi-robot scenarios. Additional scenarios can be implemented through a simple and modular interface. We demonstrate how vectorization enables parallel simulation on accelerated hardware without added complexity. When comparing VMAS to OpenAI MPE, we show how MPE's execution time increases linearly in the number of simulations while VMAS is able to execute 30,000 parallel simulations in under 10s, proving more than 100x faster. Using VMAS's RLlib interface, we benchmark our multi-robot scenarios using various Proximal Policy Optimization (PPO)-based MARL algorithms. VMAS's scenarios prove challenging in orthogonal ways for state-of-the-art MARL algorithms. The VMAS framework is available at https://github.com/proroklab/VectorizedMultiAgentSimulator. A video of VMAS scenarios and experiments is available at https://youtu.be/aaDRYfiesAY.
