Enhancing Cooperative Multi-Agent Systems With Self-Advice and Near-Neighbor Priority Collision Control

Abstract

[EN] The coordination of actions to be executed by multiple independent agents in a dynamic environment is one of the main challenges of multi-agent systems. To address this type of scenario, a key technology called Reinforcement Learning (RL) has emerged, which enables the training of optimal cooperative policies among agents. However, traditional value decomposition methods suffer fromunstable convergence when the number of agents increases. To address this problem, this article proposes a novel algorithm based on centralized learning that employs a self-advice module to replace the joint action, thereby reducing the algorithmic complexity. The proposed algorithm uses the Joint Action Learning (JAL) concept to find an optimal approach and a collision controller module that was designed to further mitigate the risk of collisions. A comparison of the algorithm proposed is carried out with two benchmark algorithms. The first one focuses on ecomposing the reward signal and the second one trains a different actor-critic network for each agent. Furthermore, multiple target points are defined to enhance cooperative scenarios during the learning process. According to the results, the proposed approach outperforms the two benchmarks by 8% and 49%, thus highlighting the effectiveness of the centralized learning approach in multi-agent systems.