Homotopic reinforcement learning for distributed consensus control of stochastic Markov jump multi-agent systems

Abstract

This paper investigates the optimized consensus problem for a class of stochastic Markov jump multi-agent systems, with particular attention to the high computational demands that arise in large-scale implementations. Firstly, an error system is constructed based on the consensus objective, and a min-max strategy is introduced to transform the consensus problem into an optimized control problem of the error system. Subsequently, a set of parallel coupled game Lyapunov equations are developed to design the consensus controller, whose solution naturally requires significant parallel computation resources. Furthermore, to address the challenges posed by unknown system dynamics and the difficulty of obtaining an initial stable controller, a novel model-free consensus control approach based on homotopic reinforcement learning is proposed. By collecting state and input data, the proposed method enables the online computation of closed-loop stable controllers and optimized consensus controllers in a scalable and distributed manner. Finally, a numerical example is presented to demonstrate the effectiveness of the proposed approach, highlighting its suitability for real-time implementation on high-performance computing platforms.