Toward multi-target self-organizing pursuit in a partially observable Markov game

Sun，Lijun1,2; Chang，Yu Cheng2; Lyu，Chao3; Shi，Ye4; Shi，Yuhui1; Lin，Chin Teng2

2023-11-01

10.1016/j.ins.2023.119475

Information Sciences   Impact Factor & Quartile

0020-0255

The multiple-target self-organizing pursuit (SOP) problem has wide applications and has been considered a challenging self-organization game for distributed systems, in which intelligent agents cooperatively pursue multiple dynamic targets with partial observations. This work proposes a framework for decentralized multi-agent systems to improve the implicit coordination capabilities in search and pursuit. We model a self-organizing system as a partially observable Markov game (POMG) featured by large-scale, decentralization, partial observation, and noncommunication. The proposed distributed algorithm–fuzzy self-organizing cooperative coevolution (FSC2) is then leveraged to resolve the three challenges in multi-target SOP: distributed self-organizing search (SOS), distributed task allocation, and distributed single-target pursuit. FSC2 includes a coordinated multi-agent deep reinforcement learning (MARL) method that enables homogeneous agents to learn natural SOS patterns. Additionally, we propose a fuzzy-based distributed task allocation method, which locally decomposes multi-target SOP into several single-target pursuit problems. The cooperative coevolution principle is employed to coordinate distributed pursuers for each single-target pursuit problem. Therefore, the uncertainties of inherent partial observation and distributed decision-making in the POMG can be alleviated. The experimental results demonstrate that by decomposing the SOP task, FSC2 achieves superior performance compared with other implicit coordination policies fully trained by general MARL algorithms. The scalability of FSC2 is proved that up to 2048 FSC2 agents perform efficient multi-target SOP with almost 100% capture rates. Empirical analyses and ablation studies verify the interpretability, rationality, and effectiveness of component algorithms in FSC2.

Interaction uncertainty Multi-target pursuit Noncommunication Observation uncertainty Self-organization

SCI

English

First ; Corresponding

National Natural Science Foundation of China[61761136008];Australian Research Council[DP210101093];Australian Research Council[DP220100803];Shenzhen Fundamental Research Program[JCYJ20200109141235597];Shenzhen Peacock Plan[KQTD2016112514355531];

WOS:001069320000001

COMPUTER SCIENCE

2-s2.0-85168751730

Scopus

1.Guangdong Provincial Key Laboratory of Brain-inspired Intelligent Computation,Department of Computer Science and Engineering,Southern University of Science and Technology,China
2.Centre for Artificial Intelligence,CIBCI Lab,Faculty of Engineering and Information Technology,University of Technology Sydney,Australia
3.College of Computer and Information Science,Southwest University,China
4.School of Information Science and Technology,ShanghaiTech University,China

Sun，Lijun,Chang，Yu Cheng,Lyu，Chao,et al. Toward multi-target self-organizing pursuit in a partially observable Markov game[J]. Information Sciences,2023,648.

Sun，Lijun,Chang，Yu Cheng,Lyu，Chao,Shi，Ye,Shi，Yuhui,&Lin，Chin Teng.(2023).Toward multi-target self-organizing pursuit in a partially observable Markov game.Information Sciences,648.

Sun，Lijun,et al."Toward multi-target self-organizing pursuit in a partially observable Markov game".Information Sciences 648(2023).