Crafting a Robotic Swarm Pursuit-Evasion Capture Strategy Using Deep Reinforcement Learning

EasyChair Preprint 7352

Abstract

In this paper we study the multi-agent pursuit-evasion problem, and present an extension of the Multi-Agent Deep Deterministic Policy Gradient (MADDPG) deep reinforcement learning algorithm. Previous pursuit-evasion advancements with MADDPG have focused on training capture strategies dependent on the restriction of evader movement with environmental features. We demonstrate a method to train pursuer agents to collaboratively surround and encircle an evader for reliable capture without a strategy rooted in environment entrapment (i.e. cornering). Our method utilizes a novel two-stage, variable-aggression, continuous reward function based on geometrical inscribed circles (incircles), along with a corresponding observation space, with agents operating in an entrapment-disadvantaged environment. Our results show reliable capture of an intelligent, superior evader by three trained pursuers in open space with our encircling strategy. A key novelty of our work is demonstrating the ability to transition behaviors learned using deep reinforcement learning from a simulated robotic system with imperfect world assumptions to a real-world robotic agents.

Keyphrases: MADDPG, Reinforcement Learning, hardware, swarm robotics

@booklet{EasyChair:7352,
  author    = {Charles H. Wu and Donald A. Sofge and Daniel M. Lofaro},
  title     = {Crafting a Robotic Swarm Pursuit-Evasion Capture Strategy Using Deep Reinforcement Learning},
  howpublished = {EasyChair Preprint 7352},
  year      = {EasyChair, 2022}}