Differential Evolution-Augmented Random Search Reinforcement Learning Algorithm

Abstract

Reinforcement learning (RL) has successfully solved many complex tasks; however, it often suffers from three key issues: insufficient exploration, challenging credit assignment under sparse rewards, and unstable convergence that is highly sensitive to hyperparameters. These limitations reduce the range of problems RL can reliably address. Evolutionary algorithms (EAs) can mitigate these issues, but they typically require many samples and struggle with high-dimensional parameter optimisation. In this work, a hybrid method, Differential Evolution–Augmented Random Search (DE-ARS), is proposed. DE-ARS maintains a population of EA agents and periodically injects an RL agent to replace the weakest individual, thereby supplying gradient-related guidance to the evolutionary search. A comparative analysis was conducted on the performance of DE-ARS and Augmented Random Search (ARS) on continuous control OpenAI Gym benchmarks. The results obtained suggest DE-ARS is a better alternative to ARS, with better stability and improved efficiency.