Evolutionary Emergence of Distributed Neural Network Controllers in Voxel-Based Soft Robots

This paper explores the evolutionary dynamics of information flow and control strategies in a simulated soft robotic system. By integrating a Genetic Algorithm with Reinforcement Learning, leveraging the Ray RLlib library together with the EvoGym simulation framework, we investigate how structural cost penalization, Lamarckian policy inheritance, and the initial population’s conditions influence the emergence of centralized versus decentralized control architectures, as well as the input activation patterns, in Voxel-Based Soft Robots (VSRs). Our findings demonstrate that introducing a cost based on the distance over which control and input signals propagate significantly alters the search space. In cost-free settings, hybrid centralized/decentralized control architectures and unstructured input preferences tend to emerge. Conversely, the presence of signal propagation costs promotes decentralized control strategies and more selective input usage. Moreover, the cost constraint acts as a regularizer, improving average task performance across the population while limiting the occurrence of exceptionally high-performing individuals; the best task performers overall were achieved in the absence of such costs. We also observed that policy inheritance significantly affects fitness and task performance, while varying the initial population’s conditions has minimal impact on the evolutionary outcomes.