Reinforcement Learning for Ballbot Balancing

The present work investigates the influence of different simulation modelling choices on the performance and stability of RL-based ballbot balancing. In this study, three modelling paradigms are compared: a commonly used linear planar model, an omniwheel model with explicit spherical rollers, and a capsule-based model with anisotropic friction. For each configuration, a PPO-based controller is trained and evaluated on the same balancing task. The findings indicate that simulation fidelity exerts a substantial influence on learning behavior and control stability. In contrast to the planar and roller-based models, which demonstrate oscillatory or unstable behavior and lack reliable balancing, the anisotropic-friction model facilitates asymptotically stable balancing, ensuring smooth control actions and precise velocity tracking. These findings demonstrate that accurate modelling of the interaction between the wheel and the ground is critical for successful control of the ballbot using RL, and that anisotropic friction provides an effective and computationally tractable solution for stable learning in simulation.