Reinforcement Learning-Based Control Strategies for a‬ ‭Ballbot System‬

Dynamically unstable robots operating in human-centered environments must achieve not only high performance but also robustness to failures such as loss of balance and falls. While much research has focused on preventing such failures, comparatively less attention has been devoted to controlling their consequences, particularly in terms of minimizing damage to the robot during a fall. This challenge is especially pronounced for ballbots, which balance on a single spherical wheel and exhibit inherent dynamic instability combined with limited mechanical means to absorb impacts. This dissertation investigates reinforcement learning–based control strategies for improving the resilience of ballbot robots, with a focus on damage minimization during falls and stable balancing behavior. First, the influence of reward function design and the ratio between controller and simulator frequencies is systematically analyzed, revealing their critical role in achieving stable learning and reducing impact forces across varying fall scenarios. Second, a multi-policy learning approach based on clustering is introduced to address high variability in initial configurations, demonstrating improved robustness and damage reduction compared to single-policy and curriculum learning baselines. Finally, the impact of modeling choices on reinforcement-learning-based ballbot balancing is examined through a comparative study of different simulation models, highlighting how model fidelity affects learning stability and control performance. Overall, this work provides empirical insights into the interplay between learning configuration, policy structure, and modeling assumptions in reinforcement learning for dynamically unstable robots. The findings contribute to the design of safer and more reliable learning-based control strategies for ballbots and offer guidance applicable to a broader class of unstable robotic platforms.