Standing Balance

Reinforcement Learning Controller for Lower Limb Standing Balance

This project is selected for an oral presentation at the Canadian Society of Biomechanics (CSB) 2024 at the Balance and Perturbation Session.

MyoLeg undergoing perturbation in the AP direction in healthy (left) and sarcopenia (right) cases.

Fall-related injuries due to loss of balance are detrimental to the health, independence, and quality of life among the elders (e.g. hip fractures, traumatic brain injuries). The issue is highlighted by the onset of sarcopenia in the lower limbs, an age-related condition of muscle loss that heightens the risk of falls. Our research aims to quantify the impact of sarcopenia on an individual’s ability to remain stable under perturbation during bipedal stance through the advanced musculoskeletal simulation, MyoSuite.

Sarcopenia condition: decrease in muscle volume and coordination due to aging contributes to falling.

The purpose of this study is to understanding if reinforcement learning can reproduce motions of human standing balance under sarcopenia condition. We proposed that under sarcopenia condition, baseline stability policy would lead to higher falling rates and shorter standing time under perturbation than in healthy subject.

MyoLeg is a lower limb musculoskeletal model powered by MyoSuite that allows for 80 muscle-tendon units and 20 DoFs.

Top: MyoLeg on a forceplate, trying to balance. Bottom: A sarcopenia agent choosing between standing and stepping.

We employ the Proximal Policy Optimization (PPO) algorithm to train the model across both healthy and sarcopenic muscle. The PPO uses a two-hidden-layer neural network with 256 neurons and a ReLU activation function. Our reward function consists of minimizing the metabolic cost, maintaining the agent’s center of mass (CoM) within the base of support (BoS), and minimizing the pelvis position error.

General framework for training the lower limb agent to maintain balance.