Teaching Robots the Fundamentals of Movement: A Step Towards Real-World Applications

Sunday - April 21 2024, 18:40 UTC - 7 months ago

Humanoid robots are becoming increasingly agile, but there have been few real-world applications. Researchers are now using sim-to-real reinforcement learning to teach robots the fundamentals of movement and navigate various environments. This technique involves training AI models in simulations before deploying them to robots in the real world, resulting in faster learning. Successful experiments have been conducted, with robots now able to stand, walk, and pick up objects. The focus is now on making robots more adaptable and robust for everyday usage.

Humanoid robots have come a long way in terms of their agility, with videos showcasing their ability to run and jump with ease. However, these impressive feats lack practical applications in the real world. In order for robots to truly become useful and safe around humans, they must first master the fundamentals of movement. This is why researchers are now focusing on using sim-to-real reinforcement learning to train robots for more modest tasks.

One such robot, Digit V3, has been successfully trained by a team of researchers led by Alan Fern from Oregon State University. The robot can now stand, walk, and pick up objects, as well as move them from one location to another. Separately, researchers from the University of California, Berkeley have also applied this technique to teach Digit to walk in unfamiliar environments while carrying various loads without falling over. Their findings have been published in a paper in Science Robotics today.

The concept of sim-to-real reinforcement learning involves training an AI model in a simulated environment billions of times before deploying it to a robot in the real world. What may take years for a robot to learn in real life can now be accomplished in days through repeated trial-and-error testing in simulations. A neural network guides the robot to its target location using a mathematical reward function, which rewards it with a high score every time it makes progress towards its goal. On the other hand, incorrect actions, such as falling, are punished with a negative score, encouraging the robot to avoid these behaviors in the future.

This approach has yielded success in the past, with researchers from the University of Oregon using it to teach a two-legged robot named Cassie to run. The robot became the first to run an outdoor 5K and set a Guinness World Record for the fastest bipedal robot to run 100 meters. It also mastered the ability to jump from one location to another effortlessly.

Ilija Radosavovic, a PhD student at Berkeley who trained Digit to carry various loads and stabilize itself when poked with a stick, explains that this type of research focuses on fairly simple skills in broad environments. This is in contrast to previous projects that have focused on complex movements in controlled environments. The ultimate goal is to make humanoid robots more adaptable and robust, which are essential qualities for their real-world usage.

Currently, humanoid robots are a rare sight in work environments due to their struggles with balance when carrying heavy objects. This is why they are mostly confined to controlled, simulated environments. However, with ongoing advancements in sim-to-real reinforcement learning, it is only a matter of time before we see more humanoid robots in practical, real-world applications.