The idiom “actions speak louder than words” first appeared in print almost 300 years ago. A new study echoes this view, arguing that combining self-supervised and offline reinforcement learning (RL) could lead to a new class of algorithms that understand the world through actions and enable scalable representation learning.

Machine learning (ML) systems have achieved outstanding performance in domains ranging from computer vision to speech recognition and natural language processing, yet still struggle to match the flexibility and generality of human reasoning. This has led ML researchers to search for the “missing ingredient” that might boost these systems’ ability to understand, reason and generalize.

In the paper Understanding the World Through Action, UC Berkeley assistant professor in the department of electrical engineering and computer sciences Sergey Levine suggests that a general, principled, and powerful framework for utilizing unlabelled data could be derived from RL to enable ML systems leveraging large datasets to better understand the real world.

To set some benchmarks for their simulator, the researchers tried out three different design algorithms working in conjunction with a deep reinforcement learning algorithm that learned to control the robots through many rounds of trial and error.

The co-designed bots performed well on the simpler tasks, like walking or carrying things, but struggled with tougher challenges, like catching and lifting, suggesting there’s plenty of scope for advances in co-design algorithms. Nonetheless, the AI-designed bots outperformed ones design by humans on almost every task.

Intriguingly, many of the co-design bots took on similar shapes to real animals. One evolved to resemble a galloping horse, while another, set the task of climbing up a chimney, evolved arms and legs and clambered up somewhat like a monkey.