Music can be used to get people to move to the beat, but can you apply the same trick to robots?
The robots Szorkovszky is working on are currently digital but the plan is to test the robots’ ear for music using real robots like the one in the photograph. Photo: Elina Melteig
This article was first published by Titan (in Norwegian)
Have you ever tried standing completely still when your favourite song comes on? It’s difficult not to move your body when a catchy tune is played. Standing still to a good beat is actually so difficult that it has triggered an avalanche of research projects. One of them looks at whether music can also be used to get robots to move to a beat.
"A big focus of our research is the involuntary and irresistible urge to move to a good beat," says postdoctoral fellow Alexander Szorkovszky
Szorkovszky works at RITMO Centre for Interdisciplinary Studies in Rhythm, Time and Motion at the University of Oslo. He wants to find out whether it is possible to coordinate robots using the rhythms that can be found in music.
Robots losing control to music
“The most common approach in robotics is to give robots maximum control and stability in their movements. But what happens if you give up this control to music, for example?” Szorkovszky ponders.
According to Szorkovszky, movement to music can be viewed in two different ways: One is that music literally sets you off – you are passively reacting to the rhythm. The other is that you predict the rhythm and deliberately move in time with the beat.
“I am exploring the idea that music sets you off,” Szorkovszky explains.
The video above shows what it could look like when a robot spontaneously moves to a rhythm.
Movements are rooted in the spine
The robot created by Szorkovszky is currently digital. The screen shows a simple robot with four legs. In the software, the robot is a passive recipient of the rhythm and is controlled by the music – that is if it is able to pick out the beat.
“In order for people to want to move, the rhythm should have the right complexity. It gets boring if the rhythm is too simple. If it is too complex, it becomes difficult to identify the beat,” he explains and adds “This robot does not currently have the ability to experience boredom, so it works well with simple rhythms”.
The robot controller takes inspiration from neural circuits known as central pattern generators. Researchers believe that these circuits contribute to several biological rhythms, such as the rhythms of breathing or walking. For humans, the circuit for walking is literally found in the spine. Models of central pattern generators can also be used to control the movements of legged robots. This is why robots may be useful for investigating whether these circuits can spontaneously synchronize to external stimuli, such as the rhythm of music.
To develop the central pattern generators, Szorkovszky explains, they are using evolutionary optimization on a large group of virtual robots. Using evolution in the context of robots means that the robots with the best characteristics are selected from each generation to move on to the next. Minor “mutations” or changes may appear in each generation. The algorithm also combines parent robots in order to create new combinations of characteristics. In this way, the evolutionary algorithm generates a variety of movement styles.
When the robots test their dance skills, Szorkovszky selects the best robots from the evolution software. That is, the robots that are most able to synchronize to the tempo.
Szorkovszky starts the simulation and the music begins. The robot is moving on all four legs. After a short while, the robot starts to follow the beat. As time passes, the music becomes more complex and it becomes harder for the robot to follow the beat. Eventually the robot struggles to “shuffle” its legs around.
In the Hall of the Mountain King was too hard – how about a waltz?
Szorkovszky has tested various types of music.
“I tried In the Hall of the Mountain King, but it started off too slow,” Szorkovszky says with a smile.
The simulation currently includes just one four-legged robot. Eventually, Szorkovszky would like to find out what happens if the robot is given more legs.
“I want to try and find out what happens if a six-legged robot is tasked with dancing to a waltz, would it end up moving straight or around in circles?
Humans naturally start to twirl around when we hear a waltz as we move on two legs, but this is not a given if the number of legs is different.” He has yet to test this theory.
So far, Szorkovszky has proven that it is possible to control a robot using simple beats. In time, he intends to test whether robots can coordinate their movements with music and with one another. They also intend to test the experiment using physical prototypes.
Read the scientific article:
Alex Szorkovszky, Frank Veenstra and Kyrre Glette: Central pattern generators evolved for real-time adaption, ArXiv, October 2022
