Autonomous buses: It’s all about when, not how, they sound

Newswise — ITHACA, N.Y. – Researchers at Cornell University are using sound to help autonomous vehicles navigate complex social situations, like communicating with people in traffic. After testing multiple scenarios, they discovered the timing of a sound, rather than the type of sound, was most important. 

“If we want to create sounds for social engagement, it’s really about shifting the focus from ‘what’ sound to ‘when’ sound,” said study co-author Malte Jung, associate professor of information science. 

Lead author and visiting scholar Hannah Pelikan presented their study, “Designing Robot Sound-In-Interaction: The Case of Autonomous Public Transport Shuttle Buses,” at the 2023 ACM/IEEE International Conference on Human-Robot Interaction. 

The researchers focused on the town of Linköping, Sweden, which has a small fleet of autonomous electric buses that carry riders along a predetermined route. The vehicles have one main problem: Pedestrians and cyclists regularly get too close, causing the buses to brake suddenly, making riders late for work. 

The researchers designed potential bus sounds through an iterative process. They played sounds through a waterproof Bluetooth speaker on the outside of the bus, analyzed video recordings of the resulting interactions and used that information to select new sounds to test. 

Initially, the researchers tried humming sounds that became louder as people got closer, but low-pitched humming blended into the road noise and a high-pitched version irritated the safety drivers. The repeated sound of a person saying “ahem” was also ineffective. 

They found that “The Wheels on the Bus” and a similar jingle successfully signaled cyclists to clear out before the brakes engaged. The song also elicited smiles and waves from pedestrians. Standard vehicle noises – beeps and dings – also worked to grab people’s attention; repeating or speeding up the sounds communicated that pedestrians needed to move farther away. 

In analyzing the videos, Pelikan and Jung saw that regardless of which sound they played, the timing and duration were most important for signaling the bus’s intentions – just as the honk of a car horn can be either a warning or a greeting. 

The study’s approach represents a new way of designing sound that is applicable to any autonomous system or robot, the researchers said. While most sound designers work in quiet labs and create sounds to convey specific meanings, this approach uses the bus as a laboratory to test how people will respond to the sounds. 

“We’ve approached sound design all wrong in human-robot interaction for the past decades,” Jung said. “We wanted to really rethink this and bring in a new perspective.” 

Pelikan and Jung said their findings also underline another important factor for autonomous vehicle design: Traffic is a social phenomenon. While societies may have established rules of the road, people are constantly communicating through their horns, headlights, turn signals and movements. Pelikan and Jung want to give autonomous vehicles a better way to participate in the conversation.

The research received funding from the Swedish Research Council and the National Science Foundation.

For additional information, see this Cornell Chronicle story.

Media note: Video of the buses can be viewed and downloaded here: https://cornell.box.com/v/autonomousbus

Cornell University has dedicated television and audio studios available for media interviews.

-30-