Newswise — Electrical engineers at UC San Diego created games on Facebook in order to improve their experimental music search engine that is capable of listening to new songs and accurately labeling them with words—with no help from humans. These computer-labeled songs can then be retrieved later when someone types these same words into the cutting-edge music search engine. In April, the engineers launched the music discovery games on Facebook as an application called Herd It (http://apps.facebook.com/herd-it).
"The Facebook games are a lot of fun and a great way to discover new music. At the same time, the games deliver the data we need to teach our computer audition system to listen to and describe music like humans do," said Gert Lanckriet, the electrical engineering professor and machine learning expert from the UC San Diego Jacobs School of Engineering steering the project. To play Herd It, log in to Facebook, open the Herd It app, select a genre of music, and start listening to song clips and playing the games. Some games ask users to identify instruments, while others focus on music genres, artist names, emotions triggered by the song, and activities you might do while listening to a song. The more your answers align with the rest of the online crowd playing the game at the same time, the more points you score. Watch a two minute video about the making of the games and the new search engine at http://www.jacobsschool.ucsd.edu/news/news_video/play.sfe?id=28.
"The more examples of romantic songs our search engine is exposed to, the more accurately it will be able to identify romantic songs it has never heard before," explained Luke Barrington, the UC San Diego electrical engineering Ph.D. student leading the project.
Part of Barrington's Ph.D. dissertation will involve demonstrating that data collected from the Facebook games reliably improves the accuracy of their automated search engine for music.
"Once enough people play our new music discovery games on Facebook, I'll have the data I need to both improve our search engine and finish my Ph.D.," said Barrington.
The song-word combinations collected by the Facebook games also enable the researchers to grow their music search engine's vocabulary and increase its coverage in genres and classes of music.
How it works For the music search engine from UC San Diego to "listen and describe music like a human," it must find patterns in the songs using the tools of machine learning. For example, for the system to learn to identify and label romantic songs, it must be exposed to many different romantic songs during the training period. The Facebook games provide the data necessary for the algorithms to learn to label songs on their own.
This exposure to songs tagged with the relevant labels enables the machine learning algorithms find patterns in the wave forms of the songs that make the songs romantic. Once trained, the system can identify romantic songs that it has never before encountered, offering the tantalizing possibility of amassing a huge database of songs that can be tagged and retrieved based on text-based searches with no human intervention. This technical capability is increasingly important as music goes entirely digital and the ways in which people find music change.
2009 ICASSP (IEEE International Conference on Acoustics, Speech, and Signal Processing): "Dynamic Texture Models of Music," by Luke Barrington, Antoni Chan and Gert Lanckriet from the Electrical and Computer Engineering Department at UC San Diego's Jacobs School of Engineering: http://cosmal.ucsd.edu/cal/pubs/Barrington-MusicDTM-ICASSP09.pdf
Herd It game on Facebook:http://apps.facebook.com/herd-it or http://herdit.org
Online Game Feeds Music Search Engine Project UC San Diego press release from 2007: http://www.jacobsschool.ucsd.edu/news/news_releases/release.sfe?id=692
A Search Engine with Ears from Jacobs School of Engineering alumni magazine, Pulse: http://www.jacobsschool.ucsd.edu/pulse/spring2008/research2.shtml
The National Science Foundation (NSF) funded some of the research leading to this publication, as well as some of the students who contributed.
Professor Gert Lanckriet from the Department of Electrical and Computer Engineering also leads UC San Diego's Computer Audition Laboratory, housed at the UC San Diego division of Calit2: http://cosmal.ucsd.edu/cal/
RSS