Newswise — In 1715 the British Admiralty set up a contest offering a cash prize for determining the longitude of a ship at sea. For a long time this was the only really big incentive prize competition, but nowadays the idea is commonly used to speed the achievement of some technological tour de force--creating a world-class chess computer, say, or running a car on solar power.
Now Netflix, the online movie-rental company, is using prize contests to improve the way it does business. It has offered US $1 million to whoever devises an algorithm that is at least 10 percent more accurate in judging a customer's taste in movies than the company's own algorithm is.
No one has yet reached that mark, but three researchers at AT&T Laboratories came closest by the contest's first deadline, achieving an 8.43 percent improvement, and that was enough to win them the intermediate prize of $50,000. The researchers--Robert M. Bell, Yehuda Koren, and Chris Volinsky--together with Jim Bennett, a former Netflix executive, describe their work in the May issue of IEEE Spectrum.
The contest was set within the bounds of collaborative filtering, a strategy based solely on ratings given by a company's customers. (An alternative method employing external ratings, in this case that of music experts, is described in a sidebar on Pandora, the Internet radio company). Collaborative filtering works best when the ratings are explicit, as when a Netflix customer clicks on an icon to show how well he likes a given movie. But even if the customer omits such a rating, his tastes can still be inferred from the pattern of his rentals.
One technique predicts how a customer will rate a particular movie by comparing his other ratings with those given by all the other customers. Let's say you like Terminator, Blackhawk Down, and Saving Private Ryan, but hated Mary Poppins and were lukewarm about Coraline. The algorithm would look at all the other people who had exactly the same profile and then would ask which one they liked the most. If the answer is Star Wars, then that is the movie the algorithm would recommend to you.
Another technique analyzes the movie ratings of all the customers to infer the 40 or 50 factors that together work best to "place" both movies and customers. A factor is a two-dimensional axis, ranging between two extremes--say, "male-oriented" to "female-oriented," or "funny" to "serious." Many such axes define a multidimensional space, and the closer you stand in it to a given movie, the more you are expected to like it.
The authors speculate that many companies that have compiled the purchase decisions of customers stand to benefit fom the use of such recommender systems. Further, they argue that the Netflix Prize may itself be emulated. The contest has demonstrated that a relatively small investment can induce a vast number of smart people to work on your problem, provided you can frame the enterprise as a game.
RSS