Can Cartoons Be Used to Teach Machines to Understand the Visual World?

Article ID: 623754

Released: 25-Sep-2014 10:00 AM EDT

Source Newsroom: Virginia Tech

  • Credit: Virginia Tech

    Virginia Tech's Devi Parikh received her doctorate in electrical and computer engineering from Carnegie Mellon in 2009, and is already a U.S. Army Research Office Young Investigator, working with the government on ways to reduce failures in computerized vision recognition programs.

Newswise — An enormous gap exists between human abilities and machine performance when it comes to understanding the visual world from images and videos. Humans are still way out in front.

“People are the best vision systems we have,” said Devi Parikh assistant professor in the Bradley Department of Electrical and Computer Engineering at Virginia Tech. “If we can figure out a way for people to effectively teach machines, machines will be much more intelligent than they are today.”

In her research, Parikh is proposing to use visual abstractions or cartoons to teach machines. She works from the idea that concepts that are difficult to describe textually may be easier to illustrate. By having thousands of online crowd workers manipulate clipart images to mimic photographs, she seeks to teach a computer to understand the visual world like humans do.

Parikh has expertise in computing areas such as computer vision and pattern recognition. Based on her earlier successful creative work on how to learn from visual abstractions, Google has selected Parikh to receive one of its Faculty Research Awards.

Google’s innovative award provides Parikh with $92,000 of unrestricted funds and allows her to work directly with Google researchers and engineers as they explore how to best learn from visual information.

Parikh, formerly a research assistant professor at the Toyota Technological Institute in Chicago, received her doctorate in electrical and computer engineering from Carnegie Mellon in 2009, is already a U.S. Army Research Office Young Investigator, working with the government on ways to reduce failures in computerized vision recognition programs.

“We need to build intelligent machines that can understand our visual world from images just as humans do. These machines must be capable of answering high-level semantic questions about an image such as what objects are present, where they are, and how they are interacting,” Parikh said. The Google award is for one year and only full-time, tenure-track university faculty members are eligible.

Parikh, who joined Virginia Tech in 2013, is a core member of the Discovery Analytics Center and the Virginia Center for Autonomous Systems http://www.unmanned.vt.edu/ , both part of the Virginia Tech Institute for Critical Technology and Applied Science. She also heads the Computer Vision Lab.

This award marks the second time Google has honored her. She was granted a Google Faculty Research Award in 2012 for her research on relative attributes-based feedback for image search.


Comment/Share





Chat now!