Newswise — ITHACA, N.Y. – Computer scientists at Cornell University have developed a Minecraft modification that uses artificial intelligence to help players improve their in-game architecture skills.

Minecraft is a popular 3D video game where players build and navigate their own digital environments. The modification will tell players whether their buildings fit into certain architectural styles and offer ideas for how the structures could be improved. 

“One of the things that’s important to learn when you’re a kid and throughout life is creativity, abstraction – how to envision what you want and then create it,” said senior author Ross Knepper, assistant professor of computer science. “This is a tool that helps people not get discouraged, maybe if they’re beginning at Minecraft and don’t know how to use their imagination right off the bat.”

Knepper, along with authors Erik Anderson, assistant professor of computer science; Irene (Euisun) Yoon ’19; and Bharath Hariharan, assistant professor of computer science, will present the paper, “Design Mining for Minecraft Architecture,” at the Association for the Advancement of Artificial Intelligence Conference on Artificial Intelligence and Interactive Digital Entertainment Nov. 13-17 in Canada.

Based on buildings Minecraft players created and uploaded for others to use, the researchers created a deep neural network – a kind of machine learning trained to predict whether data belongs in a certain category. Through that network, players could learn whether their building is medieval, modern, Asian or classical – four especially popular tags used by Minecraft players. Once the building is classified, another algorithm can show the users similar buildings to inspire them to make improvements to their own.

“People are really interested in having more design spaces in Minecraft, and being able to build certain types of architecture, but there weren’t any design tools as far as we were aware that could teach them,” said Yoon, who is first author on the paper.

For Knepper, a roboticist by training, the Minecraft project helped answer questions about how a robot might follow a human’s instructions.

“If I say, ‘Build a house,’ today a robot is going to say, ‘I don’t know what that means.’ ‘Which brick should I put where?’” Knepper said. “We’d like humans to be able to interface with robots more like we interface with each other. So if I tell it to build a medieval house or an ancient house and give some of the high-level details, it would know at that point how to turn it into a plausible thing that does everything you want. We’re not there yet, but this is the first step towards that goal.”

The paper is based on work supported by the National Science Foundation.

For additional information, see this Cornell Chronicle story.

Cornell University has dedicated television and audio studios available for media interviews supporting full HD, ISDN and web-based platforms.