Skip to main content

Testing human teammates in Minecraft

ASU researchers are developing artificial intelligence that can communicate with humans within Minecraft

Sparky in Minecraft
September 01, 2020

A fire is sweeping through a small town putting people’s homes and lives at risk. The response team arrives and has to operate as a unit to make quick, life-saving decisions, but what happens when one member of the team is not a person, but a virtual agent? How does that affect team performance and trust? And how can such a team best train and prepare for such scenarios?

A team of Arizona State University researchers is answering those and other questions using a popular video game — Minecraft. The Center for Human, Artificial Intelligence, and Robot Teaming (CHART), part of the Global Security Initiative, built an entirely virtual training environment in Minecraft as part of an approximate $3 million award from the Defense Advanced Research Projects Agency (DARPA). The goal of the project is to improve the social intelligence of artificial intelligence and make it better able to assist teams of humans working in complex environments, including in national security missions.

“AI should not replicate humans — it should complement them, each serving interdependent roles,” said Nancy Cooke, director of CHART and the principal investigator on ASU’s portion of the project. “A functional teammate — whether a human or an agent powered by artificial intelligence — understands his or her or its strengths and will have your back when needed.”

ASU is teaming with Aptima to evaluate hypothesis and AI agents, developed by over 200 performers on the Artificial Social Intelligence for Successful Teams (ASIST) program, including representatives from the Massachusetts Institute of Technology, Cornell, the University of Southern California and Carnegie Mellon University. The program is one component of a broader DARPA initiative called the "AI Next" campaign aimed at developing the next generation of AI machines that are more than just tools that execute human-programmed rules, but that function more as colleagues.

“(The ASIST) program will develop foundational AI theory and systems that demonstrate the basic machine social skills needed to infer the goals and beliefs of human partners, predict what they will need, and offer context-aware interventions in order to act as adaptable and resilient AI teammates,” said Joshua Elliott, DARPA program manager.

Rescued victim in Minecraft testbed

Rescued victim in Minecraft test bed, image by Christopher Corral

While the project is sponsored by the Department of Defense, the implications for the research go beyond national security.

“Teams that incorporate AI and humans effectively will have an advantage on the battlefield and business,” said Jared Freeman, chief scientist and principal investigator for Aptima. “Our team will help DARPA create and assess socially competent AI by applying our expertise in AI, human behavior, experimentation, and measurement.”

CHART researchers recently pilot tested the Minecraft test bed at a virtual hackathon in June, bringing together more than 100 performers from over 20 organizations. Due to COVID-19, the team pivoted from a planned in-person event to an entirely virtual exercise. Hackathon attendees received data from the ASU research team and reported on the progress that was made through the week either in building agents or testing hypotheses regarding social intelligence.

“AI has its own capabilities and limitations, just as humans do,” Cooke said. “The DARPA ASIST project is about imbuing AI agents with social intelligence so that they understand the intentions of their human teammates and can step in and provide appropriate assistance when needed.”

ASU students connected with the researchers and collaboratively combined forces to conduct experiments within the Minecraft test bed.

“Using Minecraft for the simulated task environment allows us to design complex and highly dynamic tasks,” PhD student Verica Buchanan said. “Working with the researchers is a collaborative effort and building working relationships is very important.”

“Watching the researchers transform the experiment data into stunning and useful data stories has expanded my perspective,” master's degree student Anagha Mudigonda said. “Connecting virtually across various time zones is often challenging and yet this experience enhances our team spirit and collaboration during COVID-19.” 

Top image: ASU mascot Sparky rendered in Minecraft.

More Science and technology


Illustration of a semiconductor being put together

Advanced packaging the next big thing in semiconductors — and no, we're not talking about boxes

Microchips are hot. The tiny bits of silicon are integral to 21st-century life because they power the smartphones we rely on, the cars we drive and the advanced weaponry that is the backbone of…

Four people sitting around a computer screen

Securing the wireless spectrum

The number of devices using wireless communications networks for telephone calls, texting, data and more has grown from 336 million in 2013 to 523 million in 2022, according to data from U.S.…

Illustrations showing game icons including a young girl, sunglasses, a t-shirt, water bottle and more

New interactive game educates children on heat safety

Ask A Biologist, a long-running K–12 educational outreach effort by the School of Life Sciences at Arizona State University, has launched its latest interactive educational game, called "Beat the…