Skip to main content

Testing human teammates in Minecraft

ASU researchers are developing artificial intelligence that can communicate with humans within Minecraft

Sparky in Minecraft
September 01, 2020

A fire is sweeping through a small town putting people’s homes and lives at risk. The response team arrives and has to operate as a unit to make quick, life-saving decisions, but what happens when one member of the team is not a person, but a virtual agent? How does that affect team performance and trust? And how can such a team best train and prepare for such scenarios?

A team of Arizona State University researchers is answering those and other questions using a popular video game — Minecraft. The Center for Human, Artificial Intelligence, and Robot Teaming (CHART), part of the Global Security Initiative, built an entirely virtual training environment in Minecraft as part of an approximate $3 million award from the Defense Advanced Research Projects Agency (DARPA). The goal of the project is to improve the social intelligence of artificial intelligence and make it better able to assist teams of humans working in complex environments, including in national security missions.

“AI should not replicate humans — it should complement them, each serving interdependent roles,” said Nancy Cooke, director of CHART and the principal investigator on ASU’s portion of the project. “A functional teammate — whether a human or an agent powered by artificial intelligence — understands his or her or its strengths and will have your back when needed.”

ASU is teaming with Aptima to evaluate hypothesis and AI agents, developed by over 200 performers on the Artificial Social Intelligence for Successful Teams (ASIST) program, including representatives from the Massachusetts Institute of Technology, Cornell, the University of Southern California and Carnegie Mellon University. The program is one component of a broader DARPA initiative called the "AI Next" campaign aimed at developing the next generation of AI machines that are more than just tools that execute human-programmed rules, but that function more as colleagues.

“(The ASIST) program will develop foundational AI theory and systems that demonstrate the basic machine social skills needed to infer the goals and beliefs of human partners, predict what they will need, and offer context-aware interventions in order to act as adaptable and resilient AI teammates,” said Joshua Elliott, DARPA program manager.

Rescued victim in Minecraft testbed

Rescued victim in Minecraft test bed, image by Christopher Corral

While the project is sponsored by the Department of Defense, the implications for the research go beyond national security.

“Teams that incorporate AI and humans effectively will have an advantage on the battlefield and business,” said Jared Freeman, chief scientist and principal investigator for Aptima. “Our team will help DARPA create and assess socially competent AI by applying our expertise in AI, human behavior, experimentation, and measurement.”

CHART researchers recently pilot tested the Minecraft test bed at a virtual hackathon in June, bringing together more than 100 performers from over 20 organizations. Due to COVID-19, the team pivoted from a planned in-person event to an entirely virtual exercise. Hackathon attendees received data from the ASU research team and reported on the progress that was made through the week either in building agents or testing hypotheses regarding social intelligence.

“AI has its own capabilities and limitations, just as humans do,” Cooke said. “The DARPA ASIST project is about imbuing AI agents with social intelligence so that they understand the intentions of their human teammates and can step in and provide appropriate assistance when needed.”

ASU students connected with the researchers and collaboratively combined forces to conduct experiments within the Minecraft test bed.

“Using Minecraft for the simulated task environment allows us to design complex and highly dynamic tasks,” PhD student Verica Buchanan said. “Working with the researchers is a collaborative effort and building working relationships is very important.”

“Watching the researchers transform the experiment data into stunning and useful data stories has expanded my perspective,” master's degree student Anagha Mudigonda said. “Connecting virtually across various time zones is often challenging and yet this experience enhances our team spirit and collaboration during COVID-19.” 

Top image: ASU mascot Sparky rendered in Minecraft.

More Science and technology


Headshot of Yoan Simon in an outdoor setting

Chemist joins ASU to tackle problems surrounding polymers, sustainability

Trained as a chemist, Associate Professor Yoan Simon’s research straddles synthetic organic chemistry, materials science, chemical engineering and energy research. He has recently joined the School…

Portrait of Chao Ma in suit smiling.

A ceramic renaissance

Rising from the smoky embrace of kilns, ceramics played a significant role during the Renaissance era, with the resurgence of sculptors who originally used the material as a form of classical…

Three women and a man stand in front of a banner that reads Indo-Pacific Space and Earth Conference

ASU-based space workforce training program expands to Australia and New Zealand

The Milo Space Science Institute, led by Arizona State University, will offer its space workforce training program to university and vocational students in Australia and New Zealand starting in March…