Solutions

Testing human teammates in Minecraft

September 1, 2020

ASU researchers are developing artificial intelligence that can communicate with humans within Minecraft

A fire is sweeping through a small town putting people’s homes and lives at risk. The response team arrives and has to operate as a unit to make quick, life-saving decisions, but what happens when one member of the team is not a person, but a virtual agent? How does that affect team performance and trust? And how can such a team best train and prepare for such scenarios?

A team of Arizona State University researchers is answering those and other questions using a popular video game — Minecraft. The Center for Human, Artificial Intelligence, and Robot Teaming (CHART), part of the Global Security Initiative, built an entirely virtual training environment in Minecraft as part of an approximate $3 million award from the Defense Advanced Research Projects Agency (DARPA). The goal of the project is to improve the social intelligence of artificial intelligence and make it better able to assist teams of humans working in complex environments, including in national security missions.

“AI should not replicate humans — it should complement them, each serving interdependent roles,” said Nancy Cooke, director of CHART and the principal investigator on ASU’s portion of the project. “A functional teammate — whether a human or an agent powered by artificial intelligence — understands his or her or its strengths and will have your back when needed.”

ASU is teaming with Aptima to evaluate hypothesis and AI agents, developed by over 200 performers on the Artificial Social Intelligence for Successful Teams (ASIST) program, including representatives from the Massachusetts Institute of Technology, Cornell, the University of Southern California and Carnegie Mellon University. The program is one component of a broader DARPA initiative called the "AI Next" campaign aimed at developing the next generation of AI machines that are more than just tools that execute human-programmed rules, but that function more as colleagues.

“(The ASIST) program will develop foundational AI theory and systems that demonstrate the basic machine social skills needed to infer the goals and beliefs of human partners, predict what they will need, and offer context-aware interventions in order to act as adaptable and resilient AI teammates,” said Joshua Elliott, DARPA program manager.

Rescued victim in Minecraft test bed, image by Christopher Corral

While the project is sponsored by the Department of Defense, the implications for the research go beyond national security.

“Teams that incorporate AI and humans effectively will have an advantage on the battlefield and business,” said Jared Freeman, chief scientist and principal investigator for Aptima. “Our team will help DARPA create and assess socially competent AI by applying our expertise in AI, human behavior, experimentation, and measurement.”

CHART researchers recently pilot tested the Minecraft test bed at a virtual hackathon in June, bringing together more than 100 performers from over 20 organizations. Due to COVID-19, the team pivoted from a planned in-person event to an entirely virtual exercise. Hackathon attendees received data from the ASU research team and reported on the progress that was made through the week either in building agents or testing hypotheses regarding social intelligence.

“AI has its own capabilities and limitations, just as humans do,” Cooke said. “The DARPA ASIST project is about imbuing AI agents with social intelligence so that they understand the intentions of their human teammates and can step in and provide appropriate assistance when needed.”

ASU students connected with the researchers and collaboratively combined forces to conduct experiments within the Minecraft test bed.

“Using Minecraft for the simulated task environment allows us to design complex and highly dynamic tasks,” PhD student Verica Buchanan said. “Working with the researchers is a collaborative effort and building working relationships is very important.”

“Watching the researchers transform the experiment data into stunning and useful data stories has expanded my perspective,” master's degree student Anagha Mudigonda said. “Connecting virtually across various time zones is often challenging and yet this experience enhances our team spirit and collaboration during COVID-19.”

Top image: ASU mascot Sparky rendered in Minecraft.

Oliver Dean

Manager of Marketing and Communications, Knowledge Enterprise , Global Security Initiative

480-727-4419 Oliver.Dean@asu.edu

ASU finds innovative solution to parts shortage during ASU Sync installation

In the weeks leading up to the start of the fall 2020 semester, Arizona State University was busy preparing its classrooms, installing equipment for the new ASU Sync experience. But because of a parts shortage due to the pandemic, ASU’s timeline was almost jeopardized, leading the university to do what it does best: find an innovative solution.Like many families looking for in-demand products du...

Read the article

ASU finds innovative solution to parts shortage during ASU Sync installation

Tempe campus

September 1, 2020

University Technology Office found on-campus answer to meet upgrade deadline

Like many families looking for in-demand products during this pandemic (think disinfecting wipes and toilet paper), colleges and universities across the country were ordering camera mounts to get classrooms ready for virtual instruction. As a result, the University Technology Office was short 60 mounts, and the manufacturer working with ASU wasn’t able to deliver before Aug. 20 — the first day of classes. Once the word got out that UTO needed help, ASU’s Instrument Design and Fabrication Core Facility offered to manufacture the parts in-house on the Tempe campus.

IDF, which is part of ASU’s Core Research Facilities, was established in 2016 and is at the forefront of innovation. This is the place where dream projects come to life — built by expert staff with advanced mechanical assembly design and prototyping skills. The facility specializes in electronics, glassblowing and machining; it has a self-service machine shop.

Led by Director Brian Ipema, the IDF team supports a variety of projects universitywide and for peer universities and private industry partners. Because of its capabilities, this highly specialized research facility was able to order materials, cut, weld, paint and assemble 20 camera mounts within 10 days of UTO’s initial inquiry. Eight days later, the team made 40 more.

ASU's Instrument Design and Fabrication Core Facility made dozens of camera mounts for the University Technology Office after a parts shortage during ASU Sync installation. Photo credit: ASU

“The fact that we exhausted every external avenue for getting the camera mounts we needed in time for the fall semester is a great example of how tricky COVID-19 is to work around, especially as it concerns any equipment that could be used for distance/synchronous learning and remote work,” said Corey Marshall, manager of learning space technology design and integration. “If not for Brian and team, we’d have had to completely rethink a large piece of the ASU Sync strategy.”

The versatile design was carefully chosen so it could be used in various buildings, depending on the structure’s age and materials. Marshall added that his team was “bowled over,” explaining the mounts looked identical to the example provided to IDF.

“There are a lot of people involved in this, and we’re just one small part of it,” said Ipema. “We just happen to be fortunate enough that we have the capabilities here to be flexible, to move over to a direction to try and assist in supporting what is needed right now.”

The Instrument Design and Fabrication Core Facility installed clear safety shields in various ASU buildings, including Biodesign Institute Building A. Photo credit: ASU

This isn’t the first time IDF has assisted with campus projects during the pandemic. When ASU’s Biodesign Institute first introduced its saliva-based COVID-19 testing, IDF designed and manufactured free-standing, mobile test cells that protected both test subjects and sample collectors. IDF also assisted with the installation of clear safety shields in Biodesign Buildings A, B and C — along with multiple areas at ASU Skysong. Once testing ramped up and more samples needed to be processed, IDF made several dozen racks (each holds 50) to organize the test tubes, illustrating a level of detail that makes the team’s work very important. Ipema adds, “It’s the little things that are right there in front of you, but you don’t always see them.”

ASU's Instrument Design and Fabrication Core Facility assisted the Biodesign Institute by making dozens of tube racks for the team's COVID-19 saliva-based testing samples. Photo credit: ASU

IDF’s expertise has been critical in large ASU research projects like Biodesign’s Compact X-ray Free Electron Laser (CXFEL). For that project, IDF manufactured components to build the laser assembly. Most recently, IDF assisted with two Mars projects through the School of Earth and Space Exploration. IDF manufactured components on the Emirates Mars Infrared Spectrometer (EMIRS) aboard the Hope orbiter, and the camera mount testing fixture on Mastcam-Z aboard the Perseverance rover. Both launched in July.

The IDF team is comprised of 11 people and has been fortunate to be a part of many research projects with small-scale and global impacts — one of the many rewarding aspects of the work.

“We are always seeing something that is different, and in many cases, it’s never been done before,” said Ipema. “We’re here to assist (in the research) and to make certain that we can do as much as possible to aid in that forward movement.”