A professional esports player faces down international opponents in a video game championship. A radiologist examines hundreds of images from a patient’s CT scan. An aerospace engineer carefully sends instructions to a spacecraft as it navigates the stars.
On the line are millions of dollars, a patient’s life and irreplaceable scientific equipment. Yet one common threat puts all these at risk: The person at the screen is battling fatigue.
When working in front of a screen, a person’s attention rises and falls in a regular pattern. By tracking this pattern, a machine learning program can predict when someone’s performance is about to drop — before it happens. This early finding comes from an ongoing study at the adidas-ASU Center for Engagement Science.
The center, a partnership between Arizona State University and the athletic company adidas, focuses on understanding human behavior, perception and performance. While the research mostly aims to improve outcomes for athletes, it also has applications for other jobs where people work under pressure.
Taking data to the next level
The project that led to the center’s recent finding looks at measuring and predicting human performance in esports. HyperX, a company that develops products for gamers, teamed up with the center to figure out whether it’s possible to predict a player’s drop in performance using biometric data.
In the gaming world, players will eventually come to a point when their performance takes a downward spiral, known as tilt. Predicting it could one day lead to interventions that prevent it from ever happening.
“We figured out that we can actually predict tilt about 15 to 20 minutes before it occurs,” says Aurel Coza, director of the center. “Obviously for gamers it's super important, but then you can extrapolate that to any high-performance task that involves prolonged attention.”
HyperX sponsored the study and provided gaming products that are regularly used by professional gamers globally.
"Player well-being and performance are important to HyperX, and stress factors are important to observe and learn about to effectively manage tilt factors in gaming,” says Dustin Illingworth, head of culture marketing at HyperX. “Many benefits can come from the collaboration between ASU and HyperX, both in the gaming industry and for people who spend long periods of time in front of screens or monitors."
Coza leads a team of student researchers who spent six months gathering biometric data from 45 study participants in the center’s lab. The participants were all experienced gamers. For the study, they spent five to six hours playing one of three popular video games: League of Legends, Valorant or Call of Duty.
As they played, the team tracked participants’ heart rates, skin conductivity, eye movements and facial expressions using wristband health monitors and webcams. They also took note of their scores throughout each game.
The students then fed all the data into a machine learning algorithm that looked at the connections between players’ performance and their biometric data.
Biomedical engineering grad student Justin Irby was a study participant before he became a research assistant on the project. The team gathered data on gamers’ performance and biometric measurements, like facial expression, eye movement and heart rate. Image courtesy adidas-ASU Center for Engagement Science
The biometric data offered a way to measure the participants’ physical stress, emotional states and recurring cycles of overall fatigue and attention.
“Ironically, at first I was a participant,” says Justin Irby, another student researcher on the project who earned his master’s degree in biomedical engineering this May. “I knew a lot about video games, so I was able to advise on the project. Then for the data analysis stage, I got brought on as a research assistant and my focus was in areas of overtime trends and fatigue.”
The team found that no single biometric variable could point to players’ performance or fatigue cycles on its own. However, when the algorithm brought all the variables together, it revealed a pattern that clearly showed when tilt occurred and allowed the team to predict it before it happened.
“What was interesting was we got an accuracy of around 80% for all three games, when we actually thought it might come in around 50% or so,” says Karthikeyan Manikandan, a student researcher who graduated with a master’s in biomedical engineering in May and will begin his doctoral program at the University of Iowa this fall.
“It was a eureka moment for us,” adds Coza.
While some may think of video games as a casual pastime, gaming represents a growing, multibillion-dollar industry that touches many lives.
According to the Entertainment Software Association 2022 report, two-thirds of Americans across all ages play video games weekly. Additionally, professional esports players compete for millions of dollars in prize money, and world championship games attract millions of viewers from around the globe.
But Coza sees another area of potential for gamers.
“Esports has been overlooked as a serious topic of study. But when you think about it, it is the best place to actually study human behavior,” he says. Players are enthusiastic study subjects, and most of the biometric sensors are already housed in the machine they use to play.
“We see this as a massive platform for research,” Coza says.
While the team continues to examine and publish their data, they hope the project’s next big step will be a phase two with HyperX. This phase would focus on putting what they’ve learned into practice by exploring interventions that will prevent gamers from going into tilt.
“If a technology could predict the tilt beforehand, it could give a heads-up to the players and they could adjust themselves. That will be a very impactful thing in competitions. And players can also be trained in such a way that they won't actually go into the tilt,” says Krishna Suketh Madduri, a student researcher on the team and a biomedical engineering master’s student.
According to a National Skill Coalition 2023 report, 92% of U.S. jobs use digital technology. So not only would such interventions change the gaming industry for hobbyists and pros alike, they could also help those working in the health care field, in emerging industries like aerospace, or even in everyday office or learning environments — any context where people perform under constant pressure.
“The results are really broadly applicable, especially if we're talking about workplace performance,” Coza says. “Most jobs involve staring at a screen for hours a day. That's not much different, honestly, than any video game.”
Top photo illustration by Nathan Stetson.
