Mind reading in the lab: New ASU professor decodes brain activity to understand memory and attention

December 4, 2019

The X-Men’s Professor Charles Xavier uses Cerebro to read minds. Arizona State University’s Gi-Yeul Bae “reads minds” by decoding the brain’s electrical activity.

Neurons in the brain communicate with each other using electrical activity, and Bae records this activity using a technique called electroencephalography, or EEG for short. He then feeds the EEG data into a computer algorithm that learns the relationships among patterns of brain activity and the information encoded in the brain. This algorithm helps Bae better understand memory and attention. Gi-Yeul Bae Arizona State University’s Gi-Yeul Bae “reads minds” by decoding the brain’s electrical activity. Photo by Robert Ewing Download Full Image

“We can decode what people saw through the spatial patterns of brain activity, and we can use this information to answer challenging questions in psychology, like whether paying attention to an item is the same thing as holding it in your memory,” said Bae, who recently joined the faculty in the ASU Department of Psychology as an assistant professor. “It is very difficult to decode abstract thoughts, but we can figure out visual features like the tilt angle of an object.”

People who participate in an EEG experiment wear a tightly fitting cap that positions an array of electrodes across the scalp. These electrodes measure the brain’s electrical activity. To decode how the electrical signals are related to brain functions, Bae and his collaborators divide the EEG dataset into several parts. One part of the dataset is used to train the decoding algorithm. In this step, the algorithm finds associations between patterns of brain activity and events like remembering how an object is positioned in space. The remaining parts of the EEG dataset are then used to test the associations found by the algorithm.

“Thanks to the high temporal resolution of EEG — we can take a measurement every 4 milliseconds, which is 25 times faster than an eye blink — we can track the temporal dynamics of brain activity and see how information used in forming memories evolves over time,” Bae said.

Bae and his collaborators recently tested whether they could separate brain activity related to remembering an object’s location or its appearance. Each trial in the experiment showed a tear-shaped droplet that was rotated at a random angle. The droplet briefly appeared on a computer screen, and after it disappeared the participants had to remember the exact tilt angle of the droplet. Following a short delay, they used a mouse to report the droplet angle.

In the decoding experiments, the research team focused on two types of EEG signals called event related potentials (ERPs) and alpha waves. ERPs are the average electrical response to a stimulus like seeing a tilted droplet. Alpha waves are EEG signals that oscillate eight to12 times a second.

During the delay period, when the droplet was no longer visible and the participants were remembering its location and tilt, the decoding algorithm accurately predicted both the location and tilt angle from the pattern of ERPs on the scalp. This was not the case for alpha waves though: This EEG signal only tracked where the droplet was located.

Bae and his team also used the decoding algorithm to test how the recent past affects what people are currently holding in their memory.

How the participants reported the droplet orientation was biased by the orientation of the droplet they had just seen on the previous trial, even though the previous and current droplet tilt angles were unrelated. When the team used the decoding algorithm to look at a trial, they found patterns of brain activity that encoded the tilt angle from the immediately preceding trial.

“The information for the previous trial was still in the brain during the current trial. This decoding result suggests that the past memories get reactivated in the brain when it processes some similar information,” Bae said. “For example, your brain may reactivate your memory of a snake when you see a rope.”

Bae is currently applying the decoding method to the study of mental illness. One of his projects uses EEG decoding to understand how working memory — the process of temporarily keeping information available — differs between people with schizophrenia and their healthy peers.

Science writer, Psychology Department


ASU School of Music’s production of Bernstein’s ‘Mass’ wins first place in the National Opera Association competition

December 4, 2019

“Evocative, imaginative, engaging.” “Exquisitely controlled chaos.” “… from the mesmerizing opening scene, I was completely drawn in and it gave me HOPE.”

The Arizona State University School of Music’s production of Bernstein’s “Mass: A Theater Piece for Singers, Players and Dancers” received endless praise from reviewers for the 2018-19 National Opera Association Opera Production Competition, and the production placed first in Collegiate Division VIII. Bernstein Mass cast Cast of Bernstein's 'Mass.' Download Full Image

The school put on the production as part of its 2018-19 season to mark the centennial of the legendary American composer's birth. More than 300 people came together from across the Herberger Institute for Design and the Arts to participate in the monumental collaboration.

Brian DeMaris, associate professor and artistic director of ASU Music Theatre and Opera in the School of Music, said he had always been intrigued by Bernstein's "Mass," knowing it to be one of those pieces that can only be understood through experience.

“As soon as I came to ASU in 2015 I knew the School of Music was a place that was positioned to be able to accomplish the piece,” said DeMaris, production manager for "Mass." "With terrific ensembles, a faculty that worked together well, one of the nation’s top orchestral programs, an in-house musical theater and opera program with students skilled in both, and the resources of dance, media and other disciplines in the Herberger Institute’s School of Film, Dance and Theatre, I knew we had the ability to produce and perform it.”

He said Bernstein's 100th anniversary was a perfect opportunity to perform the awe-inspiring production.

Over the course of two years, DeMaris, along with Associate Professor Jeffery Meyer, director of ASU orchestras, and Professor David Schildkret, director of choral activities, discussed the idea and began bringing in potential collaborators.

DeMaris said a commonality among all the collaborators — from music to dance, design and the students — was that, though they did not know what they were getting into, they all realized just how special it was.

“The work is a deeply malleable landscape that can be filled in so many different ways, and we were in a unique position to bring a tremendous amount of creativity and talent to that landscape and really make it a contemporary statement on today’s fractured and contentious world,” Meyer said. “The work gave us the opportunity to feature so much of what makes ASU a tremendous artistic force and breathe new life into this masterpiece of Bernstein’s output.”

Meyer said that without a doubt the most challenging aspect of the production was marshalling the mind-boggling number of unique, diverse and disparate performing forces.

“It was a challenge and a thrill unlike I have ever experienced,” he said.

As the music director, Meyer was the center of gravity for the more than 300 performers, which included singers, soloists, instrumentalists, dancers, a marching band, a rock band, a jazz band and the Phoenix Boys Choir, all working across vast spaces in different mediums.

“This experience is one of the most meaningful artistic ventures I've ever been a part of and I am very grateful for the recognition, so well deserved by all 300-plus individuals involved,” DeMaris said.

The ASU production of Bernstein’s “Mass,” including bringing in guest artist Jubilant Sykes, was made possible by funding from the ASU School of Music and a Mellon Foundation Grant through the Herberger Institute for Design and the Arts.

Ancillary events included community dialogues and panel discussions curated by Schildkret in collaboration with Barrett, The Honors College, The College of Liberal Arts and Sciences’ Center for Jewish Studies and the Lincoln Center for Applied Ethics.

Lynne MacDonald

communications specialist, School of Music, Dance and Theatre