Students in ASU’s extended-reality degree programs use world building for social good

Editor’s note: This story is featured in the 2023 year in review.

Students in Arizona State University’s extended-reality degree programs are learning how to build worlds in order to solve problems using the latest technology — and they're already earning accolades.

Extended reality, called XR, creates immersive, interconnected three-dimensional worlds using technologies that include virtual reality, augmented reality, artificial intelligence and blockchain.

The four transdisciplinary master’s programs in the Herberger Institute for Design and the Arts and the College of Global Futures teach learners how to tell stories and create experiences to solve problems, widen access to more people and seek justice – all by employing visions of futures and alternative worlds that they create.

The degrees are:

• Narrative and Emerging Media – Based in the ASU California Center Broadway in downtown Los Angeles, this Master of Arts program is a collaboration between the Sidney Poitier New American Film School and the Walter Cronkite School of Journalism and Mass Communication. Students develop skills in creating immersive content using technologies such as virtual production and 360 video.
• Experience Design – This Master of Science in design, based in the Media and Immersive eXperience (MIX) Center in Mesa and offered through The Design School, focuses on design practices for virtual experiences in extended-reality technologies, including environmental design, rapid prototyping, storytelling and world building, applied to real-world challenges.
• Media Arts and Sciences (Extended Reality Technologies) – In this Master of Science program, based at the MIX Center and offered through the School of Arts, Media and Engineering, students learn to develop the technologies and software used to create virtual worlds and immersive experiences.
• Futures and Design – This Master of Science program is based at the MIX Center and offered in partnership with the School for the Future of Innovation in Society. Students in this program focus on the creative tools used to imagine alternative futures, such as speculative design, science fiction and foresight studies.

Nearly 40 students are enrolled across the four programs, and the goal is to increase that to 150–250, according to Jake Pinholster, founding director of the MIX Center and executive dean for enterprise design in the Herberger Institute for Design and the Arts.

Programming experience not needed

Students come from a wide variety of undergraduate degree backgrounds, he said, including media arts and science, graphic or industrial design, user experience and even film, theater and sculpture.

The master’s degrees, called XRts, for “extended-reality technologies,” do not require proficiency in coding, except for the media arts and sciences degree, which focuses on software development, he said.

“A lot of students feel hesitant about technology so we try to make sure they know you don’t need to come in with programming experience and you don’t need to be a Python wizard,” he said.

At the ASU California Center Broadway, students in the Narrative and Emerging Media program are getting help with the technology side with modules on coding, according to Nonny de la Peña, founding director of the program and a professor of practice.

“I’m training students to become producers and directors, not engineers. But they need to know how the sausage is made, and I do everything to support that,” she said.

And students are embracing it, Pinholster said.

“You just can’t be scared,” he said. “If you’re courageous with technology, even if you don’t know it now, you’ll be fine.”

Students in Los Angeles and Mesa can work together.

“Jake and I work hard to collaborate across campuses,” de la Peña said. “With the haptics lab, the gear we bought is also at the MIX, so there are ways for people (to work) together. We have a wonderful and fluid connection across the programs.”

The XR degrees prepare students for a variety of careers, not only in entertainment and journalism, but also in urban planning, health care and education.

“A minority of the work our students are doing has an entertainment focus,” Pinholster said. “It’s not the next round of Beat Saber or augmented-reality bonus content for a Marvel movie. Most of the worlds our students are creating using these technologies are for workforce development, health care, safety training, conservation,” he said.

Students already putting their degrees to work 

Much of the work is to benefit society. Students are now working on an AI project to help people who are vision-impaired, and students in California learned about how to use blockchain technology for social good.

Other students are building worlds to create empathy.

One of de la Peña’s students, Cameron Kostopoulos, debuted his project, “Body of Mine VR,” an immersive virtual-reality experience, at the SXSW 2023 festival in March. It also was screened at the Venice Film Festival earlier this month. The experience places the viewer into another body for an exploration of gender dysphoria and trans identity. He used a combination of several technologies, including Planar virtual-production technology, to create “Body of Mine VR,” which combines full-body, face and eye tracking with audio interviews. At one point, the viewer looks into a mirror and sees themselves blink.

Kostopoulos’ project won “Best XR for Change” award from the Games for Change Festival event in June, as well as the Special Jury Award at SXSW and a student BAFTA Award.

Calvin Stanley is a second-year student in the Experience Design degree program, having earned his undergraduate degree in human-centered design at a university in India.

“After my undergrad I worked a little bit in UX design, and being in the field of technology and seeing how the trends were going, I felt like I was working on something very screen-oriented and 2D,” he said.

“I saw that the field was going to 3D and XR, so I wanted to get into what was next, and this was the best university I found that offered that skill set.”

He has been learning to use the Unreal world-building engine, motion-capture technology, immersive digital platform Spatial as well as 3D printing and laser cutting.

The technology is constantly being updated.

“Once I feel like I’ve learned something and have a grasp of how it can be used, the next update rolls in or the next resolution comes in and we have to think about how it will be in the future, because this is future-based,” Stanley said.

“I’m now working on a project in 8K, but maybe in two years it will be in 16K.”

Stanley, who expects to graduate in the spring, already has his projects out in the public. As a student worker in ASU's Resilient Visions CoLab, he helped to create “Mission to the Future: Arizona in 2045,” an interactive exhibit at the Arizona Science Center. The lab used motion capture to create an animated AI character that talks to kids.

Now, Stanley and the lab are working on a project with the City of Hope Cancer Center.

“We’re working with women who are undergoing breast cancer treatment, and our research is along the lines of how a natural environment and breathing techniques can reduce stress,” he said.

“People in chemotherapy can’t always have access to those environments, so we’re trying to see if the same drop in stress levels happens in a VR environment. I’m in charge of creating the photorealistic environments of five national parks across the U.S.”

Stanley also is in discussions with the city of Mesa on an urban-planning project that would create an immersive visualization of different architectural approaches.

“With these projects, the narrative drives the technology and vice versa,” he said. “Some things are tough to explain without actually experiencing them inside of a space.”

Undergrads from any university invited to Worlds for Change Challenge

Undergraduates who are interested in learning more about world building and ASU’s extended-reality degrees are invited to participate in the Worlds for Change Challenge Kickoff this Saturday, Sept. 23, from 9 a.m. to 5 p.m. The event, open to any undergraduates or recent graduates, not just those from ASU, will be held in person at the MIX Center in Mesa or virtually on the Spatial platform.

No experience is required, and MIX Center artists and engineers will facilitate workshops throughout the day.

“The event is partially intended to drive applicants into the grad programs but also to diversify and make more accessible and inclusive the population that has access to authorship of these worlds and the ability to use these tools,” Pinholster said.

“We’re asking youth from around the nation and world to submit their vision of a positive future they want to see, and once submitted and signed up, they’ll get access to workshops and faculty to walk them through the process of translating their vision into a virtual world, whether that’s a video game, VR experience or augmented-reality app.”

The challenge will continue throughout the academic year, and the grand prize will be two financial aid packages worth $60,000 each that will fully cover tuition to one of the grad programs. Those who aren't eligible for the graduate school prizes can win $10,000 in cash prizes and XR technology.

The winners will be announced at the SXSW 24 festival next March in Austin, where ASU will be a presenting sponsor.