Using augmented reality to teach real construction

Classroom environments have transformed in the last few decades. Chalkboards have become interactive whiteboards and handwritten transparencies on overhead projectors are now high-resolution document cameras or screen-sharing technology.

These changes were all made in an effort to solve problems, such as being unable to read a professor’s handwriting or show videos to large audiences.

But what happens when a new technology is implemented without a problem to solve?

Steven Ayer, an associate professor of construction engineering in the Ira A. Fulton Schools of Engineering at Arizona State University, found a passion for introducing technologies such as augmented and virtual reality into his construction engineering classwork, and even the construction industry. While he started with a backward approach of looking for problems the technologies could solve, his innovative teaching techniques could now lead to a more impactful way of educating the next generation of construction industry professionals.

Learning to make a dazzling technology practical

Augmented reality, or AR, is a way of adding digital elements to a live view, often by using the camera on a smartphone. While it’s been available for years, it became popular with the creation of social media tools such as Snapchat filters and mobile device games like Pokemon Go.

On the other hand, virtual reality, or VR, is an experience that seeks to place an individual in an entirely virtual world. This immersion is typically accomplished through the use of VR goggles or a headset.

Ayer says he was first introduced to augmented and virtual reality as an undergraduate architectural engineering student at Pennsylvania State University. He was working as a lab assistant managing the equipment when he says his interest took off.

“I got very dazzled by them,” says Ayer, who is a faculty member in the School of Sustainable Engineering and the Built Environment, one of the seven Fulton Schools.

Out of a desire to utilize AR and VR technologies, he says he found himself seeking problems that would fit the tools instead of the other way around, which made it difficult to measure the technologies’ impact and success.

As he entered his master’s degree program, also at Pennsylvania State University, Ayer says he had to reevaluate how he was looking at technology and the role it plays both in a classroom and in the real world.

“Through grad school, and certainly when I got to ASU, the shift was pretty polar opposite: Don’t start with the technology, start with the human and the problem that the human has, and how that technology is supporting them,” Ayer says.

This perspective is something he attributes to a subtle comment made to him during his doctoral studies by a mentor and co-adviser, Chimay Anumba, who is now at the University of Florida.

“In a very sort of understated way, I remember him just saying to me, 'Sometimes when you have a hammer, the whole world looks like a nail,'” Ayer says.

Addressing the problems

Over the years, Ayer has come to identify two major challenges AR and VR technology can help students face when it comes to construction education and entering the workplace.

The first is visualizing design concepts from two-dimensional plans that represent a three-dimensional space.

“We take this 3D concept; we have this building around us. And the way we communicate that is we dumb it down to flat paper plans,” Ayer says. “Instead, we can give them augmented reality glasses with the idea of saying, let’s make it easy to understand the design. They just see the model show up almost like it was there, but it’s virtual.”

Ayer says once students can get past the dazzled phase, they can dial in and learn the underlying construction competencies they need to be successful on the job.

For example, Ayer says he conducted a study a few years ago aimed at helping students explore buildings as if they were the end-users, like a facilities manager tasked with keeping a building up and running. He says they gave one group of students an augmented reality setup and the other group a computer setup. Both were given the same task of exploring the building to find flaws in the design. He says both groups could identify the flaws, but the group utilizing AR was able to come up with ways to improve the design and correct the flaws.

“The students using the computer setup, which was still a 3D model on the screen, knew something had to be considered, but couldn’t effectively articulate what about the design was problematic,” Ayer says.

Kieren McCord, a construction management doctoral student in the Fulton Schools, says while doing research with Ayer for her dissertation, she was inspired by the use of AR and the ability it gives students to visualize designs. 

“Physical builds are a great way to learn, but they can be extremely cost-prohibitive to bring to a classroom. So, a virtual simulation can be a valuable, cost-effective alternative,” McCord says.

She says there are far fewer physical restraints on virtual environments, meaning if you can dream it, you can create it in a virtual environment.

The second challenge Ayer says he wants to improve is job site safety.

What makes people change behavior is when they see or experience a bad thing.

— Steven Ayer, associate professor of constuction engineering

“We see a lot of times where we use very antiquated modes of teaching safety courses that are ‘chalk-and-talk’ lecture-style learning, which by almost any accounts have been ineffective, and, by empirical data on sites, still don’t stop injuries,” Ayer says.

“People from industry will say, ‘I didn’t care about safety until …’ and they’ll tell you a story of when they saw someone hurt, or someone lost a life,” he says. “And when they’re the one that makes the phone call to the husband or wife saying, ‘Your spouse isn’t coming home today,’ it hits them.”

With that impactful moment in mind, Ayer says he sought to create an experience for students that balanced real-life decisions with the dangerous outcomes created by mistakes.

“What makes people change behavior is when they see or experience a bad thing,” Ayer says. “What we’re doing with virtual reality is putting students and even industry personnel into this environment. But, unlike most virtual reality training environments that give a report card when something goes unrecognized and they fail to identify the hazard, we will show them the impact of their decision.”

Ayer says showing the impact is accomplished through the use of slow-motion video or animations. In addition, the negative effects never impact the AR user, but another character within the virtual environment.

“The situation would be to see if we can have a virtual artificial stimulus, the VR experience, trigger a real psychological response,” Ayer says. “So, now students or industry professionals can say, ‘I didn’t care about safety until I had this really impactful training experience that didn’t actually harm anyone.’”

He says the biggest challenge is overcoming how students and industry professionals first react when they experience these technologies. Many people find it “cool,” he says, but this impression isn’t what he wants to see.

“What are the metrics you would track to know if this provided a return on investment, or saved lives, or reduce rework, or whatever the underlying value is? How we get them back to thinking about that can be a challenge,” Ayer says.

It’s a challenge that he’s willing to take on because he says technology is something that education and industry need to take seriously, as it can be the solution to several problems, not only in construction, but in society as a whole.

“I think in the future, as technology becomes more prevalent, the role the human plays may be more critical because we will be slightly more out of the loop in terms of decision-making tasks and that kind of thing,” Ayer says. “So getting the human to interact with those technologies really well, for the time that they do, will be even more critical.”