Independent living

ASU engineering teams design assistive technologies that aid in everyday tasks


June 6, 2016

Jordan Rodriguez’s preferred mode of transportation around campus is walking, just like thousands of other Arizona State University students. But sometimes Rodriguez could use a little help getting his bearings.

The civil, environmental and sustainable engineering graduate student, who is legally blind, is getting help from computer science and software engineering students in the form of a belt that uses haptic feedback to indicate which direction is north. The Low Vision Food Management app was developed by computer engineering doctoral student and IGERT Fellow Bijan Fakhri, and computer science graduate students Jashmi Lagisetty and Elizabeth Lee to help their client manage food inventory. The Low Vision Food Management app was developed by computer engineering doctoral student and IGERT Fellow Bijan Fakhri, and computer science graduate students Jashmi Lagisetty and Elizabeth Lee to help their client manage food inventory. Photo by Pete Zrioka/ASU Download Full Image

This is just one of many projects students from computer science, computer engineering and software engineering, among other disciplines, are working on in CSE 494/591: Assistive Technologies, an Integrative Graduate Education and Research Traineeship (IGERT) Program course led by computer science assistant research professor Troy McDaniel. Through this class, McDaniel is getting students involved in directly helping members of the community by making independent living easier.

“I invite clients — individuals with disabilities, disability specialists, clinical partners and collaborators — to propose projects that address unsolved, real-world needs of those with sensory, cognitive and/or physical impairments,” McDaniel said. “Clients put together a description of their proposed semester-long, team-based project, which I then present to the students during the first day of class.”

Clicking a mouse, skimming the names of food items in the pantry, and knowing what emotion your conversation partner is expressing are just some of the proposed projects that can be made possible with an interdisciplinary focus.

This is the second semester McDaniel has taught the Assistive Technologies class, but he has more than a decade of experience working on assistive, rehabilitative and health-care applications in the Arizona State University Center for Cognitive Ubiquitous Computing (CUbiC), founded and directed by Sethuraman “Panch” Panchanathan.

CUbiC research encompasses sensing and processing, recognition and learning, and interaction and delivery solutions that are made possible through signal processing, computer vision, pattern recognition, machine learning, human-computer interaction and haptics. Students in McDaniel’s class solved similar problems with these technologies in their client-driven projects.

Employing other senses to convey information

Haptic technology uses the sense of touch to convey information, which can be helpful when visual cues are unavailable.

Graduate software engineering student Dhanya Jacob and computer science students Dylan Ryland, Alejandra Torres used vibrating pancake motors evenly spaced around a belt to help their client, Rodriguez, navigate while walking. The Haptic Compass Belt uses a digital compass board and an Arduino to determine which direction is north and vibrate the corresponding motor to indicate its direction to the wearer.

Enhancing senses to overcome difficulties

Computer engineering can also help people with low vision to enhance what they have difficulties seeing.

Computer engineering doctoral student and IGERT Fellow Bijan Fakhri and computer science graduate students Jashmi Lagisetty and Elizabeth Lee created the Low Vision Food Management app to remove a hassle of independent living for people with low vision. The app can perform tasks such as reading labels on containers and keeping track of food stocks in a pantry. Users label shelves with X and Y coordinates and sections labeled A, B, C and so on and enter in the app where they put an item. Later, when they need to take inventory, it can tell them what they have and where to find it in a cupboard.

When individuals with low vision or other visual impairments need more than just magnification for visual identification, software engineering graduate student Aditya Narasimhamurthy and computer science graduate students Jose Eusebio, Krishnakant Mishra and Namratha Putta created the QwikEyes Video Calling Assistant, a way for people to call a service and share live video from a smartphone camera to have an assistant help with a variety of tasks from reading a label to finding a small object to navigating. QwikEyes was proposed by client and ASU engineering student Bryan Duarte, who is legally blind, to form the basis of his start-up company under the same name. Bryan just completed his bachelor’s degree in software engineering and is continuing on to ASU’s computer science doctorate program.

Another team worked on a project that helps others understand a sensory impairment — in this case, hearing loss. Computer science undergraduate students Jeremy Ruano and Leonardo Andrade were tasked with creating a hearing loss simulator. Their app applied a filter via signal processing technology to simulate mild, moderate and severe hearing loss in several audio clips, which included music and conversation snippets. They also created a filter that simulated their client’s specific levels of hearing loss at different frequencies.

Reading and notetaking can pose challenges to a wide variety of people, from individuals with low vision to others with reading disabilities, such as dyslexia, and text-to-speech/speech-to-text can make this task more manageable. Computer science doctoral student Shang Wang and computer science graduate students Kulvir Gahlawat and Deepthi Pothireddy created the Audio Reader/Note Taker, an iPad-based reading and notetaking app that uses text-to-speech technology to read book content to users, and speech recognition that allows users to dictate notes.

Improving physical abilities with the help of computing

Clicking a mouse is a no-brainer for many, but extra help in the form of a neural interface can help people who have ALS, cerebral palsy and other conditions that can create physical limitations. Computer science graduate student Vigneshwer Vaidyanathan, Human and Social Dimensions of Science and Technology program doctoral student and IGERT Fellow Denise Baker, and computer science doctoral students Anish Pradhan and IGERT Fellow Corey Heath created a method of performing a Neural-Controlled Mouse Click using a neural interface that enables people to use a specific thought to signal they want to click the mouse, in this case, the visual of a plus (+) sign. The neural interface headset, a commercial product sold as an EEG headset, looks for an aggregate of four types of brainwaves measured while the user thinks about the plus sign to create the signature that will trigger a mouse click.

For helping people recover from physical injury, physical therapists want to monitor motions of their clients after assigning specific exercises. Computer science undergraduate student Sanket Dhamala, computer science master's student Arun Scaria, and computer science doctoral student and IGERT associate Meredith Kay Moore created Shift, a balance training app that can be used by physical therapists and their clients to hold those clients accountable for doing assigned exercises — and ensure they’re doing them correctly. For example, in a clock exercise where the user tilts his or her body in the 12 directions of a clock face, the user holds the smartphone with the app installed and leans in a given direction. The smartphone’s sensors track body movements and displays them on the app.

Improving cognitive skills for physical actions

Other individuals need help understanding their own and others’ physical expressions. Doctorate student in the Human and Social Dimensions of Science and Technology program and IGERT Fellow Shane Kula, graduate student James Kieley, and computer science doctoral student Prajwal Paudyal created the Body Language Mirror, a system that uses a Kinect sensor and the Unity game engine to read body language and identify the emotion being displayed to aid people who may not be aware of emotional body language they or others are expressing.

Delivering the prototypes

For most client proposals, students completed the project to a point where they had a working demo they showed to clients and the general public as part of the Assistive Technologies Demo in late April.

McDaniel was impressed that students worked hard to make helpful technologies to meet their clients’ specific needs.

“I was surprised that many teams went above and beyond project assignments, implementing additional features asked for by clients to make sure that the final prototype was exactly what the client wanted,” McDaniel said.

After the end of the class they’re encouraged to deliver the prototype to the client.

“I follow up with students after finals and keep in contact with clients to make sure they receive the prototypes,” McDaniel said.

Continuing development for increased usability

Though prototypes are being delivered, Shift and the Body Language Mirror will continue to be developed through student engagement.

Many functions have already been added to Shift, including uploading videos, push notifications and downloading data to a remote database so physical therapists can track progress. But right now the app must be run on a computer, and the phone must be connected via USB to be displayed. The team would like to incorporate Bluetooth or other wireless technology so the user isn’t tethered to a computer, incorporate tactile or audio feedback when the correct balance has been achieved and conduct user studies to determine usability of all ages and abilities.

The Body Language Mirror team finished their first stage of development, but they were unable to get applicable test subjects to conduct experiments, so they would like to find subjects to begin testing. The team also wants to be able to record and replay reactions. Their ultimate goal is to create a system that can rewire the mirror neuron system in the brain so test subjects would become aware of what their emotional body language looks like. Due to the brain’s plasticity, scientists believe it may be possible to train the mirror neuron system.

Creating new computing technologies that can help the world

While these projects targeted the explicit needs expressed by clients, most of whom have disabilities, the progress students achieved toward developing computing technologies are applicable to the implicit needs of the broader population. For example, we can all use help with taking notes, navigating new environments or adhering to routine exercise.

Monique Clement

Lead communications specialist, Ira A. Fulton Schools of Engineering

480-727-1958

 
image title

Top of their class

June 6, 2016

ASU boasts strong pool of award-winning and world-changing students

Arizona State University has no shortage of high-achieving Sun Devils making their mark on the world — and being recognized for it.

During the past academic year the university placed in the top 5 research universities nationwide for Fulbright scholarships to students and led the state in Flinn Scholars — in addition to boasting recipients of the prestigious Goldwater and Schwarzman awards.

Learn a little more about the great things these students are achieving.

Fulbright

Claire Cambron in South Korea

Claire Cambron, South Korea

Claire Cambron wanted a way to open her mind and her heart before she learns to heal.

Cambron, who is from Phoenix, won a Fulbright scholarship and just returned from spending eight months as an English teaching assistant at Jeungan Elementary School in Cheongiu, South Korea.

“I didn't get the chance to study abroad in college and I really wanted to travel before medical school,” said Cambron, who earned her undergraduate degree last year in biochemistry and genetics from the School of Life Sciences, in ASU's College of Liberal Arts and Sciences.

“I felt that by traveling, I could learn more about a different culture, which would help me be a more open-minded and receptive doctor and person. I also was looking for an opportunity that would push me out of my comfort zone and challenge me.” 

Jaxon Williams playing the guitar

Jaxon Williams, Spain

Jaxon Williams has spent years honing his skill in classical guitar.

And the best way to make the passion of his life become his career was to spend time in the hub of classical guitar, which is Seville, Spain.

Williams was awarded a Fulbright grant and is now pursuing his master’s degree in Spanish classical and flamenco guitar from the School of Music in the Herberger Institute for Design and the Arts.

“I've always felt that to reach the next level as a musician, I need to live abroad and connect with the classical guitar's roots, which are in Spain,” said Williams, who is originally from Ashland, Oregon.

“Much of this music is passed on orally and in person, so it's very hard to learn these things outside of Spain.”

Other recipients are:

Andrew Ahearne, Luxembourg
Mary Lou Fulton Teachers College

Chase Fitzgerald, South Korea
biological sciences with a minor in global health, School of Life Sciences, College of Liberal Arts and Sciences

Ashley Hagaman, Nepal
doctorate, global health, School of Human Evolution and Social Change, College of Liberal Arts and Sciences

Bridget Harding, South Korea
sustainability, School of Sustainability

Allegra Hyde, Bulgaria
master’s of fine arts from Department of English, College of Liberal Arts and Sciences

Eva Jeffers, India
global health, School of Human Evolution and Social Change, College of Liberal Arts and Sciences

Michelle Kunkel, Senegal 
master’s degree, teaching English to speakers of other languages, Department of English, College of Liberal Arts and Sciences

Thomas Lepke, Czech Repulic
Mary Lou Fulton Teachers College

Amalie Sielaff, Germany
German, College of Liberal Arts and Sciences 

Jenna Smith, South Korea
classics and philosophy, with a minor in symbolic systems, College of Liberal Arts and Sciences

Sophie Sylla, South Africa
Mary Lou Fulton Teachers College

Cameron Tattle, Sri Lanka
global studies, School of Politics and Global Studies, College of Liberal Arts and Sciences

Derek Townsend, Armenia
German/business minor, College of Liberal Arts and Sciences and W. P. Carey School of Business

Mitzi Vilchis, Mexico
secondary education, Mary Lou Fulton Teachers College

Matt Ykema, the Netherlands
molecular biology and economics, W. P. Carey School of Business and College of Liberal Arts and Sciences

Lin Wang, Taiwan
chemistry and dance, Herberger Institute for Design and the Arts and School of Mathematical and Statistical Sciences

Allison Weidemann, Turkey
undergraduate degree in sustainability, master’s degree in global health, School of Sustainability and School of Human Evolution and Social Change, College of Liberal Arts and Sciences

Fulbright Summer Institute

Six Barrett, the Honors College students were chosen for the Fulbright Summer Institute this year. The institute is a program of the U.S.-U.K. Fulbright Commission, the only bilateral, transatlantic scholarship program offering awards and summer programs for study or research in any field, at any accredited U.S. or U.K. university. Each year, approximately 60 U.S. and U.K. undergraduate students are selected for the program.

ASU's recipients are:

Austin Cotter, biology
Victoria Crynes, business
Advika Dani, biochemistry
Anirudh Koka, economics
Carolina Marques de Mesquita, political science
Maggie Tucker, philosophy

Flinn Scholars

portrait of ASU Flinn Scholar Maggie Zheng

Maggie Zheng performed her first surgical procedure when she was just a preschooler.

Granted, it was on one of her stuffed animals.

But in hindsight it was a relevant precursor to where she finds herself today: one of an elite group of winners of this year’s Flinn Scholarship, who will be attending Arizona State University in the fall.

Zheng, who will study biomedical sciences in the College of Liberal Arts and Sciences, has loved the idea of being a doctor since she was child watching medical shows on public television.

“I just always found it really fascinating, so I want to become a surgeon,” said Zheng.

She is a member the 31st class of Flinn Scholars. The award, which started in 1985, is offered to outstanding Arizona high school students on the condition that they attend one of the state’s three public universities: ASU, which will have 13 Flinn scholar enrollees this fall; the University of Arizona, which will have six Flinn scholars; or Northern Arizona University, which will have one.

Other recipents are:

Aidan McGirr is going to study astrophysics in the School of Earth and Space Exploration. He attends Anthem Preparatory Academy.

Martín Blair is coming to ASU from the Phoenix Union Bioscience High School. He’ll study mechanical engineering in the Ira A. Fulton Schools of Engineering.

Rohini Nott, at BASIS Chandler, will major in biology and society in the School of Life Sciences.

Maeve Kennedy, from Westwood High School in Mesa, plans to study chemical engineering in the Fulton Schools.

Ivette Montes Parra, also from Westwood High School, will also go to the Fulton Schools, to study mechanical engineering. 

Cameron Carver of Sabino High School in Tucson will be a mechanical engineering student in the Fulton Schools.

Anagha Deshpande, from Hamilton High School in Chandler, will study genetics, cell and developmental biology as a biological sciences major in the School of Life Sciences.

Andrew Roberts will study electrical engineering in the Fulton Schools. He studied at Westwood High School.

Yisha Ng wants to be an aerospace engineer. The Flagstaff High School student will study in the Fulton Schools.

Enrique Favaro, from the Tempe Preparatory Academy, is going into accountancy at the W. P. Carey School of Business.

Vaibu Mohan, focused on the STEM subjects at BASIS Scottsdale, will immerse herself in performance and musical theater in the Herberger Institute for Design and the Arts. 

Tina Peng, from Chandler Preparatory Academy, will study computer science in the Fulton Schools.

Goldwater

Three students from ASU were named Goldwater Scholars, a prestigious national scholarship awarded to students across the nation who are doing research in science, technology, engineering and math fields.

Barrett Anderies, Christopher Balzer and Kaleigh Johnson, all juniors at Barrett, the Honors College, were awarded this scholarship out of only four chosen from the state of Arizona.

Each university can nominate only four students per year, and fewer students overall were awarded scholarships than in many years prior.

“The academic atmosphere of living and working at Barrett, the Honors College, elevated my performance in my classes and led me to get involved in research in the first few weeks of my freshman year,” said Balzer, a chemical engineering major. “Having other researchers pouring time and experience into my life gives me insight that I wouldn’t have otherwise.”

“My involvement in this engineering academic program led me to explore sustainability from many different angles, including entrepreneurship, community service, interdisciplinary course work, global studies and research,” said Johnson, a chemical engineering major.

Anderies, a double major in biomedical engineering and mathematics, offered a similar sentiment, reflecting on the breadth of options for undergraduate research that allows students to experiment and tailor their academic experience.

“I had access to a huge number of research opportunities in both mathematics and engineering. These opportunities allowed me to extensively explore and develop my interests,” he said. “I am especially appreciative of the numerous funding opportunities available to undergraduates interested in conducting research.”

Schwarzman

Jessica Hocken is fascinated with Chinese culture. Since middle school, she has wanted to visit China and explore the country.

“I’ve always wanted to explore and experience everything China has to offer," Hocken said.

This fall, she will have the opportunity to go to China as a Schwarzman Scholar. Hocken will be a part of the inaugural class of scholars, a group of 111 students who will study at Tsinghua University for a year to earn a master's degree and broaden their understanding about China.

The scholarship, which accepted less than 4 percent of all applicants, fits well with Hocken’s entrepreneurial spirit.

Alfred I. duPont-Columbia Award

Hooked Documentary

Last fall, ASU's Walter Cronkite School of Journalism and Mass Communication won an Alfred I. duPont-Columbia Award, which has recognized the very best in broadcast journalism for more than 70 years. 

Cronkite News, the school's student-produced news division of Arizona PBS, received the honor for “Hooked: Tracking Heroin’s Hold on Arizona,” a 30-minute documentary produced in association with the Arizona Broadcasters Association (ABA), which reached more than 1 million Arizonans. The report, the final product of more than 70 dedicated student journalists and which aired on all 33 broadcast television stations and 93 radio stations in Arizona in January, examined the rise of heroin use and its impact on the state.

The win marks just the third time in the history of the duPont Awards that a Phoenix-based news operation has received the honor. Cronkite News joins 12 News KPNX-TV, which won the award last year, and KOOL-TV, which won the award in 1979 when it was the region’s CBS affiliate.