Note-Taker device promises to help students overcome visual impairments

See a video about Project Note-Taker

Millions of students will someday be less hindered by their visual impairments if a recent Arizona State University graduate and three current ASU students realize their ambitions.
 
David Hayden, Michael J. Astrauskas, Qian Yan and Shashank Srinivas are developers of the Note-Taker prototype. It’s a portable custom-designed camera connected to a pen-and-multitouch tablet personal computer.
 
The Note-Taker camera can tilt up and down, sweep side to side, and zoom in on a target.

The tablet computer provides a split-screen display. One half of the screen has a window that shows live video from the camera, while the other half has a window used for handwriting or typing notes.

The device is designed help students with low vision follow classroom instruction and take notes as easily as fully sighted students.

The split-screen interface enables students with visual impairments to glance back and forth between the live video view of the classroom presentation and their notes.

The video window allows the user to aim and zoom the camera by using simple dragging, tapping or pinching motions within the video window.

Calling themselves Team Note-Taker, the four developers of the device won the first prize in the Software Design category of the Microsoft Imagine Cup U.S. Finals last spring. Then in July they took second place in the same category of the Imagine Cup World Finals, the premier international student technology competition.

Those achievements are only a warm-up exercise, the team members say.

Now they will concentrate on the ultimate purpose of their endeavor:  To develop next-generation Note-Taker prototypes and see the device developed into a viable commercial product that can be made available to students with visual impairments.
 
“It’s not a manufacturable product right now, but I think the team can improve the design to the point where the Note-Taker could be manufactured affordably and marketed,” says John Black, an ASU research scientist and Team Note-Taker’s mentor.

Astrauskas is working toward an undergraduate degree in electrical engineering, and Srinivas toward a graduate degree in computer science, in ASU’s Ira A. Fulton Schools of Engineering.

Yan is seeking a master’s degree in The Design School in ASU’s Herberger Institute for Design and the Arts.

Hayden, the initial inventor of the Note-Taker – who created the device to help him overcome his own visual impairment – has earned an undergraduate degree in computer science in the Fulton Schools of Engineering, as well as a math degree through the ASU’s College of Liberal Arts and Sciences.

He spent the summer in an internship with Google, and will begin studies this fall toward a doctorate in computer science at the Massachusetts Institute of Technology.

Despite the demands of their academic pursuits, the members of Team Note-Taker intend to push steadfastly forward with refinements to their Note-Taker design.

Their first-place prize in the Imagine Cup U.S. Finals earned a $25,000 grant from Microsoft Corporation for the Center for Cognitive Ubiquitous Computing – or CUbiC – the laboratory in ASU’s School of Computing, Informatics, and Decision Systems Engineering where much of the work on the Note-Taker has been done. The award will provide support for their ongoing work on the Note-Taker’s development.

CUbiC focuses on the design of technologies and devices to assist people with perceptual or cognitive disabilities.

The team will conduct usability testing, part of which will include giving visually impaired students Note-Taker prototypes to use in their classes. The students will then provide feedback on how well the Note-Taker meets their needs.

The current Note-Taker is a third-generation model. The team plans to develop a fourth-generation prototype that can be more affordably manufactured and is more compact. A planned fifth-generation device will take advantage of the latest advances in digital video technology to boost the Note-Taker’s performance.

Hayden wants to see the device made smaller and lighter, with reduced noise from the motors that control its moving parts.

The team plans to add audio and video recording functions that provide synchronized playback of lectures, along with the corresponding handwritten or typed notes – enabling students to quickly review lessons after classes.

These improvements will allow students to select any portion of the handwritten or typed notes previously taken in class, and then see the video that was recorded while the notes were being taken.