Helping robots follow a new path

​While science fiction movies depict robots moving freely on their own, sometimes running to avoid perilous explosions or collapsing buildings, today’s technology doesn’t have that capability — yet.

Ying-Cheng Lai, an ASU Regents Professor of electrical engineering in the Ira A. Fulton Schools of Engineering at Arizona State University, his doctoral students and two collaborators from the U.S. Army DEVCOM Army Research Laboratory have moved fantasy one step closer to reality with their new method for programming robots’ movements.

Lai led a research team in the use of reservoir computing, a type of machine learning, to program a robot to move two arms on a 2D plane in a computer simulation. This method allows the robot to change trajectory between predefined paths with only partial knowledge of the surrounding environment.

“The innovative aspect of this approach lies in its capacity to operate effectively with only partial observation of the state of the system, in contrast to the traditional requirement of comprehensive knowledge about the robot and its environment,” Lai says. “It is akin to attempting to solve a jigsaw puzzle by focusing solely on a few pieces rather than the complete image.”

Robots’ movements are traditionally programmed using a mathematical function-based machine learning method known as linear quadratic tracking, which requires in-depth knowledge about both the machine and its environment. Frequent time-consuming and finicky recalibration are required, limiting the effectiveness of movement programming and prohibiting on-the-fly trajectory changes sometimes necessary for robots to avoid obstacles in their paths.

Lai says this new research, supported by the U.S. Air Force Office of Scientific Research and the U.S. Army Research Office, focuses on enabling a breakthrough steppingstone to unlock potential applications in autonomous drones and ships, devices to aid humans in performing certain tasks, laser cutting tools to create complicated shapes and more.

The science journal Nature Communications has published the research findings in the team’s paper “Model-free tracking control of complex dynamical trajectories with machine learning.”

Rewriting robot movement programming

The shortcomings of linear quadratic tracking motivated Lai and his team to seek new solutions for robot control.

“Linear quadratic tracking is like attempting to teach a pet dog various tricks, relying solely on one type of treat for each trick,” he says. “Such an approach may have limited applications due to its lack of flexibility.”

The research team set out to use partial observations of the system’s state, which requires the robot to learn from its memory of previous experiences. Ultimately, Lai and his collaborators chose to focus on a technique called reservoir computing because of its ability to instill memory that gives a system learning ability.

Reservoir computing allows a system to correlate its training inputs to outputs and analyze what outputs work best to achieve the system’s programmed goal. Lai compares reservoir computing to the distinct ways ripples interact with each other in a pond depending where and how forcefully stones are thrown into the water.

“If we observe the ripples carefully, we may be able to tell where and how hard the stones were thrown, even without seeing the actual throwing process,” Lai says.

“We can relate the pond to the ‘reservoir,’ or memory, in reservoir computing, where the dynamic system reacts to and interprets inputs, which are similar to the stones thrown into the pond. The way the reservoir reacts to these inputs can be analyzed to make sense of the input or even predict future inputs.”

Reservoir computing’s adaptability makes it well suited to the research of Lai and his team, which includes electrical engineering doctoral students Mohammadamin Moradi and Zheng-Meng Zhai. With fine-tuning and further experimentation, the team achieved its goal of controlling two simulated robotic arms in any trajectory, along with the ability to switch to another desired path without prior notice.

A significant learning experience for students

For Zhai, the research paper’s primary author, the work represents a significant accomplishment of his doctoral thesis project.

Zhai performed the project’s computations and ran its simulations, in addition to drafting the paper summarizing the investigation’s findings. He says the experience even inspired him to pursue a potential career path in academia.

“Through this research, I mastered advanced analytical techniques in nonlinear dynamics and became more familiar with control theory,” Zhai says. “After this successful outcome, maybe I will do more work in machine learning control and try to be a professor in the future.”

Moradi, the research paper’s secondary author who contributed to each step of the project, was chosen to join the research team because of his expertise in system control theory. He says the project was a success because of ASU’s extensive robotics and machine learning resources.

“We had access to state-of-the-art computing facilities, and we were able to collaborate with world-class researchers in the robotics and machine learning fields,” Moradi says. “ASU also has a strong focus on interdisciplinary research, which was essential for this project, as it required expertise in both fields.”

Transforming the future with robotics and machine learning

The research team plans to take full advantage of ASU’s machine learning and robotics expertise, along with what they’ve learned from this project, to further develop trajectory control capabilities for robots. Zhai says the researchers plan to tackle the challenge of applying their findings to controlling robots in three-dimensional space, which would enable a robot such as an autonomous drone to move vertically and horizontally.

The researchers are also investigating the integration of model predictive control, a method of system control in which a mathematical function seeks to use minimal resource costs to produce a desired outcome in the use of machine learning in robotics. Zhai notes model predictive control is useful for its ability to operate systems with multiple variables while considering operational constraints.

“Integrating model predictive control with machine learning for a dynamical system is an exciting fusion of classical control theory and modern computational techniques,” he says.

Moradi is also excited about the future robotics control possibilities.

“Machine learning can be used to develop controllers that are more robust, adaptable and efficient than traditional controllers,” he says. “I believe machine learning control will play a major role in the development of next-generation robotic systems.”