A new dawn for supercomputing at ASU

Arizona State University’s supercomputing capability is about to jump light-years ahead with the addition of its new Sol supercomputer.

Sol will allow ASU to expand its suite of large-scale computing services to further advance research and learning across the university and the region.

“This represents an unprecedented investment in ASU’s research computing ecosystem,” said Sally C. Morton, executive vice president of the ASU Knowledge Enterprise. “Supercomputing already undergirds a great many of our efforts, and the addition of Sol provides researchers across all disciplines incredible additional resources to advance their work.”

An initiative of Knowledge Enterprise’s Research Computing facility, Sol was funded through the Office of the President and the University Technology Office’s Information Technology Bond and co-designed by Dell Technologies. As a Dell Technologies High Performance Computing and Artificial Intelligence Center of Excellence, ASU leveraged a close relationship with the computer and software giant to design and fabricate the new supercomputer. Multiple academic and industry counterparts also contributed to Sol’s design and specifications.

Currently in early access, Sol is expected to be available to all faculty and students by late summer 2022. Researchers can request access to Sol via Research Computing.

The power of Sol

“Sol adds capacity, power and speed to our supercomputing infrastructure by an order of magnitude,” said Neal Woodbury, vice president and chief science and technology officer at Knowledge Enterprise. “It will allow our researchers and partners to tackle even bigger challenges and create more impact.”

Sol is a welcome addition to ASU’s existing Agave supercomputer, which will also remain operational. Deployed in 2018, Agave has been continually updated with new components to meet the needs of researchers and students alike — but technology advances quickly, and Agave can only be upgraded so much.

“The Agave system is aging, and as parts age out, it becomes harder to keep them going,” said Douglas Jennewein, senior director of Research Computing, which manages ASU’s supercomputing resources. “Replacements are harder to come by, and warranties expire.”

Through regular upgrades, Agave has approximately doubled in size since launch, further necessitating a new supercomputer.

“We’re sort of hitting our heads on the ceiling with Agave,” Jennewein said. “We’ve filled up all of the network ports on our core network switches, and we’re running out of places to upgrade.”

Computers use central processing units, or CPUs, as their main “hub,” processing instructions from their components to carry out tasks. While your desktop or laptop typically uses a single CPU, supercomputers contain scores of them, clustered as nodes. This puts the “super” in “supercomputer,” as nodes pool their combined processing power and collaborate on large-scale problems beyond the scope of traditional computers. Sol will bring 18,000 state-of-the-art CPU cores to the table, not only increasing processing and computing power, but also expanding access to these resources.

Sol will also add more graphical processing units, or GPUs, to Research Computing’s overall infrastructure. These aren’t the GPUs that deliver pristine graphics in a gaming PC, however. Supercomputer GPUs are used for specialized scientific computing, such as machine learning and 3D modeling.

“We are making a big commitment to GPUs with this new cluster, and it’s exciting,” said Gil Speyer, director of the Computational Research Accelerator, which identifies opportunities to optimize and accelerate scientific approaches across all disciplines through high-impact hardware and software solutions.

“GPUs have received a lot of attention recently because they're really good for solving artificial intelligence-related problems,” Speyer said. “Not only are they useful when applied to image data sets, but researchers are now applying AI to other kinds of problems as well."

For instance, GPUs can power AI modeling of protein folding, a precise cellular process in which proteins become specific three-dimensional shapes to carry out biological functions correctly. Proteins that unfold or misfold contribute to many diseases, such as Alzheimer’s disease and cancer.

While GPUs have traditionally underpinned protein modeling, the employment of AI to more rapidly solve protein structures is a recent paradigm shift in the discipline, Speyer said.

While Agave launched with little GPU capability, Sol possesses 224 Nvidia A100 80-gigabyte GPUs right out of the gate. This will provide much broader access to GPUs, a transformative and highly sought-after computing capability that has been sorely lacking, Jennewein said.

Sol will lighten the load on Agave, which completed a record-breaking 9 million CPU-hours of computing in April 2022. Sol’s additional capacity will double that number to 18 million CPU-hours of computing when fully operational. That means each month Sol will complete the equivalent of 2,000 years’ worth of computation on a conventional computer.

True to its namesake, Sol is larger and produces more heat than its predecessor, requiring a more specialized facility than ASU offers. While Agave is located at ISTB1 on the Tempe campus, Sol is housed at the nearby Iron Mountain Data Center. The Phoenix data center is the first in North America to achieve a BREEAM design certification, a top ranking for sustainable construction. Iron Mountain’s facility features industrial cooling, FISMA high-compliant security, high-speed network connectivity and redundant power to ensure safety and reliability.

“We have plenty of room to grow, and we will,” Jennewein said. “Agave is now more than twice as big as it was when it launched, and I think we’ll see at least that same trajectory with Sol.”

Making light work of research problems

Research Computing’s organizing principle is to accelerate research and reduce time to discovery. Sol will allow them to step on the gas.

“I've seen the GPU alone just play such a transformative role in people's research,” Speyer said. “When you go from weeks to minutes to run a program, it really changes the way you look at your research.”

Research Computing supports an array of disciplines. Users range from physicists and engineers modeling novel materials to cosmologists characterizing soils on Mars, and from biologists analyzing the genomics of high-altitude monkeys to sociologists analyzing political discourse on Twitter.

“We have many bright, bright shining stars at ASU,” Speyer said. “We have such spectacular faculty, and I am always just blown away by the things that they do — and I get goosebumps thinking about the processing power that will soon be available to them through Sol.”

Antia Sanchez Botana, an assistant professor in the Department of Physics, plans to put that processing power to good use. She is a theoretical physicist who investigates the properties and behaviors of novel materials — work she says wouldn’t be possible without supercomputers.

Last fall, Sanchez Botana and a team of ASU researchers discovered a new superconducting material, publishing their supercomputer-enabled findings in the journal Nature Materials. Their work paved the way for the discovery of an exotic material by a group from MIT soon after.

“Both articles were possible thanks to the resources of Research Computing,” she said. “It has been the most useful resource to develop my research at ASU starting Day One, basically. The ease of use and the resources provided by the personnel have been outstanding to kick-start my research at ASU."

Bringing more into Sol’s orbit

Since its launch, Agave has steadily increased its user base. As part of ASU Core Research Facilities, Research Computing resources are also available to researchers at the University of Arizona and Northern Arizona University. Supporting not only a range of research projects but academic courses as well, the growth in users is due in large part to ongoing outreach and education efforts.

“One of our goals — and challenges — is to raise awareness amongst disciplines and domains that might not otherwise know that the resource exists or how they can use it for their research,” said Marisa Brazil, associate director of outreach and engagement in ASU’s Research Technology Office.

Brazil’s team helps faculty and students learn about the supercomputing resources available to them, offering more than 50 courses on everything from introductory supercomputing and software language training to advanced computing techniques and applications.

>>Learn more about Research Computing’s training and education.

“We want to reach not only typical supercomputing users, but nontraditional domains as well,” Brazil said.

To do this, the team implemented a more user-friendly, web browser supercomputing interface.

“Making the supercomputer more accessible by providing an easy-to-use web interface has really lowered the barrier to entry and allowed us to bring in clients beyond our traditional users,” Brazil said.

Onboarding new users is a challenge if they don’t know the resource exists, however. With the launch of Sol, Brazil saw a unique opportunity to raise awareness of the resources available to the university and get people invested in the new addition. She launched a naming competition, drawing in about 200 suggestions for the new supercomputer.

Around 15 top contenders were submitted to the Research Computing Governing Board and then to ASU leadership, where Sol emerged as the winner. The name was submitted by Tullio Geraci, a chemistry doctoral student in the School of Molecular Sciences.

“I feel really honored that the name I submitted was chosen,” said Geraci, who was trying to encapsulate different aspects of Arizona, ASU and the community with the name “Sol.”

As a winner of the naming contest, Geraci received a powerful Dell Precision 7760 Mobile Workstation with AI laptop, gifted by Dell. The computer also contains a Nvidia Data Science Workstation application stack and is capable of intense computational work, which Geraci says will serve him well in his studies.

Chemistry doctoral student Tullio Geraci (second from left) receives a Dell Precision 7760 Mobile Workstation for submitting “Sol” in the supercomputer naming contest from the Research Computing team (from left) Douglas Jennewein, Marisa Brazil and Gil Speyer, in front of Old Main. Photo by Andy DeLisle/ASU

“My old laptop is going on six years old now, so it was time for an upgrade,” Geraci said. “Winning the laptop was perfect timing, and it'll work great for the rest of my education and hopefully beyond.”

Brazil says she was pleased with the choice of “Sol,” the Spanish word for “sun.”

“As a Hispanic woman working in technology, I dedicate a lot of time and energy into ensuring that we are diverse, equitable and inclusive in all aspects of our work,” she said. “The name Sol not only reflects the diversity of our community at ASU and Arizona, but is also highly symbolic: energy, light, heat — all things that we need to thrive.”