Open Source Powers Supercomputing

Open source experts say it comes as little surprise that the top 500 supercomputers in the world are running Linux, and there are several aspects that make it a good environment for supercomputers: it is free, it is modular, and it allows for customization far more easily than commercially available operating systems.

Linux provides the flexibility needed to streamline the key aspects needed for massive parallelism,'' observes Kevin Curran, IEEE senior member and a professor of cybersecurity at Ulster University. "A proprietary OS would also hamper innovation."

"The open source nature allows new optimizations and patches to be applied instantly, rather than relying on central OS development teams,'' Curran adds. "Linux has exceptional network support and is more amenable to network reconfigurations, which can be enacted without complete system reboots."

Because the Linux kernel is generic, it can easily adapt to highly scalable processor deployments, Curran says. "It also scales well and copes with massive loads as required by supercomputers. There is also a substantial supercomputer Linux code base from which to build. This saves time and the expense of licenses."

The U.S. National Aeronautics and Space Administration (NASA) Beowulf system is one of the space agency's first supercomputers. It "marked a shift in the design of supercomputers that occurred in the [1990s], in a long-running battle between competing system architectures,'' recalls Bob Ciotti, chief architect and supercomputing systems lead at NASA's Ames Research Center in Mountain View, CA. Beowulf, he says, was "the first of a new breed of high-performance computer systems that integrated all commodity off-the-shelf hardware components and used open source software."

Today, NASA can pick and choose from a wide variety of commodity components such as processors, memory, networks, and storage to customize a high-performance computing system in a way that best suits the problems they are trying to solve, Ciotti says. Because almost all these components have well-defined and open interface standards, systems architects can mix and match components from different manufacturers as needed, he says. "Nearly all these systems run some form of Linux, which is critical to the stability and interoperability of all these different components."

The research world has also embraced Linux. The Pittsburgh Supercomputing Center (PSC), a joint effort between the University of Pittsburgh and Carnegie Mellon University, built its high-performance computing system, Bridges, on Linux, says the center's interim director, Nicholas Nystrom. Bridges combines high-performance computing with big data and artificial intelligence in a unified workflow to do modeling and simulations, Nystrom says. "The ability to do it all of that in one system is really transformative."

Daniel Lemire, a computer science professor at the University of Quebec who has contributed to open source projects, finds it "a little bit of a stunning fact" that the top 500 supercomputers all are running Linux. "When I first found out about Linux in the early '90s, I thought it was some kind of toy installed by grad students and not for serious computing,'' he says.

At NASA, Linux is proving to be anything but a toy. Among the many NASA projects currently running on Linux systems is the development of a Space Launch System Booster Separation Database, to ensure a space vehicle's safe and successful launch.

NASA researchers also are working on Estimating the Circulation and Climate of the Ocean (ECCO), running simulations on the Pleiades supercomputer "to study global, time-evolving maps of earth's ocean and sea-ice system at unprecedented resolution—1km horizontal grid spacing—resulting in more than 5k terabytes of model output from 24 million processor-hours on 70k cores,'' says Ciotti.

Another supercomputer project at the NASA Advanced Supercomputing (NAS) facility at the Ames Research Center is studying the complex airflows around small- to medium-sized electric multi-rotor vehicles to better understand turbulence, safety, efficiency, and noise production, he says.

Part of the appeal of Linux is driven by applications, says Nystrom. "Portability is very good in the Linux app base library; the frameworks are there." PSC's goal in designing Bridges, which runs on CentOS, was for it to be able to transparently scale from a user's laptop. That's a big change from the way supercomputers used to be built, he notes. Tools including python, R, MATLAB ,and Java were used to build Bridges; "the tools that are not like all the proprietary supercomputing systems vendors used to support, but they're what people are using now," especially those coming out school, he says. "That's what's moved the world largely to Linux. It's all about the applications and portability."

However, there is always room for improvement. Both Nystrom and Curran say an ongoing challenge with Linux is the issue of it being able to support new devices and drivers.

Curran cites other areas where he sees Linux lacking: fragmentation of desktop and package managers; limited software, and lack of stability.

An "important piece of the puzzle," says Nystrom, is the OpenHPC industry consortium, whose goal is to bring together "a complicated ecosystem" of complementary and compatible software for deploying and managing high-performance computing around Linux clusters.

Esther Shein is a freelance technology and business writer based in the Boston area.