The Latin American Supercomputing Ecosystem for Science

Large, expensive, computing-intensive research initiatives have historically promoted high-performance computing (HPC) in the wealthiest countries, most notably in the U.S., Europe, Japan, and China. The exponential impact of the Internet and of artificial intelligence (AI) has pushed HPC to a new level, affecting economies and societies worldwide. In Latin America, this was no different. Nevertheless, the use of HPC in science affected the countries in the region in a heterogeneous way. Since the first edition in 1993 of the TOP500 list of most powerful supercomputing systems in the world, only Mexico and Brazil (with 18 appearances each) made the list with research-oriented supercomputers. As of June 2020, Brazil was the only representative of Latin America on the list.

HPC represents a strategic resource for Latin American researchers to respond to the economical and societal challenges in the region and to cross-fertilize with researchers in the rest of the world. Nevertheless, the Latin American countries still lag behind other countries in terms of size and regularity of investments in HPC. The table here compares the HPC capacity of the BRICS countries, which together represent almost half of the world population. As a reference, in 2018, South Africa’s GDP was 29.1% lower than Argentina’s and only 11.2% higher than Colombia’s, the two countries in Latin America with largest GDPs after Brazil and Mexico. In spite of the overall picture described here, the landscape of the Latin American HPC ecosystem for science is promising, with many initiatives and outstanding concrete results.

Back to Top

HPC in Latin America: The Cases of Brazil, Mexico, and Uruguay

Comparing the situation of HPC in three different countries in Latin America helps understanding the region’s distinctions, not only in terms of overall capacity, but also in terms of adopted policies for creating and operating this kind of scientific instrumentation systems. The presented examples are representative of other important initiatives in the region, for example, NLHPC in Chile, Tupac in Argentina, SC3UIS in Colombia, and CeNAT in Costa Rica.

In Brazil, the National Laboratory for Scientific Computing (LNCC) is the major player for HPC services to the scientific community. LNCC is a public, interdisciplinary research center with a mission-oriented approach to computational and mathematical modeling and simulation of complex problems. LNCC coordinates a network of 10 HPC centers (SINAPAD) funded by the Brazilian Ministry of Science, Technology and Innovations (MCTI) (see Figure 1). The SINAPAD centers offer resources, training, and scientific portals^9,11 to the Brazilian community.

Figure 1. The main centers of SINAPAD, Brazil (www.lncc.br/sinapad).

Figure. The Santos Dumont supercomputer at LNCC, Brazil (www.lncc.br).

Figure. The ABACUS Laboratory at CINVESTAV, Mexico (www.abacus.cinvestav.mx).

Figure. Cluster-UY at Massera datacenter, Uruguay (cluster.uy).

LNCC hosts Santos Dumont, which until June 2020 was the largest supercomputer dedicated to research in Latin America. With a performance of 4.2 petaflops and a storage capacity of 2.3 petabytes, Santos Dumont plays a central role in promoting high-quality research initiatives in Brazil.

---

Comparing the situation of HPC in three different countries in Latin America helps understanding the region’s distinctions, not only in terms of overall capacity, but also in terms of adopted policies for creating and operating this kind of scientific instrumentation systems.

---

The Santos Dumont supercomputer has housed more than 160 peer-reviewed projects from institutions spread throughout Brazil, which together have consumed more than 500 million CPU hours, produced more than 300 scientific papers in scholarly articles in journals, and supported more than 60 Ph.D. graduations since its inauguration in August 2016. An example of a strategic project on Santos Dumont is the computational modeling of key aspects of the operation of Sirius, the new fourth generation Brazilian synchrotron light source (see https://www.lnls.cnpem.br/sirius-en/). Besides, Santos Dumont has fostered important international collaborations, such as in the rational design of Zika vaccine candidates.⁷ The LNCC research staff also uses Santos Dumont’s capacity to develop high-impact projects, including the development of efficient computational models that quantify the functional severity of coronary artery stenosis,³ and the genome sequencing, at the beginning of the COVID-19 pandemic, of 19 viruses from different regions of Brazil, demonstrating the state of community transmission.

In Mexico, ABACUS, the Laboratory for Applied Mathematics and HPC at the Center for Research and Advanced Studies (CINVESTAV) exemplifies the diverse HPC initiatives grouped in the Mexican Network in HPC (REDMEXSU), whose main members are shown in Figure 2.

Figure 2. Main members of the Mexican HPC Network—REDMEXSU (www.redmexsu.mx).

CINVESTAV is a public institution ranked within the top Mexican National Research Centers and Postgraduate Education Institutions and through ABACUS and LANCAD a prime provider of HPC resources to the scientific and technological communities in Mexico. CINVESTAV has an outstanding record regarding initiatives to foster interaction between academia, government, industry and society, in conjunction, with a very successful track of worldwide collaboration. ABACUS houses one of the principal Latin American research supercomputers, placed 255th in the TOP500 list of July 2015, with an updated total performance of ~0.5 petaflops and a storage capacity of 1 petabyte.

Figure. GDP and investment in supercomputing in the BRICS countries (sources: World Bank, TOP500 List).

Since 2016, ABACUS has supported scientific efforts to solve complex problems through collaborative work between different research communities spread throughout Mexico. ABACUS has assisted more than 140 research projects and over 250 academic articles. Examples of projects are: numerical simulation of vascular malformations in the brain; studies of racemization of molecular helices; numerical simulation of environmental hazards;⁶ sandpile simulations and applications;⁵ covering arrays and software testing; cryptographic algorithms; simulation of subatomic processes; simulation of astrophysical phenomena;⁴ and the ENERXICO Project: Supercomputing and Energy for Mexico (see https://enerxico-project.eu). Some projects have direct impact in daily life and industry, for example, the numerical simulation of volcanic ash dispersion near important airports of the country and early warning systems for coastal floods.

In Uruguay, National Supercomputing Center (Cluster-UY) is an initiative for operating a collaborative scientific HPC infrastructure to foster research and innovation projects with high computing demands. The platform and services provided by Cluster-UY are available to all research and development efforts by scientific institutions, academia, and public/private companies. Important institutions support the initiative, as described in Figure 3.

Figure 3. Institutions supporting Cluster-UY, Uruguay.

Cluster-UY applies a collaborative self-managed and self-financed model based on work and support agreements, signed with institutions, organizations, and companies, to guarantee sustainability. Partnerships allow consolidating the collaborative model and are very valuable to open the Center to a wide variety of users, with special emphasis on promoting inclusive development in areas with social impact (health, education, research for development, and so forth), under the “University of Development” model.⁸ The approach encourages and consolidates an open data model, closely linked to the ideal of collaborative systems. Furthermore, an egalitarian model is applied for access to the provided services. The same benefits are offered to all users. In turn, all users have the same responsibilities regarding the correct utilization, maintenance, and updating of the platform and services.

Cluster-UY supported more than 50 research projects that used more than 11 million hours, produced more than 250 articles, and supported 100 postgraduate theses since 2018. Relevant projects have been developed using the computing capabilities of Cluster-UY, including: the development of forecasting tools for renewable energy management in Uruguay (Energy);¹⁰ socioeconomic analysis of short and long-term prices and the impact on welfare of low-income citizens (Economy/Social Sciences); free and publicly available database of biomolecular simulations of SARS-CoV2 proteins (Bioinformatics, Institut Pasteur Montevideo & Cluster-UY);⁸ and analysis of dumping of La Teja oil refinery in Montevideo Bay (Environment/Sea Hydraulics/Water quality), among others. These projects have been significant contributions to the aforementioned research lines in Uruguay.

The aforementioned descriptions demonstrate different approaches have been applied for developing and operating HPC facilities in Brazil, Mexico, and Uruguay.

In Brazil, its territory size and federative political model led to different initiatives for fostering HPC in the country. Among them, the most prominent was SINAPAD. Initially devised as a network of HPC centers with similar capacity, the SINAPAD gradually changed to a tiered model with the Santos Dumont supercomputer being its Tier-0.

Mexico has fostered different HPC public infrastructures according to specific regional research and technological needs. As a consequence, at least 10 HPC centers are housed in state universities, federal research centers, and National Council for Science and Technology (CONACYT) centers, among them four with computational capacities similar to ABACUS. These centers are all members of the Mexican Network in HPC, which offers coordinated HPC resources and training to diverse communities in the country.

Finally, in Uruguay a collaborative model has been applied. The benefits of this model are twofold: on the one hand, it allows implementing an egalitarian model for access to the provided services and incorporating users (scientists, partners, companies) as real owners of the Center; on the other hand, it provides a way to get the much-needed funds for operation, maintenance, and growth in small countries, where funds for research are scarce. This model can be replicated in other small countries in Latin America.

Back to Top

Networking in Support of Supercomputing Activities

Networking is also a crucial component of collaborative efforts to build an ecosystem for science in Latin America, in particular for sharing HPC infrastructures among its countries. RedCLARA is a Latin American organization created and led by National Research and Education Networks (NRENs), with the main goal of strengthening the development of science, education, culture and innovation in the region through the innovative use of advanced networks (see Figure 4). Created in 2003, RedCLARA led, operated, maintained and developed several infrastructure projects to support research and education (ALICE, ALICE 2, the Bella Program), which allowed the creation of an advanced high-bandwidth network, interconnecting Latin American NRENs and with NRENs worldwide. RedCLARA also supported other projects (ComGridLatAm, RISC, RICAP) in order to create a continental ecosystem in accordance with the Advanced Computing System for Latin America and Caribe (SCALAC), a non-profit civil association that, as of June 2020, brings together nine countries in the region.

Figure 4. RedCLARA connections (www.redclara.net).

Regarding connectivity, the current situation of Latin America is that the main ring of the RedCLARA network (in orange in Figure 4), connecting Brazil, Chile, and Panama with the U.S., has a bandwidth of 100Gbps, and links from 500Mbps to 20Gbps connect the other Latin American countries with the main ring. In turn, the connection with Europe (via France and U.K.) is over 10Gbps, similar to the connection to the U.S. from Mexico.

Back to Top

International Research Collaborations

Latin America has a long tradition of international collaborations in areas related to e-Infrastructures for science, including grid computing, cloud computing, and HPC. Because of the cultural ties between Latin America and Europe, a large part of these collaborations involved countries in these two regions. Noteworthy is the sequence of projects EELA, EELA-2, and GISELA, partially funded by the European Commission (EC), whose goals were to deploy and consolidate grid computing infrastructures present in almost all Latin American countries, and connect them to similar European initiatives. Recently, a different initiative from the RICAP project aimed at integrating Latin American e-Infrastructures (both HPC and federated clouds) in a sustainable network. The RICAP project coordinates its activities with SCALAC, agreeing to deploy HPC calls for accessing computing resources via RedCLARA as one of their thematic services. Such initiative has been exercised during the COVID-19 pandemic, when SINAPAD and SCALAC together have made available to researchers prioritized access to HPC and AI resources for pandemic-related projects.

At the national level, Brazil has participated in international collaborations on e-Infrastructure with the BRICS countries, the U.S., and Europe. With the latter, in particular, Brazil co-funded e-Infrastructure projects carried out by large consortia involving research centers and private companies, with MCTI funding the Brazilian partners, while the EC funded the European ones. In the area of cloud computing for science, the EUBrazil Cloud Connect project involved 12 partners to produce technology for the federation of heterogeneous cloud infrastructures in Brazil and Europe, followed by others in this area (EUBra-BIGSEA, SecureCloud, and ATMOSPHERE). In the area of HPC for science, the HPC4e project brought together 13 partners in Brazil and Europe, aiming at going beyond the state-of-the-art in the required HPC simulations for wind energy production and design, efficient combustion systems for biomass-derived fuels, and exploration geophysics for hydrocarbon reservoirs.

Mexico has been involved with advanced computing international collaborations through several projects, for example, at the Large Hadron Collider (CERN), at the HAWC Gamma Ray Observatory (Mexico)¹ and at the Pierre Auger Observatory (Argentina). Recently, the ENERXICO project builds upon the expertise of a consortium of 15 institutions distributed between Mexico and Europe to deliver groundbreaking new energy solutions, from wind turbine simulations to improve the efficiency of wind farms and make wind energy more competitive, to geophysics exploration and oil reservoir modeling, to thermo- and fluid-dynamic processes of biofuel combustion for transportation. Mexican partners are developing exascale-ready application codes, among them, the first Smoothed Particle Hydrodynamics based code that has ever been elaborated for the numerical simulation of oil reservoirs. Furthermore, to reduce the uncertainties, novel applications of AI are being implemented to mimic the acumen and experience of the experts who need to make an inference based on field characteristics.

Several international collaborations including Uruguayan partners have been developed at the National Supercomputing Center. Some relevant recent projects are: “Geophysics and cosmic rays detection using Hubble Space Telescope (HST)” (National Space Telescope, U.S.), to exploit dark images from HST for cosmic ray detector to analyze Earth’s external magnetic field, applying HPC and cloud computing, to complement measurements from 93 geophysical observatories; Energy efficiency projects with LA/EU partners, ranging from machine learning for data analysis to design of new technologies for renewable energy sources; “Urban transport planning in smart cities” (LA/EU, Mexico/Cuba/Spain/France), studying and implementing methodologies for mobility planning in modern cities, applying computational intelligence and reliable/secure processing of large volumes of data, to assist users and authorities in mobility decision-making (data collection, mobility patterns identification, route/frequencies planning, sustainable mobility, traffic lights synchronization); and the already commented SYRAH-COVID initiative for building a free database of biomolecular simulations of SARS-CoV2 proteins.

Back to Top

The Way Forward

The current situation of HPC in Latin America is very promising for developing new initiatives and strengthening existing ones for further growth of science and education.

Regarding infrastructure, the region counts with several platforms and centers that not only provide computing power, but also new-generation services for developing complex research projects. Many of the existing platforms support some degree of interconnection with similar ones in the continent, or even with larger platforms in Europe and the U.S. These systems are focused on high-impact research efforts, especially those related with the current social and economic situation of the countries in this region. It is also expected that by the end of the year 2021, connectivity will be significantly enhanced by the installation of a new submarine cable connecting Europe directly to Latin America at 2Tb/s. This will foster collaboration and allow the development of new capabilities and the access of NRENs to additional HPC resources, using services provided by RedCLARA.

These are exciting times for HPC development, with exascale computational capabilities at the turn of the corner. There is a convergence under way; bringing together HPC and AI, along with data analytics (DA), in what may become the profile of a single, integrated system. What is manifest in recent years is that it is becoming difficult to reach specific research and technological goals without the interplay among these three technologies. Aligned with this trend, LNCC and members of the Mexican Network in HPC including CINVESTAV have recently expanded their infrastructures to provide better support for AI-oriented research.

All the advancement in these new integrated systems will trickle down to society through smaller systems that will incorporate landmark innovations developed in the way to the exascale era and allow researchers to collectively benefit from these new technologies. Latin American public and private centers for research and technology will certainly be using these new systems to solve highly demanding HPC numerical simulations of complex problems, running innovative AI applications and confronting intricate DA challenges. Examples include those involving medical imaging, genomic analysis, astrophysics, climate models, smart cities, and digital agriculture, to mention a few. Moreover, situations like the COVID-19 pandemic calls Latin America for agile collaboration arrangements—such as the one proposed by SCALAC—that cross institutional and governmental boundaries.

Back to Top

Acknowledgments

This work would not be possible without the help of Artur Ziviani, Augusto Gadelha, Francisco Brasileiro, Pablo Blanco, Jose Maria Cela, Jaime Klapp, Leonardo Sigalotti, Salma Jalife, Rafael Mayo, and the boards of RedCLARA and SCALAC. Map in Figure 1 adapted from original work by Felipe Menegaz under license I CC-BY-SA-3.0 (http://creativecommons.org/licenses/by-sa/3.0/).