A Global Collaboration to Deploy Help to China

On Monday May 12, 2008, an earthquake measuring 7.9 on the Richter scale struck in Sichuan Province in southwestern China, destroying homes, schools, hospitals, roads, and vital power and communication infrastructure. More than 45 million people were affected—tens of thousands were killed, hundreds of thousands injured, millions of people were evacuated and left homeless, and millions of buildings were destroyed.^a

When the earthquake hit, several members of what became an international, volunteer, disaster-management IT team were attending a workshop in Washington, D.C. The workshop was organized by the IBM Office of Corporate Citizenship and Corporate Affairs department to train IBM personnel and others in the use and deployment of Sahana, a free and open source software (FOSS) disaster management system.

Sahana, which means relief in Sinhalese, is a Web-based collaboration tool that helps manage information resources during a disaster recovery effort. It supports a wide range of relief efforts from finding missing persons, to managing volunteers, tracking resources, and coordinating refugee camps. Sahana enables government groups, non-governmental organizations (NGOs) and the victims themselves to work together during a disaster recovery effort.

Over the next several weeks, the team members, distributed among several cities (Beijing and Chengdu in China, Hartford and New York in the U.S., and Colombo in Sri Lanka), worked together over global communication channels to configure and deploy Sahana in Chengdu, in order to support the disaster recovery effort there.

The organizations involved in the collaboration included:

• The Lanka Software Foundation (LSF), developers of the Sahana system. Three LSF members, led by the second author, were conducting the training workshop.
• Various departments of IBM, including Business Continuity and Resiliency Services. Ten employees led by the fourth author, who organized the workshop, were receiving instruction in how to deploy and use Sahana.
• The Humanitarian FOSS Project (H-FOSS), an NSF-funded effort aimed at revitalizing undergraduate computing education. Four students and their mentors, the first and third authors, were attending the workshop as developers and undergraduate members of the Sahana community.
• IBM China. Initially, local teams in Beijing and Chengdu consisting of corporate citizenship, government relations, and technical professionals led in demonstrating Sahana to local officials, securing buy-in, and establishing channels to proceed. Then a large team of developers, language specialists, and others, including a team based in Chengdu, Sichuan, eventually took charge of the deployment effort in Chengdu. The fifth author was a member of the China development team.

Almost immediately after the earthquake, discussions were held between IBM, IBM China, China’s Ministry of Civil Affairs, and the Chengdu city government in Sichuan province. Once the Chengdu government expressed real interest in deploying Sahana, a team was formed to begin the process of localizing Sahana—that is, translating its user interface into simplified Chinese. The team was led by executives and software developers from IBM-China and assisted by Sahana team members in Colombo and student H-FOSS volunteers in Hartford.

The team’s organizational structure followed the normal procedure involved in previous Sahana deployments—a local group in close proximity to the incident supported by volunteers from the global Sahana community. In this case IBM-China, including some who were directly affected by the disaster, took the lead in deploying Sahana over an intensive three-week period.

The decision by the Chengdu government to proceed with the deployment was taken on May 21, 2008 and a revised and localized version of Sahana was deployed in Chengdu on May 25. On June 12 we learned that 42 families had been reunited with the help of Sahana.^b

This article provides an inside look at the deployment effort. It describes how a diverse, multidisciplinary team—professional programmers, software engineers, executives from a large global enterprise, students, faculty, and humanitarian IT specialists from a global FOSS community—worked together to assist the earthquake recovery effort. The success of the collaboration illustrates the power of virtual communities working across international boundaries using a variety of electronic communication software. It also demonstrates that the Internet has truly made us all neighbors and is constantly forcing us to redefine our concept of community.

Back to Top

The Sahana System

The Sahana project (www.sahana.lk) was started by a group of Sri Lankan programmers and IT specialists in the immediate aftermath of the December 2004 Asian Tsunami.^13,15 Sahana has been successfully deployed in a number of disasters, including the 2005 Pakistan earthquake, the 2006 Philippine mud slide, and the 2007 Peru earthquake. More recently, it was translated into Burmese and deployed in Myanmar disaster in May 2008 by the InSTEDD organization.⁶ In 2010 Sahana was deployed in the aftermath of the Haiti earthquake and the Pakistan floods. It has also been pre-deployed for local government by the New York City Office of Emergency Preparedness. As these examples show, the Sahana system can be rapidly customized for a diverse range of disaster situations. The Sahana project has been recognized for its work with numerous awards, including the Sourceforge Project of the Month award for June 2006 and the 2006 Award for Social Benefit from the Free Software Foundation.^14,17

H-FOSS,¹ as noted earlier, is free and open source software that promotes human welfare, including, in this case, the support of humanitarian response and disaster recovery. Within the humanitarian response context, FOSS offers the following advantages:

• No Discrimination on Access: Once available under a FOSS license, the software effectively becomes a global public good, available for anyone around the world to use freely.
• Transparency and Trustworthiness: The transparency of the FOSS development process makes the software more trustworthy and resistant to any particular political agenda. Such software can serve as a transparent mediator, particularly during conflict situations.
• Low Cost and Local Capacity: FOSS helps reduce the digital divide as there is no additional cost for the product itself. FOSS communities encourage local developers to undertake the ongoing maintenance of the software.
• Shared Inter-Organizational Development: NGOs and humanitarian relief organizations are free to collaborate in a mutually beneficial way on the development of FOSS tools in the humanitarian domain.
• Adaptability: The software can be freely adapted to suit the particular problem and the surrounding culture.

Back to Top

The Humanitarian FOSS Education Project

The Humanitarian FOSS Project (H-FOSS Project) is an NSF-CPATH-funded project aimed at helping revitalize undergraduate computing education by getting students engaged in building FOSS that serves the community in some way.^3,4,7,8,c

The H-FOSS Project (www.hfoss.org) combines two seemingly unrelated ideas first proposed by former ACM President David Patterson—helping our neighbors and getting involved in the FOSS movement. In his November 2005 President’s Letter published in Communications, “Rescuing Our Families, Our Neighbors, and Ourselves,” Patterson urged computer scientists to become more involved in the effort to help lessen the impact of disasters such as Katrina, stating that “Perhaps it is our civic duty to do so.”¹⁰ In a subsequent column, Patterson urged educators to “join the open source movement,” noting that despite its acceptance in the software industry, FOSS has received little attention in computing curricula.¹¹ He suggested that getting students “contributing immediately to the real world” might help distinguish CS from other academic disciplines.

Through courses, independent studies, and student internships, the H-FOSS Project has engaged undergraduates from several colleges (Trinity College, Connecticut College, Wesleyan University, Bowdoin College, and University of Hartford) in a number of community-based FOSS projects.⁸

The relationship between Sahana and the H-FOSS Project goes back to January 2006. The H-FOSS students and faculty at the May 2008 Washington, D.C. workshop had previously developed Sahana’s Volunteer Management (VM) module, which was one of the modules under consideration for the China deployment. The VM module is designed to register and manage volunteers during a disaster recovery effort.

The deployment effort. Throughout the five days of the Washington, D.C. workshop, discussions among IBM executives at headquarters, IBM China leaders, and Sahana leaders explored the scope of a possible Sahana deployment. Although Sahana runs on a wide variety of platforms, there are a number of logistical and security issues that must be solved in a successful deployment. How and where will it be hosted? How many instances will be deployed? Who will be responsible for installing and configuring the system for the disaster locale? What type of network infrastructure will be used? What government or nongovernmental organization will take responsibility for its administration? What data will be collected? Who will have access and how will data be secured?

IBM’s role in addressing these questions was crucial. IBM has been the main industry supporter of the Sahana project, helping not only with early requirements gathering and crisis response expertise after the 2004 Asian tsunami, but also helping deploy it during disaster situations in Pakistan and Peru. IBM has introduced Sahana to many partners and has provided funding and technology to Lanka Software Foundation to support Sahana’s ongoing development. Moreover,

IBM was in an excellent position to help facilitate the Sahana deployment in China. It has successful business operations in China, including a large office in Chengdu, and has long-term strategic philanthropic and community relationships in China focusing on education.

The fact that the D.C. workshop brought together key members of the Sahana team and IBM’s crisis management group helped facilitate the decision-making process. By Friday, May 16, 2008, the last day of the workshop, work began on the customization of Sahana in preparation for a possible invitation from the Chengdu government to deploy it. IBM arranged airfare and a visa from the Chinese embassy, and a member of the Sahana team left for China to advise the IBM-China team and serve as a liaison with the Sahana core team. Soon thereafter a member of the IBM Crisis Response Team also made the trip to Chengdu.

Sahana internationalization and localization. The main customization required was to develop a Chinese language user interface. Internationalization is the process of designing software to support multiple languages. Localization is the process customizing an internationalized program for a given language and locale.

Sahana’s internationalization architecture is based on the GNU gettext API.⁵ Strings in the source code can be marked for translation by embedding them as arguments to getText() function calls. The marked-up code can then be run through a utility program that will generate a set of PO (Portable Object) files containing all the strings that need translation. These human-readable files can then be edited by a bilingual translator, inserting the appropriate Chinese characters for each English phrase. The PO files are then compiled into the machine readable MO (Machine Object) files, which are accessed at runtime to generate a localized (Chinese) user interface.

---

Sahana, which means relief in Sinhalese, is a Web-based collaboration tool that helps manage information resources during a disaster recovery effort. It supports a wide range of relief efforts from finding missing persons, to managing volunteers, tracking resources, and coordinating refugee camps.

---

Back to Top

Internationalizing the VM Module

To convert Sahana’s interface to Chinese, translation (PO) files had to be created for the three modules that were initially identified for deployment—the Missing Person Registry (MPR), Disaster Victim Registry (DVR) and the Volunteer Management (VM) module. However, only the MPR and DVR had previously been internationalized. Therefore, all 6,400 lines of the VM module, which had been developed by the H-FOSS students, had to be visually inspected and tagged for translation.

Beginning on Saturday, May 17—and foregoing a planned visit to the Air and Space Museum—the H-FOSS students began editing the VM code base in a D.C. hotel room. When they left for Hartford at 3:00 P.M., they were still only about 75% finished. They continued working on the code base during the drive to Hartford, stopping at a McDonald’s restaurant on Route I-95 in Stanford, CT to transmit the revised VM source code—now nearly 100% internationalized—to the Sahana team in Colombo.^d

Back to Top

A Virtual Development Team

Thus began a week of multiway Skype calls between Colombo, Beijing, Chengdu, and Hartford to coordinate the internationalization and translation efforts. In Sri Lanka, a group of developers from the Lanka Software Foundation had given up their vacation period to make themselves available “24×7” to help respond to the disaster. The Sri Lanka team served as liaison over Skype with both the China and U.S. teams to coordinate work on the internationalization and localization effort. They also took initial responsibility—along with the H-FOSS students—for patching and modifying the Sahana source code based on bug reports and feature requests received by the China team.

For the first few days a typical work cycle would go as follows: Source code updates on the VM module would be sent daily from Hartford to Colombo. Language translation files for the MPR, DVR and VM modules would be sent daily from China to Colombo. The Colombo team then generated the PO files for the VM module and sent the PO files for all three modules to the China team for further translation. The China team would test the translations on the installed Sahana user interface and report bugs and missing elements back to the teams in Hartford and Sri Lanka. And the cycle would repeat.

To help coordinate the localization process a demo site was set up on an IBM server in China and the developers in Colombo and Hartford were given access to it through a Virtual Private Network (VPN). Unfortunately, the developers in Colombo and Hartford did not have access to a common code repository, which led to some confusion and inefficiency. Often bugs that had been fixed one day would still show up in the next day’s bug reports. However, amazingly, the various teams managed to work together under this initial arrangement and each day the MPR, DVR, and VM modules came closer to being 100% localized.

The conference calls took place in English. Eventually, since different teams of developers were working on different modules, separate two-way conference calls were initiated to make the process more efficient. The Hartford (HFOSS Project) team was soon joined (virtually again) by a Chinese-speaking liaison, a native of Sichuan province now employed by IBM in New York City.

---

The Sahana system was deployed on the Chengdu government Web site. In addition to helping the government manage its recovery effort, it provided a public search function that helped disaster victims find their missing family members.

---

After a few days, the China team assumed direct responsibility for the deployment, doing both the translation and building the code on site. A medical doctor was recruited to help translate medical terms. Since there were many parts of the code that had not been internationalized, the team had to test all functions of the required modules and identify all the strings that were not translated. When the localization process was completed, development work began on customizations, bug fixes, and other changes to the code base. At this point, the Sahana team in Colombo moved on to a support role, as the on-site team in Chengdu quickly became proficient and self-sufficient in making changes to the code. The Hartford team went into “standby” mode to await whatever additional support might be requested by the on-site team.

Back to Top

The On-Site Development Effort

Sahana’s PHP-based plug-in architecture makes it relatively easy to adapt and customize. For the China deployment all unnecessary modules were disabled and removed. Moreover, Sahana’s open source license eliminated the need for the Chengdu government to acquire permission from copyright holders in order to make the needed modifications. These features contributed significantly and crucially toward Sahana’s rapid and successful deployment.

To help decide which modules would be used in the deployment the team in China conducted several presentations and demos of Sahana’s capabilities with the Chengdu city government and the Ministry of Civil Affairs. As the discussions proceeded, the officials were able to see how certain modules addressed critical needs for information gathering to support the local efforts and eventually decided to deploy the Disaster Victim Registry (DVR), the Camp Registry (CR), and the administration and data synchronization modules. The China team also created and deployed a new search module, Disaster Victim Search. The Chengdu police used these modules for victim information management, and opened the search function on a public Web site for public access.

The China team created many customizations to DVR and CR to fit the requirements of the Chengdu police, including a utility to Create, Read, Up-date, Delete (CRUD) victims based on identified needs. New features were also added to the synchronization module, including incremental import, photo support, export logging, and batch export/import at the local site. In addition to the development effort, the team’s testing efforts were also significant, focusing on both new functions and existing systems.

The China team also improved the system’s scalability and security. The initial estimate of victims from the police was 300,000, but they indicated that the number could increase to 3,000,000 or more. To address these projected needs, the Sahana team came up with three possible configurations. The China team set up a remote, simulated testing environment and within three days identified a solution that offered high availability, high performance caching, load balancing, and scalability. Once a configuration was determined and documented from the testing environment, the Chengdu on-site team configured the system immediately during the night.

To ensure the security of the system, the Chengdu team deployed a high-performance firewall to isolate the Sahana system from the complex network environment. A Web monitor was installed to monitor the real-time network package flow. To optimize the system’s performance, the IBM engineers performed much analysis on-site in Chengdu and made some Linux customizations, including closing unnecessary services and performing access control for system files.

Back to Top

Outcomes, Observations, and Lessons

The Sahana system was deployed on the Chengdu government Web site. In addition to helping the government manage its recovery effort, it provided a public search function that helped disaster victims find their missing family members. As reported in the Chinese media, just after the site came online, it successfully helped a victim find his family.

After deployment, the China team cooperated with SINA (SINA.com.cn), the largest portal site in China, to merge with their victim database. Given that most telecommunication facilities were destroyed during the earthquake, the Sahana site was very important during the first few weeks of the disaster. It provided valuable and accurate information to people who eagerly wanted to know the status of their relatives. There were many positive reports in the local newspaper and television media of how Sahana helped victims.

The successful deployment of Sahana in Chengdu had a significant impact on the recovery effort. However, during the rush and chaos that attends an emergency effort such as the Sahana deployment in China, there are bound to be problems, and glitches and inefficiencies. Here, we look back at some of these, now made clearer with 20:20 hindsight.

• Community-based localization—The Pootle server. The majority of the localization work was in translating the interface into Chinese, which required effective collaboration among multiple translators. The translation process was hampered somewhat by the lack of a shared repository that would have enabled translators distributed over different time zones and locations to collaborate more efficiently. The Sahana community has now adopted Pootle (translate.sourceforge.net), an open source collaboration and translation tool to help facilitate this process in the future.
• Rapid deployment vs. best practices. The best way to deploy Sahana is before a disaster takes place and in preparation for it. However, one rarely knows the shape and extent of a disaster beforehand, and thus it is often necessary to perform a rapid customization of the necessary disaster relief tools. The Sahana system is designed to enable this rapid customization. More and more of these customizations are being supported by internal administrative interfaces. However because the code employs a simple PHP design, it is not difficult for an entry-level developer to modify code to suit a purpose. This is evidenced not only by the fact that Sahana has now been deployed successfully in several disaster relief efforts, but also by the fact that undergraduate developers, initially in Sri Lanka but now in the U.S. and other areas, have been able to make significant contributions to the Sahana system as members of its global community.

Still, this rapid customization needs to be coupled with a very strong quality assurance effort to ensure that Sahana can meet all the demands placed on it during deployment. Sahana could benefit from a test harness to automate a good amount of this testing.

• Is it a bug or a feature? Understandably, the IBM-China team was under extreme pressure to get Sahana up and running on-site and with functionality tailored to meet their actual needs. One of the challenges they faced was importing location data into the Sahana database. Sahana uses a hierarchical design for its locations—for example, country, state, city, and so on—whereas the data sets available from the Chinese local government were flat (non-hierarchical). While it is relatively straightforward to configure Sahana to handle a flat data set, it simply won’t work correctly with a flat data set in its default configuration.

The bug report to the H-FOSS Project team concerning this issue presented a classic case of the bugvs.- feature question. The cause of the problem was easily traced to the incompatibility between the hierarchical configuration and the flat data set. However, the China team was understandably reluctant to reconfigure a well-functioning implementation. Instead, they developed a workaround by modifying one of the Sahana library files. Such a modification could pose risks to other parts of the system and would pose challenges when incorporating enhancements and bug fixes back onto the Sahana code base.

Practically speaking, the change to the library file did not have adverse side effects on the other modules. Although it would have been possible to reconfigure and reinstall the system to handle the flat data set, the demands of the emergency and the need for rapid deployment won out over concerns about best practices and similar abstract principles. However, for future deployments, Sahana’s configuration guidelines should be improved to try to forestall this kind of problem.

• Contributing back to Sahana. An ongoing issue in this and other deployments of Sahana is how best to incorporate changes back into the main Sahana repository. Of course, not all changes to the code should necessarily be incorporated into the permanent code base. But how does an open source community, such as Sahana, best incorporate genuine improvements and extensions?

The answer to this question is important to the sustainability and vitality of H-FOSS in general and Sahana in particular. Successful FOSS development depends on the willingness of those who use the software to give back to the project. This is certainly a factor in the ongoing viability of highly successful FOSS projects, such as Linux and Mozilla.

Within the disaster relief domain the answer to this question is complicated by the realities of the situation. For example, in the Chengdu deployment, Sahana was deployed on local government servers with the assistance of IBM. The system contains sensitive data that must be protected from unauthorized users and developers. This means that responsibility for submitting patches and code to the Sahana repository probably rests with the development team in China—a team that likely has other responsibilities and priorities.

When employees of a corporation contribute code to an open source project, they must typically follow guidelines to ensure that no proprietary code becomes entangled with the custom code. IBM China employees followed such guidelines, and IBM also explicitly approved the donation of IBM-created code to Sahana and to the Chinese government. Here again, the task of packaging and submitting changes and updates to the Sahana repository rests with those who used it, who may or may not see the need for giving back to the Sahana community as a main priority at this time.

• Student engagement in Humanitarian FOSS. The H-FOSS project is one of several efforts in a growing trend in academia toward incorporating socially beneficial computing into undergraduate computing education. Other projects include the Socially Relevant Computing project at SUNY-Buffalo,¹⁶ USC’s Project:Possibility, which focuses on developing FOSS to help the disabled,¹² and the Computing for Good project at Georgia Tech, which focuses on FOSS that improves the human condition. These projects share the belief that focusing on socially beneficial computing projects is good for undergraduate computing education.

Such projects introduce an additional element of risk into both the educational process and the community service. When the H-FOSS Project began the collaboration with Sahana, there was considerable uncertainty on both sides. On the academic side, it was unclear whether undergraduates could design and write production quality code. Would the code work? How would it be maintained? Would we be able to respond to requests for bug fixes and enhancements, possibly under crisis situations? On the Sahana side, it was unclear to what extent the academics would remain committed to the project. Students come and go, and the academic calendar contains many breaks during which it may be difficult to sustain work.

The Chengdu deployment experience went a long way toward answering these questions and toward supporting the view that the academic community can contribute to and learn from “joining the open source movement.”

Back to Top

Conclusion

As this account illustrates, the collaboration involving Sahana, IBM, IBM-China, and the H-FOSS Project led to the deployment of free and open source software that played a positive role toward helping the people of Chengdu recover from the earthquake. It is clear that in this instance at least all of the partners played a crucial role in the deployment. Without the Sahana project there would be no suitable disaster management software. Without IBM’s involvement—both in the U.S. and in China—the deployment would have been impossible, for neither the Sahana project nor the Chinese government had the resources to manage the deployment. Without the H-FOSS Project, the deployment would have been lacking the option to deploy the VM module.

The collaboration was mutually beneficial to all participants. The Sahana community was able to effect another successful deployment of its humanitarian software, learning valuable lessons in the process and extending its growing reputation. The H-FOSS Project was able to provide a unique and satisfying learning experience to its students, who not only learned about the rewards of contributing to the public good, but also learned how real-world software development and deployment works. For IBM, in addition to spreading good will about IBM among the Chinese people and inspiring pride among employees worldwide for contributing substantially to alleviate a crisis, they may also have been able to identify future business opportunities and contacts. Most importantly, of course, the IBM-China team was able to apply their knowledge and skills toward helping their community recover from this devastating disaster.

In addition to these immediate and practical benefits, there are also educational benefits worth noting:

• For FOSS communities like Sahana, this collaboration suggests new ways to grow—namely, by getting more students involved in its efforts.
• This type of engagement provides students with technologies, and learning opportunities, that can’t be duplicated or simulated in the classroom.
• For the computing and IT industry, having well-educated students coming out of college with such realworld experience can only benefit their enterprises.
• For the broader social good, there is a growing realization that the FOSS model can be applied to benefit humanitarian and non-profit organizations, which have tremendous IT needs and limited resources.

Finally, the story of this experience speaks to some of the widespread misconceptions about what computer professionals do. We have here a rare look at computer scientists and software engineers making a difference in the world through their expertise. This story adds to the growing body of literature in community informatics, the study and practice of enabling communities through information and communications technology.² The team effort required for this successful deployment completely debunks the stereotype of the solitary programmer. The testimony of those involved—particularly the programmers and translators in China (see sidebar)—reveals the deep satisfaction that can result from working on a team to solve an important problem and help one’s community. Thus, perhaps the greatest lesson to be drawn from this experience is the one suggested by David Patterson—that by helping our neighbors we are also helping ourselves.

Back to Top

Back to Top

Back to Top

Back to Top

Back to Top

Back to Top