Hands-on course work plays an important role in information security education. Through such course work, students can apply, deepen, and extend the conceptual knowledge taught in more theory-oriented lectures. Moreover, they can acquire experience with, and an understanding of, realistic systems, which is particularly important in Information Security. Hands-on experience shows how often a system's security depends on fine, seemingly innocuous details. Such details can be the source of serious security weaknesses and it is difficult to grasp them only in theory.
The importance of laboratory-based courses in general is also emphasized in curriculum development.1,6 The advantages are manifold: labs foster teamwork, enhance the ability to work with concrete systems, and train students to develop practical solutions for realistic problems while accounting for constraints like usability, cost, performance, and security.
How can a laboratory-based course in Information Security be designed? We will present two approaches, one based on conflict, the other on review. The Conflict-Based Approach is popular and well-documented and here we just briefly recall its main features. The Review-Based Approach is less common and only partially documented. We have designed and held a review-based course that we present in detail as an exemplary realization of this approach. Finally, we compare and contrast both approaches, as well as the skills they convey.
Over the past decade, various institutions have developed and offered so-called hacker labs. These labs emphasize real-time attack and defense activities.2,3,5 We call this the Conflict-Based Approach. It is typically structured into four phases:
The most important characteristic of this approach is the conflict, which is carried out in real-time. One of the first labs of this type was, not surprisingly, designed by a US Military Academy.5,7 Warfare is used as an analogy and one speaks of weapons, battles, and battlefields. Tactics are based on the insights of military strategists like Sun Tzu and Julius Caesar.
The real-time nature of such information warfare puts the student teams under pressure. They must be able to quickly identify the cause of problems and security breaches and devise and implement effective countermeasures. This requires intense work and the students acquire considerable knowledge and understanding of technical interrelationships in a short amount of time. They also learn to organize themselves within their teams.
Based on the understanding of the Conflict-Based Approach described above, we have designed the Applied Security Laboratory course at the Computer Science Department of ETH Zurich. The course is positioned as an optional part of the major program in Information Security. Basic knowledge of Information Security, networking, and operating systemsacquired either from previous courses or self studyis a prerequisite to taking the course. We have structured the course similar to the Conflict-Based Approach, but with a different emphasis:
A security laboratory requires a technical infrastructure which the participants use for their experiments in the different course phases. The requirements on such an infrastructure reflect both the needs of the students carrying out experiments and the needs of the instructor for building experiments and administrating the infrastructure. We summarize here the central requirements that guided the design of our infrastructure.
These requirements can be met in different ways. We employ a straightforward architecture: A dedicated network within a lab room connects ten workstations and an infrastructure server. The infrastructure server in turn provides central services like user authentication, storage, and Internet access. A firewall filters traffic to and from external networks.
A central ingredient in ensuring security and stability is virtualization. In our case, we equip each workstation with a VMware installation, which virtualizes an x86 architecture. This virtualization makes it possible to run several operating systems, each in its own virtual machine but on the same workstation. The virtual machines can communicate with each other using virtual network interfaces and devices. As we will see shortly, virtualization also facilitates review by allowing the teams to exchange images containing their, possibly networked, systems.
We provide preconfigured virtual machines for the different experiments. The students download these machines to their laboratory workstations and can then work with the resulting self-contained setup consisting of one or more networked machines.
Four course phases. Knowledge acquisition. The first phase is centered on self study using course material that we have designed for this purpose. The focus of our current material is on operating system and web application security. It combines background information (including references to additional reading material), hands-on experiments, and short exercises and questions that encourage the students to reflect on what they observed during their experiments. Figure 1 and 2 show two excerpts from this material.
We also hold a series of short lectures that complement the course material. The lectures address topics such as the role of standards in Information Security, how to conduct a risk analysis, or ethical and legal issues. However, the students spend most of their time at the laboratory workstations, working through the experiments and associated questions. The students work in groups of two, mainly due to the limited number of workstations. But the teamwork also gives them the opportunity to discuss the material and learn from each other. Instructors are available at scheduled times to provide additional support.
Design and implementation. We initiate the design and implementation phase by giving the students a description of a project that is realistic but manageable within the course's time frame. The description specifies the project's goal as well as the functional and the security-related requirements (Figure 3). The students now work in teams of four. This allows us to scale up the project and allows the students to practice teamwork. The implementation part encourages the students to deepen and extend the knowledge and skills they acquired before.
Review. The review phase follows. Thanks to the use of virtual machines, the students can easily swap the systems they have implemented, including all the necessary access credentials. Each team reviews another team's system, identifies as many security-relevant weaknesses as possible, and suggests appropriate fixes. This analysis is based on all the options available. This includes a hands-on examination of the running system to discover and even exploit weaknesses as well as carefully studying the design documentation (for example, of the system architecture and application logic) and the implementation itself (for example, performing a code review). The students are also required to compare the system they reviewed with their own solution and to highlight particularly novel or elegant aspects of the other team's system.
Wrap-up. Towards the end of the course, each team submits a written report that documents its own system design, including the risk analysis and the review results. All lab participants receive copies of all reports and are thus provided with comprehensive documentation of the project work. Each team also presents and discusses in class its most noteworthy or surprising conclusions, either from the design and implementation or from the review phase.
We are aware that group work can have disadvantages. Difficulties can arise, for example, when group members commit substantially different amounts of time and effort into the project or even have very different skills. Still, we consider group work to be an important part of the project and a valuable skill, and we assess the team effort: The report is graded and each team member receives the same grade.
Finally, a written examination is used to assess each student's individual learning. It covers the first three phases, knowledge acquisition, design and implementation, and review. The final grade combines the grades from the written exam (60%) and report (40%).
Results. We have held the Applied Security Laboratory three times to date. In this section, we summarize our impressions as well as those of the students based on their responses to a survey conducted at the end of the last course. Of the 28 participants, 24 provided feedback.
The overall impression was very positive. The students liked, in particular, the hands-on work in the knowledge acquisition phase and the realistic project work in the design and implementation and the review phase. One reason often articulated was that the project assignment was open in many respects. This encouraged lively discussions during the design phase. Furthermore, the large degree of freedom gave rise to a variety of different solutions, and the students gained additional insights by examining these design variants.
The technical infrastructure we provided was suitable for the laboratory activities. It worked well from the administrative perspective, and the students were able to work efficiently. However, some teams would have preferred to have direct Internet access from within the virtual machines, that would simplify installing software and system patches. With the current setup, emphasizing security over convenience, the students had to use secondary storage to transfer software to the virtual machines.
The most frequent criticism concerned the time and effort required to complete the lab. Many students felt that the effort needed exceeded the allotted 6 hours per week. This can be compensated in the future either by reducing the scope of the assignment or providing the students more credit (and thereby a larger time allotment) for the course.
The Conflict-Based and the Review-Based Approaches are similarly structured, as depicted in Figure 4. Moreover, they both motivate the students by providing a realistic assignment and a competitive challenge. In addition, both foster teamwork. But apart from these similarities, we can identify two fundamental differences:
There is only one role in the Review-Based Approach: the students act as reviewers and receive full access to all components of the target system. This is essential, because there are many potential weaknesses that are easier to spot with complete system access and the possibility to inspect the application logic. Examples include lax file access permissions, implementation flaws in privileged system commands, inadequate backup mechanisms, or race conditions within the application.
The two approaches emphasize different activities and skills. Both approaches have proved themselves, as the references and our experience at ETH Zurich demonstrate. An educational institution must choose between the approaches based on its educational goals. It is difficult to define exact criteria for this choice. Still, the following may serve as a rough guideline.
The Conflict-Based Approach stresses the selection and application of attack and defense strategies and techniques under time pressure; in other words, it emphasizes and develops the skills needed for reacting to an ongoing attack with real-time decision demands. This knowledge and the corresponding skills are particularly useful for IT professionals working in an operational environment (for example, a service provider), but it can also serve as an important awareness-building measure for future managers. The Review-Based Approach emphasizes the design of secure systems, the thorough study of design variants, and the non time-critical analysis of implementations based on their designs. It is therefore especially appropriate for future developers and system architects.
An interesting option is to use both approaches together in a complementary way. The overall structure would follow the four phases shown in Figure 4. During the analysis phase, the students would first carry out the conflict. Afterwards, they would swap the systems they tested during the conflict and conduct a more comprehensive examination during the review. This order is sensible because the conflicts are more realistic when the opposing teams do not know what offensive and defensive measures to expect.
A combination of the two approaches clearly requires that the necessary resourcestime and supportare available. Given this, the advantages of the two approaches can be combined. However, the combination is an option, not a necessity, and the decision taken should be determined by the educational goals for the course.
2. Hill, J. M. D., Carver Jr., A. C, Humphries, J. W., and Pooch, U. W. Using an isolated network laboratory to teach advanced networks and security. Proceedings of the 32nd SIGCSE technical symposium on computer science education, 2001, 3640.
3. Micco, M., and Rossman, H. Building a Cyberwar Lab. Lessons Learned Teaching Cybersecurity Principles to Undergraduates. Proceedings of the 33rd SIGCSE Technical Symposium on Computer Science Education 2002, 2327.
5. Schäfer, J., Ragsdale, D. J., Surdu, J. R., and Carver, C. A. The IWAR range: a laboratory for undergraduate information assurance education. Proceedings of the 6th annual CCSC northeastern conference on computing in small colleges, 2001, 223232.
Based in part on M. Näf and D. Basin, "Konflikt oder Reviewzwei Ansätze fü Labors in angewandter Informationssicherheit", which appeared in Informatik Spektrum, 5(28), Oct 2005, Springer. Permission for reuse has been kindly granted by the publisher.
©2008 ACM 0001-0782/08/1200 $5.00
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2008 ACM, Inc.