Explorations in Cyber-Physical Systems Education

In 2006, Helen Gill at the U.S. National Science Foundation coined the term “cyber-physical systems” (CPS) to capture an emerging discipline concerned with the integration of computation with physical processes. A nascent research community emerged, building on a momentum that cut across fields such as embedded systems, real-time systems, hybrid systems, control theory, sensor networks, and formal methods. Discussions began about developing curricula to train students who were interested in working in the broad area of CPS.

Back to Top

During the 2006–2007 academic year, a small group of educators from the University of California, Berkeley’s Electrical Engineering and Computer Sciences (EECS) department—including Edward Lee, Claire Tomlin, and myself—met to discuss the creation of an undergraduate CPS curriculum. Berkeley had already been a pioneer in CPS research and graduate education for several years, but no undergraduate courses focused on CPS. A major challenge to this effort was the breadth of topics needed to cover the area. A further obstacle was trying to balance theoretical content with practical, lab-based coursework. From these discussions, however, emerged the basic contours of an undergraduate course, which developed over the next decade into a broader “expedition” in CPS education.

Major expeditions to explore the unknown are generally best undertaken by a team; this has been true of expeditions to discover new worlds, and it is true of expeditions in research and teaching. I have been fortunate to join several academic colleagues, industry collaborators, and students on the CPS education expedition described in this article. It has comprised several smaller explorations, undertaken roughly in parallel over the last 14 years, along various “trails” in CPS education. This article is my attempt to report these explorations, the results, the lessons learned, and to extrapolate them as ideas for the future of CPS education. It is not meant to be a survey of approaches to CPS education; for that, I refer the reader to a recent article by Marwedel et al.²⁶ The trails are presented in rough, but not strict, chronological order. An early version of this article appeared as “Cyber-Physical Systems Education: Explorations and Dreams.”³⁰

Back to Top

Trail 1: EECS 149

During the Spring 2008 semester, Edward Lee and I co-taught the first offering of “Introduction to Embedded Systems,” the undergraduate course in CPS we created at Berkeley.^18,a I will refer to this course by its number in the Berkeley course catalog: EECS 149. We limited enrollment and advertised the class as being for “advanced and adventurous” undergraduates. About 20 students showed up, and over the next 13 weeks, this small group explored a selection of topics in CPS, blending theoretical topics with experimental work. It was a rewarding experience in many ways—perhaps most satisfyingly, two of the students in that class, Jeff Jensen and TrungTran, went on to work closely with us in further developing the course’s lab and online content.

We faced a few key decisions in designing a new undergraduate CPS course. At the time, we made choices that just seemed natural to us, but 10 years on, I believe these decisions were crucial to the development of a durable and unique CPS curriculum at Berkeley, one which is also starting to impact other institutions around the world.

Diversity of topics and backgrounds. The field of CPS draws from several areas in computer science, electrical engineering, and other engineering disciplines, including computer architecture, embedded systems, programming languages, software engineering, real-time systems, operating systems and networking, formal methods, algorithms, computation theory, control theory, signal processing, robotics, sensors and actuators, and computer security. Similarly, over the past 14 years, we have had students from computer science, electrical and computer engineering, mechanical engineering, civil engineering, and even bioengineering. Integrating this bewildering diversity of subject areas into a coherent whole for students with such a wide breadth of backgrounds has been a challenge we had to overcome.

One approach would have been to not attempt such an integration. Instead, we could have opted for a collection of courses that together cover all the key areas in CPS. However, we felt that this approach would have two major shortcomings. First, the reader can observe that the collection of areas could essentially end up covering much of an undergraduate program in electrical engineering and computer science. Second, and more importantly, in CPS there is a pressing need for people who understand the intersection between the various areas.

We addressed these challenges by presenting a unified treatment of the different topics in CPS, emphasizing how they interact in the process of modeling, design, and analysis. There are two key elements to our approach. First, we teach CPS from a system-level perspective: designing a dependable, secure, and efficient CPS from a heterogeneous collection of components. For example, we teach the topic of sensors not from the perspective of someone who designs new sensor technology but from the perspective of a designer who must select and use the right sensors for a CPS design problem. System integration is known to be one of the biggest challenges facing the CPS industry, and we hope this approach can help train a new generation of engineers to address these challenges. Second, we emphasize mathematical models (such as finite-state machines, differential equations, and hybrid automata) in conjunction with their connections to underlying technology (hardware, software, networks, sensors, and actuators). A basic collection of mathematical models is taught in the first month and gives all students, regardless of their background, a common foundation for CPS. For example, we teach finite-state machines as a model for developing and verifying event-driven software for embedded systems, and differential equations as a model for sensors, actuators, and feedback control. This unified treatment has proved durable for more than a decade, even with advances in the field.

Balancing theory and practice. Many embedded-systems courses focus on the collection of technologies needed to get computers to interact properly with the physical world, including sensor calibration, interfacing with sensors and other input/output devices, programming in assembly or low-level languages, etc. Others focus more on applications, such as building a robotic system—for example, an autonomous vehicle or Internet-of-Things (IoT) application. Still others focus entirely on theoretical topics, such as models of computation for CPS, or formal modeling and verification. With EECS 149, we decided very early on to blend theoretical topics with practical, lab-based work. Further, we decided to build some flexibility into the lab work, splitting it into a six-week structured lab sequence followed by a capstone design project whose topic students could choose for themselves. From the very beginning, a programmable, somewhat-customizable, mobile robotic platform called the Cal Climber (see Figure 1) was chosen for the structured lab sequence, with the following assignment:

Figure 1. Cal Climber Laboratory Platform. The first prototype (left) was based on the iRobot Create platform, the second (right) was based on the iClebo Kobuki platform.

Design a controller to drive the Cal Climber. On level ground, your robot should drive straight. When an obstacle is encountered, such as a cliff or an object, your robot should navigate around the object and continue in its original orientation. On an incline, your robot should navigate uphill, while still avoiding obstacles. Use the accelerometer to detect an incline and as input to a control algorithm that maintains uphill orientation.

This simple assignment allowed us to interleave basic conceptual and theoretical topics with the lab sequence. For example, students learned about modeling and interfacing with sensors and actuators in class as they interfaced to an accelerometer in the lab. They learned about programming with interrupts in class as they worked with an interrupt-driven controller in the lab. And they learned about modeling with state machines, composition, and hierarchy in class as they programmed their controller with Statecharts in the lab. We have found this interleaved presentation helps students to appreciate theory in lab work and theoretical topics in class. Students continue to make these connections in their capstone projects, with very satisfactory learning outcomes.

Another aspect of balancing theory with practice is the course’s emphasis on formal methods and model-based design. Formal methods is a field of computer science and engineering concerned with the rigorous mathematical specification, design, and verification of computational systems.^3,37 We decided early on to make formal methods and model-based design a key component of the course. This is due, in part, to our research backgrounds involving extensive work on these topics. But we took this step with some trepidation—after all, there is a general impression that formal modeling and proof can be a bit “dry” and difficult for students. However, our overall experience has been very promising. Students realized the value of formal modeling, for instance, in reducing the number of design iterations needed to succeed in their capstone projects. This positive experience has extended beyond the community of Berkeley students to include students in the online course we offered a few years ago (see the section titled, “Trail 3: Online Education and Exercise Generation”). Similarly, the integration of formal methods with practical lab work forced us to prune down the subject to a core set of formal methods topics that we found to be most relevant to an introductory course in CPS. A more detailed discussion of the philosophy underlying the course, especially on the lab component and its integration with theoretical content, appears in Lee et al.¹⁰ and Jensen et al.²³

Back to Top

Trail 2: A CPS Textbook

By the fall of 2009, we had already offered EECS 149 twice. As noted earlier, the course was, at the time, unique for its coverage of a broad set of topics and its integration of theoretical and practical content. Edward Lee and I could not find a single book that covered all the content we wanted to teach. Therefore, we started developing our own course notes, which gradually grew into something more coherent. We decided to put them together as a textbook, and in 2010–2011, this effort culminated in the publication of the first edition of Introduction to Embedded Systems: A Cyber-Physical Systems Approach.²⁰

In earlier articles and in the book’s preface,^19,20,21,23 we have discussed at length the various design decisions made in writing the textbook and how it differs from the other excellent CPS textbooks available—for example, those by Alur¹ and Marwedel.²⁵ Therefore, I will focus on aspects that have not been covered in depth elsewhere.

Definition of CPS. A textbook on cyber-physical systems must define what that class of systems is. Cyber-physical systems have been informally described as integrations of computation with physical processes. Some definitions emphasize the networked aspect of these systems. Still others make distinctions between CPS and other terms, such as the IoT, embedded systems, Industrial Internet, and Industry 4.0.

We chose an inclusive approach, formulating the following definition: A cyber-physical system is an integration of computation with physical processes whose behavior is defined by both cyber and physical parts of the system.

This defines CPS as being about the intersection, not the union, of the physical and the cyber. It also emphasizes the behavior or semantics of CPS. It is not sufficient to separately understand the behavior of physical components and the computational components; we must instead understand their interaction. Our definition of CPS does not refer to networking or other specific characteristics. We believe the terms CPS, embedded systems, IoT, Industry 4.0, and so forth to be essentially equivalent, describing the same class of systems while emphasizing different characteristics of those systems.

Emphasis on models and software. Mirroring the EECS 149 course, we decided to focus the book on the interplay of software and hardware with the physical processes with which they interact. Most specifically, we chose not to focus on the design of CPS hardware components. These aspects are clearly important in any actual CPS design. However, with increasing commoditization of hardware, including processors, sensors, and actuators, we believe that the major intellectual challenge in CPS rests on how to effectively design algorithms, models, and software in a way that leverages a combination of hardware, sensors, actuators, and networking components to achieve a design objective. Moreover, industry trends (for instance, in automotive systems) clearly indicate an explosive growth in the software in CPS—in terms of opportunities and complexity. For these reasons, our textbook emphasizes a system-level perspective, combining rigorous mathematical models with software design and implementation for CPS design. Such modeling and programming must be informed by hardware, no doubt. To that end, we present ways to model relevant properties of hardware components and use those properties to make higher-level design choices. Our approach is similar to some taken for other computer science topics, such as Bryant and O’Hallaron teaching Computer Systems from a programmer’s perspective.²

Organization. We made two key decisions in organizing course content in the textbook. First, we decided to split the book into three parts, focusing respectively on modeling, design, and analysis. Figure 2 shows the organization of the current edition with the dependencies between chapters. The three parts are relatively independent of each other and are meant to be read concurrently. For example, in EECS 149, we interleave a discussion of programming via interrupts and threads (Part II on Design) with one on using interacting state machines for modeling (Part I on Modeling). This enables students to see the connections between the more theoretical modeling content and the more applied design and implementation content. Moreover, it has enabled others to use the textbook for their own, customized purposes—for instance, we are aware of more applied classes focusing mainly on Part II, whereas more theoretical classes on formal methods use portions of Parts I and III.

Figure 2. Organization of Lee and Seshia’s Introduction to Embedded Systems—A Cyber-Physical Systems Approach (2^nd edition).²¹

Finally, we decided not to include details of laboratory exercises in this textbook. A separate lab book, co-authored with Jeff Jensen, then at National Instruments, was published online¹¹ and has been updated regularly between 2012 and 2021. One reason for this separation is the difficulty in replicating a lab set-up across institutions. The lab material has evolved over the years—new labs on timers and networking have recently been added—but the overall structure has stayed largely the same. We have found a far greater number of institutions using our textbook as compared to the lab book, and this is due in part to the various hardware and space dependencies, as well as support systems required to successfully replicate the lab content.

Publishing. We decided to try a publishing experiment with the first edition of this book. Rather than going with a traditional, established publisher, we used a “print-on-demand” online publisher. This allowed us to, amongst other things, keep a free PDF version online while also providing readers with the option of purchasing a paper copy. Moreover, the paper copy could be sold at a much lower price than we believed would be the case with traditional publishers. More than a decade later, we believe we made the right choices: As of this writing, our textbook has been adopted at around 300 institutions in more than 50 countries—some of these, we are sure, would not have been possible without the PDF being available online. We found that many readers do purchase a paper copy even though a free PDF is available. In 2015 and 2016, we worked on the second edition of the book, adding many new exercises, a new chapter on sensors and actuators, and another chapter on security. This edition is published by MIT Press, with international editions and translations, and with a PDF available online for free. Having the PDF available online also made it easier for us to use the book in the online version of EECS 149 on edX, since enrollees around the world could consult whatever version was most easily accessible.

Back to Top

Trail 3: Online Education and Exercise Generation

The advent of massive open online courses (MOOCs) promised to bring world-class education to anyone with Internet access. It also placed a renewed focus on the development and use of computational aids for teaching and learning. Online education also presents a range of problems to which the field of formal methods has much to contribute. These include automatic grading, automated exercise generation, and virtual laboratory environments.

In automatic grading, a computer program verifies that a candidate solution provided by a student is “correct”—that is, it meets certain instructor-specified criteria (the specification). When the solution is incorrect, the automatic grader (henceforth, autograder) must also be able to provide the student with feedback about where the errors may be. Automatic exercise generation is the process of synthesizing problems (with associated solutions) that test students’ understanding of course material, often starting from instructor-provided sample problems. Finally, for courses involving laboratory assignments, a virtual laboratory seeks to provide remote students with an experience similar to that of a real, on-campus lab.

In 2011–2012, we began brain-storming about an online version of EECS 149, including technologies we could develop to aid such a course. We looked first at the task of automatic exercise generation. Our goal was not to completely automate the process of creating problems and solutions. It seemed both unrealistic and undesirable to completely remove the instructor from the problem-generation process, since this is a creative process that requires the instructor’s input to emphasize the right concepts. Automation is best employed in those aspects of problem generation that are tedious for an instructor. Additionally, in the MOOC setting, generating customized problems for students is impossible without some degree of automation. Finally, creating many different versions of a problem can help to reduce cheating by blind copying of solutions.

Examining problems from all three parts of the Lee and Seshia textbook,²⁰ we took a template-based approach to automatic problem generation.²⁸ Specifically, several existing exercises in the book are shown to conform to a template. The template identifies common elements of these problems while representing the differentiating elements as parameters or “holes.” To create a new problem, the template essentially must be instantiated with new parameter values. However, it is often useful to create new problems that are “similar” in difficulty to existing handcrafted problems. To facilitate this, new problems are generated using a bounded number of mutations to an existing problem, under suitable constraints and pruning to ensure well-defined results. An instructor can then select results that look reasonable.

Let me briefly outline some of our main insights²⁸ as they relate to the application of formal methods. The first insight relates to the structure of exercises. After investigating the exercises from certain relevant chapters (3, 4, 9, 12, 13) of Lee and Seshia,²⁰ we found that more than 60% of problems fit into the model-based category, where the problem tests concepts involving relationships between models, properties, and traces. Figure 3 illustrates the three entities and their characteristics. At any point, given one or two of these entities, we can ask about instances of the unknown entity. Table 1 groups exercises into different classes based on what is given and what is to be found. Each group represents an interaction between models, properties, and traces. The first column shows the given entity, the second column is the unknown entity, and the third column shows some of the variations of the same class of problem.

Figure 3. Models, properties, and traces: Entities in exercises in Lee and Seshia’s Introduction to Embedded Systems—A Cyber-Physical Systems Approach.²⁰

Table 1. Classification of model-based problems in Lee and Seshia’s Introduction to Embedded Systems—A Cyber-Physical Systems Approach (1^st edition),²⁰ Version 1.06 (as reported in Sadigh, Seshia, and Gupta²⁸).

Table 2 states a solution technique for each problem category listed in Table 1. Note that major topics investigated in formal methods, such as model checking, specification mining, and synthesis, can be applied to various tasks in exercise generation. Moreover, since textbook problems are typically smaller than those arising in industrial use, their size is within the capacity of existing tools for synthesis and verification. Indeed, we have been able to fruitfully use exercise generation, including in 2020 during the COVID-19 pandemic for generating exam problems (see the section, “Trail 5: The COVID-19 Pandemic”).

Table 2. Techniques to find solutions for model-based problems (as reported in Sadigh, Seshia, and Gupta²⁸).

Back to Top

Trail 4: EECS 149.1X and CPSGrader

Between 2012 and 2013, we began a concerted effort to develop a MOOC version of EECS 149. Berkeley had joined edX as a university partner, and a large campus effort was under way to contribute to the emerging landscape of large-scale online education. However, taking EECS 149 online was not going to be easy. For one thing, the growing interest in CPS from academia and industry had created a clear demand for CPS educational resources to be more widely accessible. Fortunately, having materials from EECS 149 and the Lee and Seshia textbook freely available online made it easier to offer a free MOOC to the broader community. Another major obstacle was the course’s lab component. Lab-based courses that are not software-only, such as EECS 149, pose a particular technical challenge for MOOCs. A key component of learning in lab-based courses is working with hardware and getting one’s hands “dirty.” We believed the learning experience to be incomplete without a lab component. However, it can be extremely difficult, if not impossible, to provide that experience online. Inspired by industrial practice, where one often prototypes a design in a simulated environment before building the real artifact, we decided to build a virtual laboratory environment for EECS 149 and blend that with suitable theoretical content to create the MOOC version.

Working with Edward Lee, Jeff Jensen, and a team at National Instruments, my research group and I developed courseware and technologies for an online CPS course. In Spring 2013, we presented a paper sketching out our main ideas for a CPS virtual lab.¹² Then, in 2013-2014, we began an effort that culminated with two key contributions: EECS 149.1x,²² the online version of EECS 149 offered on edX in 2014, and CPSGrader, an automatic grading and feedback system for virtual laboratory environments²⁹ that we continue to use and develop.

CPSGrader. For a moderate-size class, one could provide an infrastructure where students can log in remotely to a system that is preconfigured with all development tools and laboratory exercises, allowing students to deploy their solutions onto a target platform and view how they perform in a real lab setting. Projects exploring this approach are already under way, such as MIT’s iLab project⁷ and Georgia Tech’s Robotarium project.³⁶ However, in the MOOC setting, the very large numbers of students make such a remotely accessible physical lab expensive and impractical. A virtual lab environment, driven by simulation of real-world environments, appears to be the only viable solution at present.

To this end, we developed CPSGrader, which combines virtual lab software with automatic grading and feedback for courses in the areas of cyber-physical systems and robotics.^13,14,29 CPS-Grader has been used in the on-campus “Introduction to Embedded Systems” course at Berkeley¹⁸ and its online edX counterpart.²² Recall that in the lab component of this course, students program the Cal Climber¹¹ robot (see Figure 1) to perform certain navigation tasks, such as obstacle avoidance and hill climbing. Students can prototype their controller to work within a simulated environment based on National Instruments’ LabVIEW Robotics Environment Simulator (see Figures 4 and 5).

Figure 4. Cal Climber in the LabVIEW Robotics Environment Simulator.

Figure 5. Simulator with autograding functionality used in EECS 149.1x.

The virtual lab dynamical model is complex and physics-based; due to its complexity and dependence on third-party components, we decided to treat it as a black box. CPSGrader employs simulation-based verification, arguably the main scalable formal approach in this setting. Correctness and the presence of certain classes of mistakes are both checked using test benches formalized in signal temporal logic (STL).²⁴ However, coming up with these STL properties can be tedious and error-prone, even for instructors well-versed in formal methods. Therefore, in Juniwal et al.,¹⁴ we showed how these temporal logic testers can be synthesized from examples. Our approach can be viewed as an instance of machine learning from student solutions that have the fault (positive examples) and those that do not (negative examples). An active-learning framework was also developed to ease the burden of labeling solutions as positive or negative.¹³ The resulting test bench then becomes the classifier that determines whether a student solution is correct, and, if not, which fault is present. CPSGrader was used successfully in the edX course EECS149.1x offered in May–June 2014,²² an experiment we describe in more detail in the upcoming section titled EECS 149.1x.

In recent years, methods for mining temporal logic testers to capture classes of student mistakes have improved.³⁵ Several interesting directions for future work include developing quantitative methods for assigning partial credit and online monitoring of such testers to improve responsiveness.

EECS 149.1x. Over a seven-week period between May 2014 and June 2014, we offered “EECS 149.1x: Cyber-Physical Systems” free to the public on the edX platform. To our knowledge, this was the first MOOC covering a breadth of topics in CPS offered on any of the major platforms. We simplified some of the course content from the material in EECS 149, since we could not rely on students in the MOOC having the same pre-requisite background as UC Berkeley students. The course included 49 lectures comprising nearly 11 hours of video content. It also included six weekly lab assignments that somewhat mirrored the six-week structured lab sequence in EECS 149.

The Cal Climber lab was turned into an entirely online lab to be performed using the virtual lab software we created—CyberSim and CPSGrader—and described in the preceding section. We also offered an optional “hardware track” for those students who wished to purchase and assemble the hardware components themselves. Theoretical course topics covered in the lectures included inline quizzes but no separate homework assignments: The lab sequence was integrated with the theoretical course content. We organized the lectures into the following 10 modules: Introduction to CPS, Memory Architectures, Interrupts, Modeling Continuous Dynamics, Sensors and Actuators, Modeling Discrete Dynamics, Extended and Hybrid Automata, Composition of State Machines, Hierarchical State Machines, and Specification and Temporal Logic.

The impact in MOOCs can be difficult to quantify, but here are some numbers from EECS149.1x. The course attracted a peak enrollment of 8,767 students, of which 2,213 submitted at least one lab assignment. Of these, 342 students passed the course (4% of peak enrollment). Around the sixth week of the course, we conducted an anonymized survey of students still engaged in the MOOC, which produced some very interesting and encouraging data:

• We seemed to attract students who had already taken other MOOCs—54% of those who stayed until the sixth week had taken three or more previous MOOCs.
• A majority of the students were new to model-based design with a language such as LabVIEW; even so, 73% found it useful to do the lab assignment in two different languages: C and LabVIEW.
• 86% of the students rated CPSGrader as being useful in their lab assignments, an encouraging sign for the use of such tools in personalized education.
• While only a small fraction of students took the optional hardware track, more than 90% of them found that if their solution passed CPSGrader in the virtual lab, it worked on the real hardware. This statistic was very encouraging. At that time, we had been using CyberSim in the on-campus lab without autograding and saw no such correlation. Using CPSGrader seems to have pushed students to debug the corner cases, which also improved the reliability of their solution on real hardware.
• We polled the students on the lecture modules they liked most, from both a theoretical standpoint and according to relevance to lab assignments. This poll was motivated in part by our desire to learn whether a general, non-Berkeley student population—including several working in industry—would be receptive to the large formal-methods content in the MOOC. To our pleasant surprise, we found that the modules on Hierarchical State Machines and Composition of State Machines were in the top three topics found relevant to the lab, while the module on Formal Specification and Temporal Logic ranked in the top three theoretical topics. This provides some real-world validation of the receptiveness to formal methods in a general student population.

In summary, creating the online version of EECS 149 was a rewarding experience, a first step toward creating a strong learning experience for “lab-based MOOCs” in science and engineering. In the 2015–2019 period, we continued to use CPSGrader and the content created for the edX class in our on-campus course offerings. Fresh challenges came in 2020, which we detail in the next section.

Back to Top

Trail 5: The COVID-19 Pandemic

The COVID-19 pandemic has left few aspects of our lives untouched. Education at all levels has seen severe disruption. Computer science and engineering education, in general, and CPS education, in particular, have faced multiple challenges. These include effectively engaging with students online, conducting online tests and exams in a fair and equitable way, and offering an adequate lab-based coursework experience. At Berkeley, Prabal Dutta and I co-taught EECS 149 entirely online (remote) during the Fall 2020 semester. This gave us an opportunity to deploy the ideas and technologies discussed in earlier sections in the context of remote CPS education. This section reports on our experience.

Generating problems for online exams. Conducting exams online while ensuring academic integrity poses a particular challenge in the remote setting. Many courses addressed this by performing various forms of online proctoring or monitoring. Others moved to a take-home exam model. We decided to adopt an approach that generated a different, randomly generated exam for each student, to be answered within the exam time slot. This approach follows the philosophy like the one described in the “Trail 3…” section, where instructors write a template for an exam problem as opposed to a specific problem instance, and a program automatically generates instances. Software infrastructure for implementing problem templates was created by Dutta and his student Rohit Ramesh, and we implemented some of the generation strategies from our prior work,²⁸ also outlined in the “Trail 3…” section. Overall, this approach worked quite well—exam scores followed the typical pattern observed with in-person exams, and, except for a few minor errors in the templates themselves, the exams ran smoothly. A major limitation, however, was that substantial manual work was needed to craft templates that produce instances of similar difficulty levels, and which are easy to grade. More research is needed to reduce the manual effort and devise effective ways to generate problems of a similar difficulty level, perhaps leveraging the theory of algorithmic improvisation.^4,31

---

The intellectual core of CPS is in models and abstractions that conjoin computation and physical dynamics.

---

CPSGrader revisited. As in the EECS149.1x MOOC, we saw a virtual laboratory as a key component of the remote-learning experience. However, we had to make several changes to CPS-Grader²⁹ and the underlying simulator. In the years since our MOOC offering, the hardware platform had migrated to the Kobuki (see Figure 1), so we needed a simulator that supported this new platform. We also wanted a robust, open-source simulator that could be more easily adapted not only for our use at Berkeley but also anywhere else our course materials were being adopted. The open source Robot Operating System (ROS) Gazebo simulator⁶ fit the bill. We also leveraged the Scenic probabilistic programming language⁵ to automatically generate a diverse collection of test environments for use in development, autograding, and debugging. A new version of CPSGrader was interfaced to Gazebo and equipped with temporal logic testers. In a post-lab survey, more than two-thirds of the students reported that the temporal logic testers were useful in finding problems with their code. The students also repeated the lab with real hardware but with a different robotic platform. Even so, more than 35% of the students reported making no or only minor changes to their code to get it to work on the real hardware, and the rest reported fairly moderate changes involving tuning parameters or adding delays to account for timing differences between the simulated world and the real world. This has provided additional validity for using virtual laboratory software with temporal, logic-based auto-grading and debugging.

Low-cost hardware kits. Even with the virtual lab software, we wished to give students the opportunity to work with real hardware. Social distancing meant that students would need to receive their own robotic platform, unlike in traditional lab settings where students share the same robots during and across labs. With an enrollment of about 60 students, we figured that this option was cost-effective, provided the cost of each hardware kit was kept to around 100 USD. This necessitated a move from Kobuki to the less expensive TI-RSLK system.⁹ However, we interfaced the same microcontroller boards to this new platform, keeping the software experience largely the same. The lab sequence also essentially remained the same, following the lab book.¹¹ We have had a positive student experience, with many students also using the same platform for their final capstone design projects.

On the whole, our experience deploying personalized and remote-learning technologies for CPS during the pandemic has been positive. Even post-COVID-19, when done right, these technologies help take steps toward educational equity by reaching a broader community of students than traditionally possible.

Back to Top

Lessons Learned and Outlook

Evolution of the course and lessons learned. The 14-year expedition described herein has been a productive educational experiment in many ways. Here is a selection of the main lessons learned:

Durability of course topics. The set of topics we selected from 2008–2010 has proved durable over the years. One notable topic not covered by the textbook that has grown in importance is networking and distributed CPS. In the early years, we covered networking in a rather ad hoc fashion in lectures, but we noticed that the use of networking was growing in student projects. Thus, over the last five years, networking has become a core topic covered in lectures, and we also expect to add a chapter in the next edition of the textbook. Additionally, emerging areas for research and industrial practice in CPS^15,32 indicate other topics for inclusion, including machine learning in embedded systems, human-CPS (cyber-physical systems that work in concert with humans), and low-power and energy-efficient design.

---

Technologies such as CPSGrader can provide students with personalized feedback on their work.

---

Evolution of lab content. The same high-level structure of our lab—the Cal Climber lab sequence followed by capstone design projects—has served us well throughout the last 14 years. However, the details have continually been adapted. For example, microcontroller boards have evolved from an 8-bit microcontroller in the iRobot Create command module, to the 32-bit National Instruments myRio, and to the current Nordic nRF52832 combined with a homegrown Berkeley Buckler daughterboard—providing a rich combination of computation, sensing, and networking capabilities. Capstone design projects have also provided a path for the exploration of topics that have made their way into the formal course curriculum, such as networking, machine learning, and cloud computing. Finally, we have learned that simulation can play a very important role in lab education, in both on-campus and online settings, but it needs to be combined with verification and testing infrastructure that pushes students to think about corner case scenarios. With such infrastructure, solutions that work in simulation also work on the real hardware. An interesting future experiment would be to connect a CPS-Grader-style system with infrastructure such as the Robotarium.³⁶

Theory and practice. Our experience has shown that one can teach mathematical modeling and formal methods alongside practical topics and experimental work in the same course. However, it is necessary to interleave these two types of topics and make the connections explicitly for the students to appreciate. We learned that there is an appetite for theory even in the MOOC population, where students had little to no experience in theoretical topics. Our philosophy has been that the intellectual core of CPS is not so much in specific implementation technologies—for example, specific approaches to programming microcontrollers, interfacing to sensors, or using cloud computing services—but is instead in models and abstractions that conjoin computation and physical dynamics. We learned that models are useful in practice even when they are not fully supported by model-based design tools providing correct-by-construction guarantees or code generation. For example, one of our alumni became director of software at a robotics startup, where he employed a Statecharts-based design approach that effectively reduced bugs and improved maintainability under tight product schedules.

Outlook to the future. As I think about the future of CPS education, and of engineering education in general, two sets of articles come to mind. The first is a pair of thought-provoking articles written by Lee and Messerschmitt, one discussing the future of engineering education¹⁶ and the other presenting an innovative viewpoint on higher education in the year 2049.¹⁷ The second set of articles covers recent studies of and opinions about the impact of automation and information technology (IT) on jobs.^8,33,34

A few threads emerge. First, rapid technological change and increasing automation make lifelong learning ever more important. Second, humans will increasingly need to collaborate with intelligent machines in their jobs. How should we design CPS education for such a future? I approach this question from the viewpoint of leveraging the work described in the preceding sections.

The U.S. National Academy of Engineering has identified “personalized learning” as one of the 14 grand challenges for engineering in the 21^st century.²⁷ Additionally, institutions around the world are seeing rapid growth in the numbers of students who want to take classes and major in computer science and related areas, including CPS/IoT. These institutions are seeing pressure on campus teaching resources due to burgeoning enrollments. Technologies for personalized education, such as CPSGrader, can play a role in reducing this pressure while ensuring that instructor attention is used more effectively. They can also help to broaden access to CPS education.

It is harder to predict what role CPS education can play to mitigate the impact of automation and IT on jobs. Technology is clearly only part of the solution, as evidenced in this report⁸ and by Vardi.³⁴ Even so, technologies for personalized education, such as CPSGrader, can provide students with personalized feedback on their work, in both online and on-campus courses. Virtual lab technologies, integrated with low-cost hardware kits, can enable students to build up expertise in a vocational topic which can be attractive to prospective employers. However, experience with MOOCs so far has shown that the students who benefit the most are the ones who were most motivated and best prepared in the first place.

How can one ensure students who need more help can benefit as well? For this, we need more work to develop structures such as the virtual “village” discussed by Lee and Messerschmitt,¹⁷ in which personalized CPS learning technologies are well-integrated with a community of mentors and peers forming an effective support system for the students.