The Need to Balance Innovation and Implementation in Broadening Participation

A key component to the U.S. National Science Foundation’s 2012 Computing Education for the 21^st Century (CE21) solicitation (NSF 12-527) is the Broadening Participation (BP) Track. The BP track is sorely needed to address the problem of underrepresentation in the computing educational pipeline and workforce. However, we have concerns that the focus of the BP track will not bring about the increases in computing interest and pursuit that are desired. Specifically, we are concerned that the BP Track puts too much focus on proposals that "aim to develop and assess novel interventions that contribute to our knowledge base on the effective teaching and learning of computing for students from the underrepresented groups: women, persons with disabilities, African Americans, Hispanics, Native Americans, and indigenous peoples (emphasis added)." We believe the requirement for novelty or innovation may limit what can be achieved in broadening participation.

There has been a significant decline in the number of students interested in pursuing computing fields since 2001.¹ This decline in interest is seen in the reduction of the number of computer science majors by more than 50% from 2000 to 2007, with modest improvements in numbers the past few years, and in the slight decline in the number of high school students taking the Advanced Placement Computer Science Exam.^1,5 This lack of interest directly contrasts with the high demand for computing professionals nationally and internationally. In fact, the U.S. Department of Labor estimates that computer and mathematical jobs will increase at more than twice the average rate of all jobs in the economy with more than 750,000 new professional computing jobs opening from 2008 to 2018.² To help address these issues, the National Science Foundation (NSF) program called CE21 started nearly two years ago with the vision of "increasing the number and diversity of students who develop and practice computational competencies and are engaged and have the background in computing necessary to successfully pursue degree programs in computing-related and computationally-intensive fields of study." This is of national importance because computational fluency or what is often called computational thinking has become a prerequisite for most science and engineering professions, and education in it should begin long before college.

---

We believe the requirement for novelty or innovation may limit what can be achieved in broadening participation.

---

In addition to declining student interest in computing nationally, there is significant underrepresentation of women, many minority groups, and people with disabilities in computing fields. The underrepresentation of these groups weakens U.S. national competitiveness not only because of the unmet demand for computing professionals, but because industry needs diverse teams to create the best solutions to computing problems.

We are not suggesting that innovation is undesirable—it is important and necessary to continue moving research and education forward. Innovative proposals have the capacity to break new ground with their findings, but innovation also entails risks that the novel project will not be successful and may not have the desired results. We believe that in the broadening participation field, there must be a balance of innovation and implementation of known effective practices in order to achieve significant results. We feel it is too risky to put a strong emphasis on novelty when it comes to making progress on the participation of underrepresented groups in computing.

There are many effective practices for broadening participation that are well known. In fact, NSF has already invested much money to identify these practices. If the goal of broadening participation is to significantly increase participation from underrepresented groups entering computing fields, then projects should be encouraged to use these effective practices and conduct thoughtful, quality evaluations of the work. There is room for innovation in the funding model, but a balance of innovation and implementation of known effective practices will help minimize the risk and increase the participation of underrepresented groups.

We suggest the requirement for novelty be relaxed in CE21 and other NSF programs targeted at broadening participation for the following reasons, which we discuss later in further detail:

• The National Science Foundation’s 1950 Charter includes goals of research and education, with even emphasis on both. In addition, the NSF Directorate for Education and Human Resources has a responsibility to improve participation of underrepresented groups in Science, Technology, Engineering, and Mathematics (STEM) and relaxing the novelty requirements for broadening participation activities will help increase participation faster.
• The understanding of the Broader Impacts Criterion for merit review is in flux as a result of the National Science Board (NSB) Merit Review Criteria report, and there is an opportunity for NSF to ensure high impact of broadening participation activities by interpreting the NSB recommendations to include implementation of known effective practices.

Back to Top

The NSF Plays a Key Role in Human Resource Development

There is a common understanding in the computer science community that the primary role of NSF is supporting innovative research and a secondary goal is education and human resource development. However, the NSF Act of 1950 points to a balance between research and education (U.S. Code, Title 42, Chapter 16). The Directorate for Education and Human Resources (EHR) plays an important role in both education and broadening participation. Note that CE21 is a joint venture of the Directorate for Computer and Information Science and Engineering (CISE), EHR, and the Office of Cyberinfrastructure. The influence of EHR is seen in all three of the tracks of the CE21 solicitation.

We believe human resource development requires programs that implement innovations in addition to well-known effective models for broadening participation. Throughout the NSF there are examples of programs that serve a broadening participation function without a core emphasis on innovation. The NSF Graduate Research Fellowship (GRF) program, Gender in Science and Engineering (GSE) Extension Services program, ADVANCE, Research in Disabilities Education (RDE), and Louis Stokes Alliance for Minority Participation (LSAMP) are examples of programs that focus on broadening participation through effective practices, collaboration, and institutional change. Human resource development programs do not have to be innovative, but these programs are more likely to succeed by implementing known effective practices, like mentoring, research experiences for undergraduates, and inclusive pedagogy.

Back to Top

The Future of the Broader Impacts Criterion

In December 2011, the National Science Board (NSB), which is the governing board of the NSF, released a report including recommendations for the National Science Foundation Merit Review process (NSB/MR-11-22).³ The report was partially in response to the America COMPETES Reauthorization Act of 2010 that requested a revision of the Broader Impacts criteria to better match the need to improve the U.S. STEM workforce. The NSB’s revised criteria are found on pages 10–13 of the report. Overall, the NSB recommends that the two merit review criteria (Intellectual Merit and Broader Impacts) have served the NSF well, but that it is important for the NSF to clarify the meaning of the different criteria, especially the Broader Impacts criteria as it is less well understood than Intellectual Merit.

---

We propose that broadening participation activities be examined through a lens for impact, not just novelty.

---

The National Science Board recommends that five elements be considered in the review for both merit review criteria. Of particular interest to us is element two, which appears to require innovation and applies to Broader Impacts activities, including those intended to increase the participation of women, persons with disabilities, and underrepresented minorities in STEM. It suggests that broadening participation activities should be innovative. Unfortunately, this demand for innovation is counter to our belief and experience that a balance between innovation and implementation is needed to achieve broader participation of underrepresented groups. Hopefully, in the implementation of the clarified Merit Review Criteria, NSF will try to achieve a balance between innovation and implementation in broadening participation-focused awards. This may require a new interpretation or revision of element two of the criteria.

Back to Top

Implications for Broadening Participation in Computing

For the same reasons that we are concerned about the possible implementation of the revised and reinterpreted Broader Impacts Criterion, we are also very concerned with the novelty requirement for CE21 BP proposals. We propose that broadening participation activities be examined through a lens for impact, not just novelty. Finding more of a balance between innovation and implementation in the funding of broadening participation activities will improve our collective progress toward a more diverse, competitive workforce.

There is a suggestion in the CE21 BP solicitation that interventions, "if proven successful, could be implemented within a BPC-A Alliance." Generally speaking, the BPC-A Alliances lean toward the implementation of known effective practices over innovation, which seems to provide a balance between innovation and implementation in the entire Directorate for Computer & Information Science & Engineering’s Broadening Participation portfolio. Unfortunately, the alliances only represent a small fraction of effort needed to make a noticeable difference, and making a noticeable difference requires the implementation of effective practices.

From a practical standpoint, proposers and reviewers should think broadly about how the novelty requirement for CE21 BP proposals should be interpreted. We suggest a broad view of novelty that would include the implementation of known effective practices in a new way or with a new population. The activity could be a direct intervention with students, the creation of an inclusive curriculum, or a project to create institutional change. In all cases, the activity should have a rigorous evaluation.

We believe the balance between innovation and implementation is also relevant for other countries and funding agencies, although we suspect the exact balance will differ depending on the available evidence regarding effective practices for broadening participation.

Back to Top

Back to Top