3 Proposals to Change How We Teach Computing In Order to Reduce Inequality

The Black Lives Matter movement, especially the protests since the tragic death of George Floyd, have made me more aware that I need to learn more about race (as I wrote about in June for Blog@CACM). I have been reading the statements of support for the movement and calls for change, especially those targeted to computing (see links to several of them here). The one that particularly caught my eye was the statement from the University of Maryland’s CS department about their recommendations to improve diversity and inclusion. They call for:

Creating a task force within the Education Committee for a full review of the computer science curriculum to ensure that classes are structured such that students starting out with less computing background can succeed, as well as reorienting the department teaching culture towards a growth mindset.

I don’t know enough about race. I’m reading and trying to learn more. What I can do is use what I know to support the calls to action from these more well-informed sources. My research is explicitly about how to structure CS education "such that students starting out with less computing background can succeed" and addressing the teaching culture away from a fixed mindset and towards a "growth mindset." Over the last four weeks, I have been sharing a blog post series about how we might teach computing differently, inspired by the U. Maryland statement. This post summarizes the series (and extends with new information), with links back to the original (more detailed) posts.

The pandemic makes this change more urgent. Teaching on-line will likely hurt our students with the least preparation (see post here). I recommend the terrific report from Digital Promise and other studies of what happened in Spring 2020 as classes went on-line. The new CRA report on what happened to CS teaching during the Spring of 2020 suggests that the use of high-quality teaching methods declined (see report here) — and over a quarter of those teachers surveyed plan to do exactly the same things this Fall.

These reports show that all students suffered, and students with the least preparation suffered the most. The Digital Promise report shows how Hispanic and Black students were particularly stressed, suffering more connectivity issues and struggling more to find a quiet place to work compared to non-Hispanic White students. The inequities in our classes are already there. Teaching on-line this academic year is going to make it worse. We have to change now.

Teaching computing to reduce inequity requires a change to how we teach. We can’t teach in the exact same way and expect change. If the goal is to reduce inequity, it’s important to make change so that the greatest benefit goes to the students with the least privilege, with the least preparation. You can’t lift all students up equally and expect the gap to reduce.

In general, we teach computer science to privilege the majority of students. Here’s a great example of how we teach CS inequitably (see post here with more details). The Advanced Placement exam in Computer Science Principles (CSP) (see link here) uses a pseudocode in all of the programming problems. It’s not executable, so it’s different from whatever language the students learned in their CSP class. In a study with over 900 students, Allison Elliott Tew compared student performance on tests between the language they learned in an intro class (Java, MATLAB, and Python in her study) and an isomorphic test in a pseudocode.

• For the top scoring students, the difference in scores was only 7%.
• For the two quartiles with the lowest scores, the difference was 17% and 22%.

For the majority of students, having to shift from the language that they learned to a pseudocode probably won’t affect their exam score too much. But for the least prepared students, shifting to pseudocode may mean the difference between passing and failing. Using pseudocode on the AP CSP exam is a kind of tax. Everyone has to manage a bit of difficulty to map to a new language they’ve never used. But it’s a regressive tax. It hurts the least-prepared students the most.

Here are three concrete proposals that CS teachers can implement to reduce inequality in their own classes.

Proposal 1: Use research-based methods that advantage the least prepared students (link to full blog post)

We often make choices so that our students achieve "excellence." We prepare them for great software engineering jobs. The questions that get asked on technical interviews explicitly drive how many CS departments teach algorithms and theory. Whose definition of "excellence" do we care about? Are Silicon Valley entrepreneurial values the only ones that matter? Not all students want to go to Google or Amazon. The students who are not in the majority in our classes or who have not succeeded in the past in our classes may want different things out of a computer science degree. The goal of "becoming a great software engineer" does not consider the wide range of alternative endpoints for computing education (see post about alternative endpoints here). Not all our students want those kinds of jobs. Many of our students are more interested in giving back to their community or using computing to serve values beyond Silicon Valley (mostly white and mostly male) values (see post here).

Here are four evidence-based methods that have been used successfully to increase the success of the less-prepared students:

• Peer Instruction: If you think you’re using Peer Instruction, but you don’t do two rounds of answering on each in-class question, you’re doing it wrong. See http://peerinstruction4cs.com/ for how to do it right. Peer Instruction has a differential impact (see study here). It doesn't hurt the top students, but it reduces failure rate (measured in multiple CS courses) for students with less background (see paper here).
• Connect to community values: I blogged last year about a paper (see post here) that showed female, Black, Latino/Latina, and first-generation students want to use CS to help society, more than majority (typically white and male) students who study CS because they like computers. These students often do not see a connection between what's being taught in CS classes and what they want. Simply explaining how our topics could be used to address community needs could go a long way towards improving retention and student interest in staying in CS.
• Parsons problems: Mixed-up code problems are efficient and effective as a fine-grained assessment tool. Efficient matters for the students who are not privileged, who are working other jobs, as Manuel Pérez-Quiñones pointed out. This is particularly important in a pandemic when students will have more outside stresses on them. A mixed-up code problem lets students show what they understand, even if they can’t make the code work, which makes it an effective finer-grained assessment tool.
• Subgoal labelling: In a recent result, subgoal labeling in an introductory computing course, compared to one not using subgoal labeling, led to reduced drop or failure rates. That’s a differential benefit. There was not a statistically significant improvement on learning (measured in terms of exam scores), but they retained the students most at risk of failing or dropping out in the course.

Proposal 2: Make the highest grades achievable by all students (link to full blog post)

If you grade based on "excellence" or code quality, you may have an "Apprenticeship" teaching perspective. You want to prepare students to be great software developers. A student with less computing background is unlikely to earn the top grades in such a class.

If you curve the course grades, or limit the number of top grades that you give out, you are presuming a fixed mindset. If you believe that there is a random distribution of "talent" or "ability" or "Geek Gene" in the course, and (critically), there's nothing much that teaching can do to change that, then it makes sense to grade to the curve. There can be only a few "A" slots, more "B" slots, and so on. Empirical evidence suggests the opposite – good teaching can trump a lot of other factors. Belief in a growth mindset leads to better learning outcomes and better performance (see Wikipedia page). Even more than the students, it’s critical that teachers believe that their teaching can help.

I suggest instead that we give grades based on success at achieving learning objectives, which is closest to the alternative "Transmission" teaching perspective. Assessments of understanding should be as objective as possible. Grades should represent achievement of the learning objectives and nothing else. If we value teaching, and believe that students can get better at computer science, then over time, we should teach better and students should learn more. If students learn more, they should get a higher grade.

I'm reading Grading for Equity by Joe Feldman (see his website here). He points out all the other factors that influence grades that have nothing to do with learning. Some of these extra-curricular factors – like the ability to meet deadlines that presume privileged, full-time student status without outside pressures – are less likely for Black, Hispanic, poorer, or first-generation college students. These factors influence the production of high-quality code even more than they influence learning. These pressures are going to be even greater during a time of a pandemic.

Students with a lot of computing background get an easy "A" in courses that focus on teaching to the learning objectives — they may already know most of what they need. That's fine. I wrote a series of blog posts on "Boredom vs Failure" (here's the first post in the series, and here's the last one). The question is: which is worse, to bore and give easy A's to the most privileged and most prepared students, or to fail (or discourage to the point that they drop) the students with less privilege and the least computing background? Think about the students who might fail or drop out in a system that makes sure that the most well-prepared students are challenged and engaged. Each one of those students who continues on does more to change the status quo than does keeping the more privileged students from getting bored. Helping the students with less computing background succeed makes a much bigger difference for society long-term than does keeping entertained the most privileged students.

Proposal 3: Call a truce on prosecuting plagiarism on programming assignments (link to full blog post)

The calls to "Defund the Police" that are common at Black Lives Matter protests are (mostly) not about getting rid of the police. Rather (as this NYTimes article suggests), they’re about changing the focus. We should spend less on prosecuting the criminals, and more on supporting community health and welfare.

We should do the same for computer science education. Let’s make it less about finding and getting rid of the students who cheat and plagiarize code. Let’s make it more about increasing learning opportunities and helping the students with the least computing background succeed. Take the time you’d normally spend gathering evidence and making cases to academic honor councils or committees, and instead spend it on helping your students with least-privilege and least-preparation succeed.

This requires a change in how we think about programming assignments. First, programming assignments should not be the time to "figure it out," to fill in the gaps in the lecture. Most students are not going learn best by "discovering" CS knowledge while programming (see link here). Rather, programming assignments are about practice for what they’ve seen before. Programming is a great place for learning, since it provides feedback on our tests and hypotheses.

Second, programming assignments are not a great way to evaluate learning. Being able to program does not indicate understanding. The recent ITiCSE 2020 paper by Jean Sala and Diana Franklin showed that use of a given code construct was not well correlated with understanding of that code construct (see paper here). It's also the case that students may understand the concept but can't make it work in code. We have known for years how to create tests that measure programming knowledge. Using programming assignments for assessment is inefficient (takes more time than a test), and it creates stress and cognitive load on the students. I recommend the work by Kinnunen and Simon on how introductory programming assignments can depress students' confidence, with a particularly strong impact on women.

When a student cheats on a programming assignment, they are often prosecuted for "illicit collaboration." Now more than ever, we want students to collaborate. The evidence of the value of collaboration during programming is terrific – it helps with learning, motivation, and persistence in CS. But where's the line between allowed and illicit collaboration when it's all over video calls? I'm worried about students not collaborating because they fear that they'll cross that line. I have talked to students who won't collaborate because they fear accidentally doing something disallowed. It will be even harder for students to see that line in a pandemic.

Using automated tools like MOSS is still a good idea, but instead of using it to gather evidence for the academic misconduct case, use them to identify students who need help. Students most often cheat if they feel anonymous, or if they feel that the pressure of an assignment and no idea how to make progress on it. These are both going to be more common in a pandemic. Students are going to feel anonymous in an on-line setting (especially those who would feel alone, isolated, and invisible even in an in-person class). We will likely miss teaching things as we convert from in-person to on-line. I recommend this letter from a recent RPI grad to their faculty about the use of Chegg.

If you detect plagiarism, contact the student. Tell them what you found. Ask them what happened. Ask how they're doing. Are they getting lost in the class? Use this as an opportunity to explain what illicit collaboration is. Use this as an opportunity to figure out how you're teaching and what's going on in the lives of your students.

Some students cheat because they think that they have to. "If I don't cheat and everyone else does, I'm at a disadvantage." That's only true if student grades are comparative. That's why Proposal #2 is a critical step for Proposal #3 – stop pre-allocating, curving, or rationing grades. Use grades to reward learning, not "rising above your peers."

Most computer science classes are going to be wholly or partially remote over this next year. Let’s be honest and candid — we're likely going to make mistakes. Let’s not spend our time writing up plagiarism cases. Let’s spend our time talking to students, helping them to learn, and making sure that they feel connected. The least-privileged and least-prepared students are going to suffer the most.

It’s time to change what we do. As we build our online classes for this Fall, it’s time to make every possible change to advantage these students. We have always needed to reduce inequity. As the Black Lives Matter movement shows us the reality and cost of inequality, and as we put our students more at risk by going on-line, we must act now.

 

Mark Guzdial is professor of electrical engineering and computer science in the College of Engineering, and professor of information in the School of Information, of the University of Michigan.