Beyond Folk Pedagogy

Most researchers and many engineers teach. Some take teaching as a chore; some actually like teaching; and some researchers like teaching so much as to do research about teaching,  publishing in conferences such as ACM SIGCSE (Computer Science Education) or IEEE CSEE&T (software engineering education). I belong to the last category and with colleagues have published extensively in education; since this article is about someone else’s work, not mine, it does not include these references, but for anyone interested a list (including supervised theses on CS and SE education topics) appears here.  What I do want to talk about is an article which influenced me considerably and which I consider a classic. It is not that well known: 36 citations on Google Scholar, respectable but modest. (11 citations on Scopus, none that I could find on  Web of Science.) It deserves to be read more widely.

The paper is from 2008, by Raymond Lister [1] from the University of Technology Sydney: After the Gold Rush: Toward Sustainable Scholarship in Computing (ACE 2008, Tenth Australasian Computing Education Conference,  Wollongong, Australia, January 200). A copy is available here.

The Gold Rush paper was a revelation to me because of the fresh look it takes at issues and methods of teaching computing subjects, particularly programming. Judging by what one often hears in academic discussions, it is just as fresh today as 10 years ago. Lister is very opinionated (in a later blog article I might discuss one of his more recent papers and some my disagreements with its approach) but very cogent and often convincing. He asks the tough questions, maintains a candid and even at time modest attitude (not "I am a professor so I know better"), and does not have the slightest respect for conventional wisdom and fashionable ideas.

The paper is well-written. There is no point in paraphrasing it; all I want to do is send more readers to it. The rest of this discussion will simply list a few of lessons I learned from it. Unless otherwise noted, the statements below are not my statements but my rendering of Lister’s views. (But when I say  "I", ususally in parenthetical sentences, it does mean me, not Lister.) Any misrepresentation is my fault; but then again, do go to the source. 

1. Teaching expertise is learned

You may be an outstanding teacher or a decent teacher or a bad teacher. But you are not an education expert unless you have studied the topic of education and done research in the field. If in the previous sentence you replace "education" by  "cryptography" or "software verification" or any other technical topic, the statement is obvious; it should be just as uncontroversial when applied to teaching.

The paradox here is that good scientists who—in cryptography, software verification or whatever their field is—would not confuse a hypothesis with a conclusion or let their opinions get the better of them. Yet it seems that when they get to discussing teaching, they check their scientists’ scruples and principles  at the cloakroom. Lister wants them to apply the same standards to their teaching as to their research.

2. Folk pedagogues

Specifically, it does not matter that you have been teaching for 10 years, 20 years or 30 years: that does not—repeat—make you a teaching expert. At best, it makes you what Lister calls a folk pedagogue: someone who thinks that he knows about teaching just because he teaches. That may be worse than being a novice teacher, who at least has an open attitude.

3. Your student experience does not count

One of the habits betraying a folk pedagogue is the urge to start a sentence, in a teaching-related discussion, by: "When I was a student." (I have made this mistake like every single one in Lister’s list, but at least I watch myself now.) Stop right then and there! We do not care what worked or not for you as a student.

Think of yourself back as one of 500 young faces in that big auditorium in the first semester. How many of the other 499 went on to become computer science professors? Zero? One?

No offense meant, but you are not typical. Your experience does not matter.

4. The need for assessment

What does matter is objective assessment of the results: are students mastering the concepts?

In introductory programming, we widely fail this criterion: many graduating students miss even basic skills to understand simple programs, let alone writing their own.

By objective assessment Lister essentially means, I think, quantitative assessment. In a recent article on this blog I discussed, in a setting more general than just education, the lure and limits of quantitative approaches. Lister largely dismisses so-called Marco Polo papers (we went there and saw great things) although he writes he will continue to produce some himself.

Here is my view. I have written a few Marco Polo papers in pedagogy and believe there is room for them: papers that describe a new teaching approach, even if there is not much evidence yet, perhaps just one course in one institution. Sometimes just going by the power of an idea is right. (After all, the closest Dijkstra came to an empirical argument for jettisoning the goto was [2]: "For a number of years I have been familiar with the observation that the quality of programmers is a decreasing function of the density of gotos in their programs. More recently I discovered why the goto has such disastrous effects, and I became convinced that it should be abolished from higher-level programming languages." Try submitting such an argument to ESEM!) Conceptual papers have their place, although in the absence of empirical data the criteria are different: the idea must stand on its own, sustained by its novelty and the power of the logical arguments sustaining it.

(In retrospect, the reviews of our first SIGCSE paper, describing our approach to introductory programming, are amusing: in essence, the referees wrote, ‘OK, we’ll take your paper this time on the basis of your arguments, but don’t do it again, the next time we expect empirical support!’ We duly complied.)

5. All pedagogy is local

Every university has a unit that supports the development of good teaching practices. Its staff helps teachers, provides advice, organizes teaching workshops (which teachers may even be required to attend whether they like them or not), gives feedback, and possibly evaluates teachers. Such a central organization helps, but it cannot suffice because by its very nature its expertise and advice are cross-discipline, whereas many of the issues and solutions are specific to a discipline, be it biology, computing or anything else. So there is a limit to what we can learn from it. (Although Lister does not mention it, some universities do have chairs in e.g. mathematics education or computing education, attached to the corresponding technical department rather than a central office.)

Here is an example (mine, not in Lister’s paper) of a discipline-specific problem and solution. At some point in teaching introductory programming, you will introduce assignment instructions, x := y. You will explain that the effect is to give x whatever value y has. Empirical pedagogical research has shown, however [3], that a certain percentage of students somehow understand, in their "private universe" [4], that that this means the value of y somehow flows from y over to x: the assignment causes y to revert to some default value such as zero for numbers. The knowledgeable teacher—not just a folk pedagogue, but one who has taken the trouble to study the literature—is aware of this common misconception and explicit mentions in class that the assignment instruction has no effect whatsoever on y. He or she further directs the TAs to emphasize the point again in lab sessions. This is the kind of daily-grind pedagogy that step by step makes a programming course effective for the only goal that matters: turning students into effective programmers. But you will never get such tips from generic (discipline-independent) pedagogical workshops. Sure. you will be told to ask students a question once in a while and use at least 18-point fonts on your slides, but to be a successful teacher you must study the specific pedagogical challenges of your discipline.

6. As predictably as farmers will complain about the weather…

The main contribution of Lister’s paper is yet to come (section 7), but a side point in the paper will go straight to the heart of anyone who has taught introductory programming. I cited it in the preface to my introductory programming textbook, Touch of Class, and it really helped lower my blood pressure and raise my self-esteem after a contentious faculty meeting or two. Other programming teachers will know the setup: someone stands up to say that 2nd-year students cannot program any more these days (unlike in the old times when, in case you don’t remember, they all were Torvaldses). There is no need to mention names: whom is everyone silently looking at, if not the guy in charge of first-year programming (aka you)? I used to fret about such charges — not backed by any evidence, of course, in fact all the objective measures I could summon belied them, but folk pedagogues do not care about empirical data — until I ran into this extract from Lister’s paper [5]:

Teaching first-year programming is politically sensitive, as you must contend not only with  your students but also with an intimidating second  audience: colleagues who teach them in  subsequent semesters. Just as I blamed high school teachers for my course’s high failure rates, my colleagues blamed me  for their teaching problems. As surely as farmers complain about  the weather, computing  academics will complain about the programming abilities of students.

I understand why many colleagues who teach upper-year  electives remain blissful folk pedagogues. Had I not taught  first-year programming, I might also have remained a  blissful folk pedagogue to this day.

If I am still a folk pedagogue, at least reading these two paragraphs turned me into a  "blissful" one: now I know it’s not just me.

7. The theory

To grow beyond the folk pedagogue style and become professional, effective teachers, we should, says Lister, learn Cognitive Load Theory, the best conceptual approach to explain how succesful learning works.

The theory’s basic idea is that the difference between successive levels of expertise  is not just how much knowledge and skills people have amassed, but more fundamentally how they have moved their knowledge and skills further down on a scale of internalization, where the high end is for conscious decisions and the low end for automatic ones.

For example, a beginning driver thinks consciously about every step (declutch, brake, change gear, accelerate, turn the wheel to the right…), whereas an experienced driver can conduct an conversation and drive at the same time, performing the driving operations mechanically, and reserving conscious decision-making for such special situations as crossing an intersection or overtaking. Although Lister does not cite this example, seriously learning a foreign language involves a similar progression from the conscious to the instinctive.

In the same way, an experienced programmer recognizes a scheme, either simple (z: = x; x := y; y := z swaps x and y) or more advanced (Oh yes, I see, the code is using an instance of the Observer pattern here), right away.   beginner must analyze every single line. Our main task as teacher is to help students perform this downward transfer from the conscious to the automatic.

I am not enough au courant of the latest in education research to judge whether Cognitive Load Theory is the last word or has been displaced by something else. But it seems to have a lot going for it. It suffices to observe experts in various fields, from programmer to plumber, to notice one of their principal characteristics: they perform the basic tasks without thinking. It is not that they do not want to think: they reserve their thinking to cases that deserve it, wheras the beginner wastes his thinking on every trivial step. (The idea can be couched in the language of ecology: do not your waste your thinking resources; reserve them for where they are really needed.) If that observation is as fundamental as Lister says, then our basic task is to support the internalization process.

8. Against conventional wisdom

If the first general sketch of Cognitive Lead Theory above struck you as obvious, think again: it goes against most accepted views of teaching. (In general, you may find reasons to disagree with Lister, but do not expect platitudes.)

The recommendation to concentrate on cognitive load does not just contradict the traditional method of education,  which it is customary to deprecate today: the practice of simply  feeding facts into students’ mouths as if fattening ducks for foie gras. The cognitive-load-theory approach also contradicts the oppsite of that method, fashionable today: the insistence on teaching problem-solving.  The slogan there  is often "teach them not facts but skills", and the usual cliché proverb, don’t bring them a fish, teach them fishing.  I would surmise that in Lister’s view this fashionable method is even worse than old-style duck feeding, since learning facts does at least help the automation process.

(Not long ago an American pedagogical expert, brought in to address a workshop of experienced faculty, proudly told us that "If they can google it, I don’t want to teach it". The group was not impressed; do you also not rhapsodize at the thought that the surgeon, just before he puts the knife on you, will bring up a Web browser to find out how many kidneys you are expected to have?)

The cognitive-load approach also influences exams: rather than bringing out new problems to assess students’ general-problem-solving skills, test whether they have developed the right automated reflexes that will allow them to function as professionals, a concept defined as the ability to let the ordinary stuff take care of itself automatically and concentrate your thinking on the true challenges. (It is interesting to note that this description very much fits the French approach to student selection in the élite "grandes écoles" system, for which you prepare by essentially doing all the possible kinds of exercises in the relevant areas of mathematics, so that when you come to the exam you immediately recognize a pattern and solve it. This system is frequently criticized as producing predictable but conformist engineers, as opposed the US system which, in the view of these critics, promotes out-of-the-box creative thinking.)

Fashion in how society looks at these issues to oscillate between the  problem-solving and skill-drilling sides [6]. Recently, the latter seems to become more popular: witness the popularity of Malcom Gladwell’s  Outliers view ("it takes ten thousand hours of practice"), or the recent Guardian article telling us [7] that "there is no such thing as a gifted child". For programming education at least, Lister makes a strong case for skill-drilling.

The conclusions can be hard to accept. I like to teach concepts, and explain them as clearly as I can. What Lister is telling me, I think, is that I should focus on the application of these concepts. Drill example after example until the process of internalization is demonstrably complete. (I am writing "demonstrably" in deference to Lister’s goal of objective assessment.) Even if much of the drilling will take place in lab sessions rather than lectures, the responsibility to focus on the transfer process (internalization) primarily rests on the lecturer.

My main source of reluctance is concern about the best students. Objective, empirical measurements will tell you how many students you enable to meet the basic learning objectives of the course. This is clearly what counts most for Lister and in light of universities’ obligations to society this goal is paramount. But what about the fastest students, the brightest, the one among five hundred in that first-semester auditorium, those who (whatever the Guardian might tell us) are truly gifted? Should we bore them just for the sake of making sure the other 499 master the basics? I cannot reconcile myself to this idea. And we cannot evade the question by stating that these students do not need teachers anyway because they will learn by themselves; that view is simply wrong. The best and brightest need teachers too, just as much as the others. They need shoulders to stand on. Otherwise they will not realize their potential.

That dilemma, how to address the needs of both ordinary and advanced programming, has informed much of my own work on the teaching of programming and every responsible teacher faces it. I would enjoy learning about Lister’s view on the matter.

9. Assessment

The press wrote a few weeks ago about seven phrases that the current US administration has reportedly banned in publications of the Center for Disease Control and other federal  agencies. "Evidence-based" was one. Lister would not be popular in such an environment.

His work exhorts us to move beyond our personal intuition and experiences and focus instead on analyzing what works and what does not in turning students into competent programmers. The Gold Rush paper and his later ones (often provocative, never boring) make a strong case for treating the pedagogy of programming  as a serious object of study, and, as teachers, applying the lessons of objective analysis.

In a professional field, one reads papers every day; most papers teach you something, however paltry, but a few times in a career you come across one that actually changes your perspective on an important topic. That happened for me with Lister’s Gold Rush, and I hope that directing more lighting to it will also help others become better teachers.

Notes

[1] Disclaimer: I do not know Lister and have never interacted with him.

[2] This extract is slightly abridged from Dijkstra’s text, emphasis added.

[3] I learned this insight from Michela Pedroni (who also first brought Lister’s paper to my attention) when she was working towards her excellent PhD in programming education, and playing a key role in setting up our introductory course.

[4] Annenberg Foundation: A Private Universe, video documentary, available at www.learner.org/resources/series28.html.

[5] This citation is slightly abridged from Lister’s original text.

[6] The word "drill" does not appear in Lister’s article and may be an exaggerated characterization of the approach.

[7] I came across the Guardian article thanks to a tweet by Moshe Vardi.