What ‘Beginning’ Students Already Know: The Evidence

For the past eight years I have been teaching the introductory programming course at ETH Zurich, using the  “inverted curriculum” approach developed in a number of my earlier articles. The course has now produced a textbook,  Touch of Class [1]. It is definitely “objects-first” and most importantly contracts-first, instilling a dose of systematic reasoning about programs right from the start.

Since the beginning I have been facing the problem that Mark Guzdial describes in his recent blog entry [2]: that an introductory programming course is not, for most of today’s students, a first brush with programming. We actually have precise data, collected since the course began in its current form, to document Guzdial’s assertion.

Research conducted with Michela Pedroni (who played a key part in the organization of the course until last year) and Manuel Oriol (now at the University of York) led to a paper which, although not yet published, is available as an ETH technical report [3]: What Do Beginning CS Majors Know? It provides detailed empirical evidence on the prior programming background of entering computer science students.

There are some obvious qualifications to these results: The information was provided by the students themselves in questionnaires at the beginning of the course; it covers the period 2003-2008; and it applies to ETH only. But we do not think they fundamentally affect the picture: We do not know of any incentive for students to bias their answers one way or the other; the 2009 and 2010  questionnaires did not show any significant deviation; and informal enquiries suggest that the ETH student profile is not special. In fact, quoting from the paper:

We did perform a similar test in a second institution in a different country. The results from the student group at University of York are very similar to the results at ETH; in particular, they exhibit no significant differences concerning the computer literacy outcomes, prior programming knowledge, and the number of languages that an average student knows a little, well and very well. A comparison is available in a separate report [4].

 Our paper [3] is short and I do not want to repeat its points here in any detail; please read it for a precise picture of what our students already know when they come in. Let me simply give the statistics on the answers to two basic questions: computer experience and programming language experience. We asked students how long they had been using a computer:

These are typically 19-year-old students. A clear conclusion is that we need not (as we naively did the first time around) spend the first lecture or two telling them how to use a computer system.

The second chart summarizes the students’ prior programming experience (again, the paper presents a much more detailed picture):

 

18% of students (the proportion ranges from 13% to 22% across the years of our study) have had no prior programming experience. 30% have had programming experience, but not object-oriented; the range of languages and approaches, detailed in the paper, is broad, but particularly contains tools for programming Web sites, such as PHP. A little over half have object-oriented programming experience; particularly remarkable are the 10% who say they have written an object-oriented system of over 100 classes, a significant size  for supposed beginners!

Mark Guzdial’s point was that  “There is no first“: We are teaching students who already know how to program. Our figures confirm this view only in part. Some students have programmed before, but not all of them. To quote from our article: 

 At one end, a considerable fraction of students have no prior programming experience at all (between 13% and 22%) or only moderate knowledge of some of the cited languages (around 30%). At the present stage the evidence does not suggest a decrease in either of these phenomena.

At the other end, the course faces a large portion of students with expertise in multiple programming languages (around 30% know more than three languages in depth). In fact, many have worked in a job where programming was a substantial part (24% in 2003, 30% in 2004, 26% in 2005, 35% in 2006, 31% in 2007 and 2008).

This is our real challenge: How to teach an entering student body with such a variety of prior experience. It is difficult to imagine another scientific or engineering discipline where instructors face comparable diversity; a professor teaching Chemistry 101 can have a reasonable image of what the students know about the topic. Not so in computer science and programming. (In a recent discussion, Pamela Zave suggested that our experience may be comparable to that of instructors in a first-year language course, say Spanish, where some of the students will be total beginners and others speak Spanish at home.)

How do we bring something to all of them, the crack coder who has already written a compiler (yes, we have had this case) or an e-commerce site, and the novice who has never seen an assignment before? It was the in-depth examination of these questions that led to the design of our course, based on the “outside-in” approach of both using components and progressively discovering their insides; it also led to the Touch of Class textbook [1], whose preface further discusses these issues.

A specific question that colleagues often ask when presented with statistics such as the above is why the experienced students bother to take a CS course at all. In fact, these students know exactly what they are doing. They realize that their practical experience with programming lacks a theoretical foundation and that they have come to university to learn the concepts that will enable them to progress to the next stage. This is why it is possible, in the end, for a single course to have something to offer to aspiring computer scientists of all bents

Normal 0 false false false EN-US X-NONE X-NONE MicrosoftInternetExplorer4

—those who seem to have been furiously coding from kindergarten, the total novices, and those in-between.

References

[1] Bertrand Meyer: Touch of Class: Learning to Program Well with Objects and Contracts, Springer Verlag, 2009, see details and materials on the book page.

[2] Mark Guzdial: We’re too late for  “First” in CS1, Communications of the ACM Blog, see here.

[3] Michela Pedroni, Manuel Oriol and Bertrand Meyer:  What Do Beginning CS Majors Know?, Technical Report 631, ETH Zurich, Chair of Software Engineering, 2009, available here. http://se.ethz.ch/~meyer/publications/teaching/background.pdf

[4] Michela Pedroni and Manuel Oriol: A comparison of CS student backgrounds at two technical universities, Technical Report 613, ETH Zurich, Chair of Software Engineering, 2009, available here.