Abstraction is considered to be a key skill underlying most activities in computer science (CS) and software engineering (SE).3,4 As a central concept, abstraction is taught and utilized in various guises in every CS and SE course: in requirements specifications, problem solving, and modeling through to programming and debugging. Given its importance, can one assess an individual's abstraction ability? Are abstraction skills assessable at all? If no, why? If yes, how? We decided to conduct a survey to investigate this topic. As far as we know, this is one of the first trials to address this challenge.1
In 2007, Kramer proposed to develop a test that could assess abstraction skills in the context of CS and SE, and which would be more multifaceted than that used in psychometric tests.4 In this spirit, we decided to consult experts in CS and SE research and teaching concerning the suitability of various question patterns (or templates) for assessing abstraction skills. We specifically used patterns rather than specific questions in order not to limit the experts' line of thought, and, at the same time, to provide a template that each instructor could adjust and populate according to his or her needs.
Our data analysis reveals that expert instructors tend to agree about the suitability of a pattern for checking abstraction ability when it asks students to construct the abstraction rather than when applying abstraction principles. We explain this approach by the constructionism learning theory of Papert and Harel.5
We argue that directly asking experts the question "Are abstraction skills assessable?" would be unlikely to elicit a clear indication of how to go about constructing such an assessment tool. As a concrete proposal, we therefore developed a set of 10 patterns of questions and asked the experts to assess each pattern on a 110 scale according to its suitability for assessing abstraction skills. In addition, for each pattern we presented three open questions regarding:
In this way, we gained both a quantitative and qualitative perspective of the experts' opinion.
We approached two groups with similar characteristics: 20 experts were approached based on personal relationships and 11 responded to a message that we sent to a mailing list.a N = 31 is a reasonable number for such a preliminary analysis. The demographic data presented in Table 1 indicates that the majority of the experts work at research universities, have many years of teaching experience of both introductory and advanced CS and SE courses to both CS major and SE students. This background clearly indicates they understand issues related to the challenge of teaching the topic of abstraction to good CS and SE students as well as assessing their abstraction skills.
For each pattern, we calculated the average evaluation score and the standard deviation (see Table 2). Though an agreement was not reached with respect to any pattern, we can see that:
Disagreement: What was surprising in the data analysis was the vast range of scores and disagreement exhibited with respect to almost all patterns concerning their suitability to assess abstraction skills. For example, patterns 13 received all scores. These three patterns provided the students with a set of object representations and asked the students to categorize them according to the student's choice of abstraction. In other words, in these patterns, students were not asked to create/construct an object while considering abstraction.
Agreement: As mentioned, Pattern 5 has the highest average score with the lowest distribution. In this pattern and in another one (Pattern 4, which also was assessed high), the students were asked to create some system representations by referring to abstraction. We may conclude that instructors tend to agree that constructing abstraction is a suitable way to assess abstraction skills. One of our experts explains: "This is a nice concrete type of pattern question. Looks at abstraction generation skill but in a much more constrained setting."
Theoretically, we explain the experts' ranking by the theory of constructionism, inspired by the constructivist theory. According to constructivism, learners construct mental models in order to understand the world around them. Constructivism advocates student-centered, discovery learning. One outcome of the constructivism framework is that one has to be actively involved with the material he or she learns.2 Accordingly, "The simplest definition of constructionism evokes the idea of learning by making ... .5
The scores presented in Table 2 indicate that the experts, who answered our questionnaire, saw the potential contribution of Pattern 5 to students' understanding. From a constructionism perspective, this can be explained by the fact that Pattern 5 asks students to construct two representationsa less and a more abstract one than a given one, rather than to categorize, compare, or explain some aspect of a given object while referring to abstraction. Here are several instructors' answers to the question: "What specific abstraction skills does Pattern 5 measure?" which reflect this explanation [italics were added by the authors]:
The question: "Are abstraction skills assessable?" is part of the discussion on teaching soft CS concepts. The experts' responses, which reflect a constructionism perspective, indicate that they value students' active engagement and creativity in the learning process. One of the instructors suggested that Pattern 5 "... seems too open ended to me for a test questionalthough great for a studio-style discussion."
Based on the experts' answer, this Viewpoint has focused on distinguishing between the creative construction of abstractions as against the passive assessment of abstraction level. Another particularly crucial aspect in abstraction is that of purpose and the consideration of fit-for-purpose in activities such as modeling and problem solving. This includes both the creation of new abstract formalisms and languages to express abstractionsas is created by researchers and language designers, and the use of these to create specific abstractions for specific applications. We believe that abstraction purpose needs to be more directly and explicitly considered and discussed by instructors, students, and practitioners when reflecting on their use of abstraction. In this spirit, one of the experts who responded to the survey suggested to highlight "[...] what the system needs to achieve and what are the criteria for better solution. It will improve the quality of the answers and will provide wider range of good answers." Another one added: "I would recommend to couple it with a specific purpose to guide what kind of aspects that can be left out or needs to be included."
We see this Viewpoint as a starting point of the discussion of this topic and invite the ACM community to join and contribute to the discussion. Such a discussion is especially relevant today when a variety of teaching methods and frameworks are available for students who seek the highest return for their effort investment.
2. Davis, R., Maher, C., and Noddings, N. Introduction: Constructivist views on the teaching and learning of mathematics. In R. Davis, C. Maher, and N. Noddings, Eds. Constructivist Views on the Teaching and Learning of Mathematics. National Council of Teachers of Mathematics, Reston, VA, 1990, 718.
a. From the questionnaire sent to the first group (http://bit.ly/2f9GWgr), two questions were excluded in the subsequent questionnaire sent to the second group (http://bit.ly/2eQJH7N): a) Question 1 was a specific case of question 2; and b) Question 9 that the first group assessed lowest concerning its ability to assess abstraction skills. In addition, to the second questionnaire, we added the open question "Can you suggest an example which best fits Pattern X?"
We would like to thank the experts who answered our survey, invested the time, and shared their thoughts with us.
The Digital Library is published by the Association for Computing Machinery. Copyright © 2016 ACM, Inc.
No entries found