Studying programmer behavior experimentally: the problems of proper methodology
The application of behavioral or psychological techniques to the evaluation of programming languages and techniques is an approach which has found increased applicability over the past decade. In order to use this approach successfully, investigators must pay close attention to methodological issues, both in order to insure the generalizability of their findings and to defend the quality of their work to researchers in other fields. Three major areas of methodological concern, the selection of subjects, materials, and measures, are reviewed. The first two of these areas continue to present major difficulties for this type of research.