In today’s software development industry, jobs have become more cognitively complex and require workers who are more collaborative and creative in their problem-solving techniques.14 Employees also must be able to combine diverse specializations rather than just having routine knowledge in one domain.22 While the “hard” technical skills associated with programming remain a prerequisite for new hires, the industry also wants software developers who can readily demonstrate a range of so-called “soft” skills, including the capacity to communicate clearly, facilitate an open and inclusive workplace environment, and demonstrate the resiliency and flexibility to work on a range of tasks.24
Key Insights
- While coding bootcamps were once touted as an “alternative” post-secondary educational option, they are now increasingly tapping into unemployed (or under-employed) college graduates, despite their college degree.
- In surveys, coding bootcamp students and undergrad CS students didn’t perceive their learning preferences as different, each indicating they liked hands-on activities and collaborative environments. Later, however, undergrads expressed greater discontent with group work and indicated collaboration often only came at the end of their programs.
- Given the difference in program duration, undergrads unsurprisingly reported being exposed to a wider range of computing activities and concepts; but much “depends on the professor” in terms of inculcating AE opportunities.
Our own past research4 interviewing software industry hiring managers indicates that discerning such soft skills among new hires is an overwhelming priority across companies. The industry hiring managers and directors we interviewed over the past two years stated that while the capacity to code is a necessity for employment, these managers actually spend the vast majority of their recruitment time assessing a candidate’s soft skills, as these suggest the presence of adaptive expertise (AE) and the candidate’s potential for persistence and continual learning on the job.4 What was also intriguing to us in discussion with a wide range of hiring managers was their expressed willingness to consider graduates from alternative educational settings—in particular, so-called “coding bootcamps”—alongside more traditional hires from undergraduate computer science (CS) programs.4 While there is no single representative model of a coding bootcamp, these intense training programs extend, on average,14 weeks in duration, cost approximately $12,000, and emphasize teaching the programming skills that employers look for from new software developer hires (particularly front-end programming) while also enabling their graduates to grasp the most essential aspects of coding.6 Much of this expressed willingness to hire codecamp graduates stemmed directly back to hiring managers’ perceptions that what boot-camp students may lack in rigorous CS knowledge is counterbalanced with greater work experience and the interpersonal and intrapersonal skills to join a wider team while remaining resilient in the face of unexpected challenges.
This, of course, represented only one party’s perspective. Moving forward with our research, we were especially interested in exploring student perspectives from both bootcamps and undergraduate CS programs to better understand to what extent students felt prepared by their respective educational programs. We focused on three research questions: Who are these different programs attracting as learners? How do these students perceive themselves as learners? To what extent do students (at both camp and college) self-report having the opportunity to develop components of adaptive expertise? More specifically, through student perspectives, this research reports how each educational environment promotes guided self-learning, knowledge-transfer, and collaboration on a range of tasks.
Background
While the U.S. education system has redoubled its efforts to promote STEM learning across U.S. classrooms, a recent American Enterprise Institute domestic policy report21 aptly points out the STEM education gap is not simply a deficit in the hard cognitive skills associated with science and engineering but also in the soft interpersonal and intrapersonal skills linked with effective communication and collaboration and adaptability. This point has been supported by recent studies examining the requisite skills of effective software developers.13,18 However, whereas the general public differentiates between hard and soft skill classification, researchers Hata-no and Inagaki11 made the distinction between “routine” and “adaptive” expertise as a more meaningful distinction. Individuals who have mastered a designated set of routines have gained routine expertise. They will continue to learn throughout their lifetime, Hatano and Inagaki point out, but will often only apply their new knowledge in a manner that makes the existing procedures/routines more efficient. On the other hand, an individual with adaptive expertise (AE) will utilize the knowledge they obtain and apply it to new, innovative procedures and unexpected problems. According to Bransford,3 “adaptive expertise involves habits of mind, attitudes, and ways of thinking and organizing one’s knowledge that are different from routine expertise and that take time to develop.” While routine experts possess strong procedural knowledge, adaptive experts are likewise endowed with a strong conceptual knowledge base, allowing them to utilize their understanding to adapt previous mental models and frameworks to new situations.11 There have been differing views on whether or not elements of AE are learned skills versus personal attributes that individuals either have or do not have upon programmatic entry. Some research indicates that AE can be developed and practiced, but in order to do this, learners must be exposed to metacognitive practices alongside cognitive practices.12 In particular, research indicates that the development of adaptive expertise can be enhanced through guided self-learning, knowledge-transfer, and collaboration on a range of tasks.
Managers actually spend the vast majority of their recruitment time assessing a candidate’s soft skills, as these suggest the presence of adaptive expertise and the candidate’s potential for persistence and continual learning on the job.
Developing students’ adaptive expertise through metacognitive activities and collaboration. Research indicates that guided self-learning practices may enhance the development of AE by promoting metacognition through the students’ mindful processing and abstraction in order to apply to new problem sets and innovative solutions. These practices include video reviewing of self or others on the job;2 peer coaching;15 engaging with colleagues (in the field) in collaborative activities;1 meetings with and/or teaching alongside mentors;4 engaging in reflective conversation that draws upon shared real world situations;16 and error based learning/error management training.8
Developing students’ adaptive expertise through knowledge-transfer. Research also indicates the development of AE can be further fortified by providing students with opportunities for transfer of knowledge and adaptability. Effective techniques include providing variation of tasks and projects during practice; placing coursework in the field through internships and capstone coursework;19 scaffolding by starting with lower variability in tasks in the beginning in order to allow the learner to comprehend concept and abstract general rules prior to introducing higher variability in tasks;25 and helping link previous knowledge to new concepts/ practice sets.10
Based on the existing literature on inculcating AE, we decided to organize and analyze our student data according to two primary categories and 12 sub-indicators:
- First, there are Metacognitive and Collaborative Activities, which include the following activities: Video reviewing; peer coaching; engaging with colleagues in collaborative and joint planning, teaching and assessment activities; meetings with and/or teaching alongside mentors; self-reflecting or engaging in reflective conversation that draws on shared real world situations; assisting learners in being open about changing their current way of thinking about problem sets; examining learner products/portfolios that follow instruction; and error based learning/error management training.
- Secondly, the Transfer of Knowledge and Learner Adaptability entails providing opportunities to vary tasks on the job; placing coursework with field practice; deliberate scaffolding of tasks into composite parts; and helping link previous knowledge to new concepts/practice sets.
Methods
In our study, the goal was to analyze the degree to which current students and recent graduates from both coding bootcamps and undergraduate CS programs articulate their participation in, appreciation of, and learning from these designated activities associated with metacognition, knowledge transfer, learner adaptability.
Participants. In our investigation, we interviewed a total of 49 students from four different four-year college CS programs (27 students, 12 females, 15 males), as well as from a total of three coding boot camps (22 students, 9 females, 13 males). All programs were located in medium-sized (130K–150K population) southeastern U.S. cities. The sample was one of convenience, and students were recruited through direct visits to bootcamp and college classrooms, as well as through flyers delivered to instructors. Some of the bootcamp students were recruited through email messages posted via the LinkedIn website.
Data collection and analysis. From April 2017 through February 2018, as part of a larger collaborative study, the researchers conducted 49 one-on-one interviews with participating students. Given that participants were unlikely to be familiar with the designations of routine and adaptive expertise, interview questions focused on the more immediate and mundane elements of admission processes; the skills and knowledge they believed they had obtained in each training ground; and, the teaching methods/learning environments characteristic of their respective education programs. It is important to note that interviews were semi-structured in nature; while they adhered to the three categories identified earlier, questions (particularly follow-up questions) built upon specific responses related to participants’ perception of themselves as learners and the skills and knowledge they felt they themselves gained (or lacked) from coursework, Prior to the interviews, students were asked to complete a 6-point Likert survey (6: strongly agree to 1: strongly disagree) focusing on how they perceived themselves as learners based on Fischer and Peterson’s7 survey constructs (Were they open to new ideas/perspectives? Did they try multiple solutions when tackling a problem? Did they prefer to stick with a known solution as opposed to exploring other options?). All focus groups and interviews were subsequently transcribed and qualitatively analyzed using Dedoose software.
In terms of the subsequent data organization and analysis, we the-matically coded the transcribed interviews first individually and then collaboratively.5 With this more in-depth analysis, we paid special attention to what degree coded utterances from participants potentially related to our two primary categories (Meta-cognitive and Collaborative Activities and Transfer of Knowledge and Learner Adaptability) and 12 sub-criteria representing the specific activities that potentially facilitate AE among learners. Multiple examples of utterances were reviewed between the two researchers in order to determine agreement of categorization. Table 1 provides examples of student responses and the criterion under which it was coded.
Table 1. Coding schema with sample participant utterances.
Results
Profile of students. The profile of the students and educational settings are listed in Table 2.
Table 2. Participant/institutional profile.
These demographics correspond to Course Report’s6 national profile of bootcamp students, who are significantly older (mean age of 29 years) and more experienced (six years of work experience, on average) than four-year CS undergraduates. However, it is interesting to note that Course Report also indicates the typical attendee of a bootcamp has never formally worked as a programmer. Yet, 20 out of the 22 (91%) bootcamp participants in this study indicated they had some form of prior training in database management, website development, programming, software development, game creation, CS theory, and/or computer programming languages. It is also interesting how many females participated in this study when considering the national average of women in CS majors has decreased from 30% in 1984 to 18% in 2014.9 We expect this is partly reflective of the higher percentage of female undergraduates at two of the participating undergraduate CS programs, which reported having approximately 35% and 40% female enrollments.
With regards to national trends for undergraduate CS students, the results support a recent report from the National Academies of Sciences, Engineering, and Medicine,20 which points to a growing number of non-majors who take computing courses. In fact, 14 of the 27 undergraduate participants from this study entered the program as undecided-without a CS major in mind, and, notedly, all of these same students were initially leaning toward degrees in math, science, and/or engineering.
Students’ self-perception of themselves as learners. The survey results, which were measured on a 6-point Likert scale (6: strongly agree to 1: strongly disagree), revealed that, on average, both undergraduate students and bootcamp students viewed themselves as learners who are open to new ways of looking at things and are willing to change their views when presented with new facts and evidence. Figure 1 presents the average scores on select pre-survey questions from both undergraduates and bootcamp students.
Figure 1. Average student Likert rating on survey responses.
Metacognitive activities. As illustrated in Figure 2, the interviews with undergraduate students revealed their undergraduate setting provided opportunities for metacognitive activities that have been considered useful practices in helping develop metacognition.
Figure 2. Undergraduate opportunities for metacognitive activities and collaboration.
Out of the 27 undergraduate students in this study, 19 commented on colleague collaboration activities being implemented in their CS classrooms. This was mostly reflected in peer/group projects during which students had to plan, design, and implement projects and/or tasks as a group. However, it is interesting to note that while students reported collaboration a priority in coursework, many of the undergraduate students did not view group collaboration in a wholly positive manner. For example, during follow-up interviews, one male student in his early 20s stated: “I don’t really like group projects—I guess I like having all of the responsibility instead of depending on someone else … Sometimes not everyone carries their weight, which isn’t fair for everyone.” A female student at a separate university seconded his thoughts: “I do think that my peers would generally prefer less group projects. Group projects are not generally thought of positively just because you so often get bad eggs in your groups, and some professors don’t let you choose your team. Maybe in companies you’re not able to choose your teams, but if you’re working at a startup, you are absolutely able to choose who you start a company with, and I think such an important skill to acquire is your ability to identify who is a good worker and who you would mesh with and be a good business partner with.” Another student, although stating she doesn’t enjoy group projects, was able to reflect on the positives as indicated by stating “I’m not an advocate for group work in college just because if you have bad group partners, they affect your grade. I pay a lot of money to go to school. But they do emphasize teamwork, and I think maybe half of my classes included a large project with a team. And that does definitely increase the ability for people to communicate and work together, especially people who are more introverted …” In some cases, undergraduate students were not permitted opportunities to collaborate. A 19-year-old male indicated that collaboration is not a programmatic expectation but varies based on the disposition of instructors. “(S)ome professors don’t allow collaboration,” he states, “but some professors do. It just depends on if we’re allowed to talk about the assignments outside of class.”
As mentioned earlier, collaborating with mentors in the field may contribute to the development of AE. Undergraduate participants indicated they encountered these mentors over their junior and/or senior year of their respective programs through class presentations provided by the visiting industry representatives regarding the interview process and the workplace environment (if it occurred at all). Some instructional settings, according to these students, implemented peer coaching as a method by which to increase feedback about instruction and curriculum. One cited example referred to the potential of reviewing coded projects as a class in order to have peers comment on errors and/or suggest alternative solutions. In this study though, only nine undergraduate CS students (one-third of the total undergraduate participants) commented on the presence of peer-coaching within their respective programs. And eight of these nine students noted that peer review was largely informal in nature and was contingent on whether the professor allowed for students to talk to each other about projects during class time.
With regards to receiving feedback from the instructor, 19 of the 27 (70%) undergraduates spoke about their professors examining their products and/or portfolios following instruction. This allowed students to receive feedback regarding their progress in the course, their performance on a task, and/or their conceptual knowledge. Slightly more than half (11 of the 19) of these students indicated that the design of the course and the feedback they received allowed for them to discover their errors and learn from their mistakes. For example, a female student in her early 20s stated that “a couple of weeks ago, we had to make a program that sorted a list of random numbers. And I couldn’t get it to work just right … But the time was out so I had to send it in. She sent it back explaining to me how I could tweak it and she said I could send it back in. So, I looked at what she had said in her comment and it allowed me to understand what I was doing wrong.” However, not all of the university students viewed the feedback process in a positive light. According to one student, whether or not one received timely opportunities to correct errors “depends mostly on the professor. Some allow you to redo assignments … They’ll give you feedback on it and you can try again. Others, you just have to get the grade you get.” Sometimes, undergraduate students do not receive any consistent feedback in certain courses, with two students reporting that their instructors did not formally grade them until the final weeks of the semester.
Finally, in terms of metacognitive activities on the undergraduate level, several students evaluated whether or not their learning environment promotes self-reflection on the learning process. Sixteen indicated the design of the course, the professor’s instruction, and/or design of the curriculum provided them with an opportunity for self-reflection on how they learned, an attribute that has been noted in research as beneficial in the development of metacognition. One female senior in her early 20s stated the program promoted “general abstract thinking … just learning to remove myself from the implementation only and looking more at how I can think about this instead of just how I can code this.” Some 41% of these undergraduates likewise mentioned their respective programs provided opportunities for them to question their thinking and challenged them to be open about changing their current way of thinking about problem sets.
In terms of metacognitive activities among coding camps, the interviews with the 22 bootcamp research participants also revealed their training ground provided opportunities characteristic of practices used to develop metacognition. This is illustrated in Figure 3.
Figure 3. Bootcamp opportunities for metacognitive activities.
Nineteen of the bootcamp students commented on colleague collaboration activities being implemented in their bootcamp training grounds. Similar to the undergraduate students, these students indicated this mostly occurred in peer/group projects during which students had to plan, design, and implement projects and/or tasks as a group or a pair. Notably, unlike the undergraduate students, the bootcamp students did not list any negative reasons for peer/group work. With regards to mentors and collaboration, only 18% of these students indicated that industry mentors collaborated with them; and just as with the undergraduate participants, all of the bootcamp students indicated that the mentors spoke with their class about the interview process and what the industry workplace environment was like.
In the bootcamp environment, 19 students indicated their training included deliberate peer coaching as a means of receiving feedback on projects and clarification of concepts. This is interesting in that a much smaller number of undergraduates in this study either did not comment on peer coaching or indicated that it did not occur. One such example came from a bootcamp graduate, who is now employed in healthcare software development: “My instructor would give us an assignment on Monday. And then Tuesday morning from 9 AM to 9:30 AM, he would call on a few different students, and they would come to the front of the class and plug their computer up to the projector and we would look at each other’s code. You’re looking at code that you wrote the night before in front of 10 to 15 other people and he’s telling you what you did wrong and what you did right and what you could improve on, and so would the other students.” In addition, four students stated the instructor provided in-person or video demos prior to starting a project so that the students were able to visualize their task at hand. For example, a male student in his mid-30s stated: “most of the assignments that we did were demo-ed first. So, we’d be like, ‘Okay we’re gonna do this.’ This is how you do it and we would just watch, and then we would go back and do it all again. But we would, like, do it along with him, with the instructor. Then, the third time would be no instructor just try to do it yourself.”
With regards to receiving feedback from the instructor, nearly all participating bootcamp students (approximately 95%) spoke about their instructors reviewing their products and/or portfolios following instruction. In some of these cases, it was in the form of active learning using the flipped classroom model: “We did the flipped classroom model where we would do some sort of reading ahead of time and then the next day we would have activities so that we could show that we’ve grasped the concept and kinda fill in any knowledge gaps that we may have.”
A commensurate percentage of boot-camp students (63.6%) mentioned that feedback encouraged learning through errors. “It’s one of the things they tell you. Go in, break the code, once you break it you know what not to do, and so now we know that doesn’t work, so let’s try something else.” In addition, 27.3% of bootcamp students in this study explicitly mentioned their instructors and/or program assisted them in being open about changing their current way of thinking about a problem set. For example, one student stated, “they would never be like, ‘this is an error and this is a problem.’ They would just be like ‘I see that you did it this way, did you consider doing it this way instead.’ Things of that nature.”
Transfer of knowledge and learner adaptability. The responses of undergraduate students revealed the college setting provided opportunities for transfer of knowledge and adaptability. This is illustrated in Figure 4.
Figure 4. Opportunities for transfer of knowledge and adaptability in the undergraduate classroom.
Nineteen undergraduate students (70%) in this study indicated they were exposed to a variation of tasks and projects during practice. This does not necessarily take place during the same course. Instead, it is the exposure to multiple coursework practices and instruction over their college career.
Meanwhile, 25 of these students (93%) mentioned that their programs have included placing coursework in the field, such as internships, class/individual projects with real world projects, or capstone projects. The students were very positive regarding this aspect of their program. For example, one student stated “I would say the college does a very good job with their final two, kind of like capstone-ish courses with the software engineering and then the software engineering practicum. I feel like that course in its own gives you the most experience to what it’s like to work for, to work on a project in a group. That’s your first real exposure to a full-fledged product top to bottom. So, I feel like that course is probably the most important course I’ve taken here.”
Undergraduate students also spoke about how their professors and program helped them link previous knowledge to new concepts/practice sets and that this has also helped in the area of self-reflection. For example, a senior stated “going to a different programming language is really just a different wording and syntax of the same concept. As you learn one and get good at one or get good at two and you go to different ones and you start to see the same things reappearing that are very similar to the things you’ve done before. Then once you start making those connections between the subsequent ones and the first ones that you learned, that’s when it really starts to click for me.” Several of these students reflected their classes or activities, outside of the CS program, such as music, sports, and foreign languages, provided them with skills and knowledge, which they need in their CS task work.
Finally, seven undergraduate students mentioned that scaffolding has been used in order to help them transfer knowledge by starting with smaller tasks in the beginning in order to allow for them to comprehend the concepts and/or abstract general rules, followed by higher variability in tasks later. For example, a female senior student stated: “in Operating Systems, I had to design a shell, and we sort of did that piece by piece, adding in functionality as we went. At first, we could only run one command. Then we could chain commands.”
The interviews with bootcamp students revealed their setting likewise provided opportunities for transfer of knowledge and adaptability. This is illustrated in Figure 5.
Figure 5. Opportunities for transfer of knowledge and adaptability in the bootcamp setting.
Eleven bootcamp students in this study indicated they have been exposed to a variation of tasks and projects during practice. The other 50% indicated the training group focused on a single language or technology with more routine practices (that is, practice the same types of tasks throughout the program). Thus, this seems to be a matter of differences between different bootcamps.
Nine students of these bootcamp students mentioned their programs have included placing coursework in the field, such as class/individual tasks reflecting real world projects. As an example, one bootcamp student noted “[T]here’s 10 weeks of class time where you’re learning new subjects and, then, the final two weeks is when you choose a project and you write a full stack Web application. That’s the first time you really get the chance to put everything together. That was really cool because we used absolutely everything we learned without exception during that final two weeks.”
Bootcamp students (36%)) also spoke about how their instructors and program helped them link previous knowledge to new concepts/practice sets and that this has also helped in the area of self-reflection. Many times, it was also linked to employability. For example, one learner stated that “[t]hey kind of show you how what you already know can help you with what you need to learn. So how already knowing Java will help me in the future for C+ or C++ or whatever else I may need to learn in the future—showing how to kind of mesh the two so that you can quickly pick up new languages if you have to learn a new language when you get to your first job.”
Finally, five of these bootcamp students mentioned that scaffolding has been used in order to help them transfer knowledge to other tasks and/or concepts. In some cases, these scaffolding exercises also helped students build a portfolio. For example, one student stated that “as far as the assignments go, we were given assignments … early on in the course and as the course progressed we were given kind of more complex assignments that maybe built off of itself and we’d start a project on Monday and it would be get the bare bones done, and then on Tuesday, we’d implement a feature and so on and so forth and kind of build-up different individual portfolio pieces.”
Discussion
Returning to our initial research questions, the first research question about who such programs attract proved unsurprising. In terms of entering students, age and work experience were the leading differentiators between these southeastern undergraduate and bootcamp students in this study, and the student profiles closely correspond to Course Report’s national annual statistics (29 years of age, six-year work experience).
In terms of the second research question about how undergraduates and bootcamp students perceive themselves as learners, we were surprised to find little difference between the two groups. In surveys, college students and the more mature code camp students both reported themselves as hands-on learners who enjoyed working in teams on a range of tasks. Of course, some of this self-reported survey data was undercut by individual interview responses with these same students where a number of undergraduates (but interestingly no code camp students) expressed reservations about the actual productivity of working in groups.
The third research question investigating to what extent each learning environment inculcates adaptive expertise proved most telling. Clearly the short and intensive timeline associated with bootcamps meant their students had to be ready to “drop in” and work closely with each other and their instructor from day one. This is evident with the fact that virtually no bootcamp participant pushed back on the value of group work and 86% indicated that peer coaching was built into their coursework. A smaller number of undergraduates spoke on peer coaching within their program. Bootcamp students were also more likely to report receiving timely feedback from their instructor (96% compared to 70% of undergraduate students). Of course, this finding must be tempered by the fact that the majority of code camp students reported having only one to two instructors over the course of their respective programs, while CS undergraduates had eight plus instructors and consistently indicated that the nature of classroom collaboration and feedback was highly contingent on the individual instructor (that is, a regular response from undergraduates during interviews was “it depends on the professor”). Yet this range of instructors corresponds to a wider range of activities on the undergraduate level with 74% of college students reporting exposure to a variation of tasks and projects; whereas, it depended on the bootcamp chosen with regards to whether the student was exposed to a variety of tasks versus routine practices of the same task. Here, bootcamp students also reported significantly less opportunities for knowledge transfer, likely due to the relative brevity of their programs.
While it is not surprising that program duration and student age play no small role in distinguishing between college and camp, it is important to note that based on the survey data, the students themselves did not perceive themselves any differently as hands-on, collaborative learners. The peer-to-peer collaboration and immediacy of feedback more widely characteristic of coding camps (according to student interviews) is a learning environment that students from both groups prefer. Although the number of recruited participants was small, this could have implications for CS departments nationally, where CS1 classes have swollen to capacity and instructors are increasingly finding themselves using the initial coursework as a means to sort and “weed out” students from the major. Meanwhile, the “hands on” collaborative coursework characteristic of CS capstones only comes at the end of four-year programs. This pedagogical approach has wider implications around inclusivity and diversity in the CS field, which colleagues are currently investigating17 as CS persists as being inordinately populated by Caucasian and Asian males.
The alternative environment of coding bootcamps offers a new stream into the CS field, and, as a recent paper entitled “Betting on Bootcamps”23 indicated, such camps could very well be a potential disruptor in higher education. Yet, as evident with our study, with some bootcamps recruiting and admitting students already with a college degree, to what degree these camps will truly become an alternative to college seems unlikely. As an alternative to graduate education, particularly master’s degrees, their future appears much more secure. Future studies involving a larger sample size in various geographic regions could help increase knowledge regarding these two training settings.
Acknowledgment. This article was supported through a National Science Foundation (NSF) Core Research and Development award (#1561705) to the first author. The views expressed are solely those of the authors.
Figure. Watch the authors discuss this work in the exclusive Communications video. https://cacm.acm.org/videos/becoming-an-adaptive-expert
Join the Discussion (0)
Become a Member or Sign In to Post a Comment