UCF’s 30-Year REU Site in Computer Vision

2018 UCF Computer Vision group participants.

The U.S. Government’s National Science Foundation (NSF) started the Research Experiences for Undergraduates (REU) program in the mid-1980s to attract undergraduates in STEM fields into research careers and to consider going to graduate school. The REU program offers grants to universities to plan and oversee research experiences that enrich undergraduate students’ educational experiences. It is believed these experiences encourage the participants to pursue leadership careers in the fields of science, technology, engineering, or mathematics.

The University of Central Florida’s (UCF) Computer Vision group was in the selected first group of sites: only three REU sites in NSF’s Division of Computer and Information Science and Engineering (CISE) were awarded funding in 1987. The grant duration was one year, so continued funding would require a new application for renewal the following year. A few years later, the grant duration was increased to three years, and remarkably for the past 30 years, UCF has kept continuously being funded, by a total of 14 grants. The NSF funded site pays stipends to 10 undergraduates each year who immerse in research and gain useful insight into the prospect of graduate education as an option for their careers.

Figure. The Harris Engineering Center, home of the School of Electrical Engineering and Computer Science at the University of Central Florida, USA.

Three hundred undergraduate researchers (UGRs) from 38 different states and 75 different institutions have participated in this program, and about 80 have published their projects in high-quality venues. Each year, we solicit applications, and we receive well over 150. After a careful interview, we make offers until our 10 positions are filled. Given our successful streak, we try to shed some perspective over our efforts and experiences; see http://crcv.ucf.edu/REU/

Why UCF Has Kept Winning Renewals

It is instructive to contemplate our success and examine our evolution—there are several factors that appear to have contributed independently to our longevity.

Focus: Computer vision. Our site is focused on exciting and appealing topics in computer vision, which facilitate a condensed short course covering key topics, coordination among faculty and graduate students mentors, and interaction and exchanging ideas among UGRs.

Duration: 12 weeks. While the duration of the program is the most controversial aspect of our site with reviewers (because it makes ineligible those students who have fewer weeks available), it is the channel that gives us capacity for all our activities. We use the first two weeks to train UGRs in background material and then have a week of sufficient deliberation for topic selection, and the following nine weeks are for the UGR to conduct research. In contrast to our 12 weeks, many sites offer REU summers to students for as low as 8 weeks.

Immerse the UGR within the graduate students’ lab. Experiencing work in a research laboratory environment with graduate students, has innumerable benefits; the undergraduates see in so many ways the metamorphosis from their current stage to more experienced researcher. We could not have accomplished our goals each year without a large, successful computer vision Ph.D. program. The Ph.D. program offers a scaffolding for the summer REU.

We could not have accomplished our goals each year without a large, successful computer vision Ph.D. program.

We shower the REU students with guidance and caring. Like helicopter parents, we keep the undergraduates feeling attended to, valued, and consequently focused. We expend large amounts of effort each year on our REU activities, and this appears to give each participant so much to take away to the next step of their journey in life.

What we wish our activities will deliver. Our activities during the summer and beyond are intended to provide the UGR with the following quality experiences.

Logistics (payments, housing, travel to/from the site, transportation for various events). We need to ensure everything happens seamlessly, smoothly, and in a timely manner, causing the least amount of stress and distress to the student.
Meeting senior people on the same journey, but quite advanced. UGRs need to meet fully matured researchers who have followed successful career pathways. This must give the UGR the concepts of the possible and achievable levels of success, and the amount of efforts required to achieve them.
Meeting those who are just a little more senior. This escalator through different levels of metamorphosis from young undergraduates into young researchers gives the UGRs the sense of what their next short-term steps need to be.
Meeting peers. These relationships will assist the students to build networks of colleagues and acquaintances that will let them gain knowledge about the variety of career short-term steps that are available. It also provides important insights into their social roles within their peer groups of potential researchers. This is an initiation into the process that will accelerate in graduate school.
Training for understanding the research of others. This involves having the ability to obtain the necessary background to understand research papers, knowing what is needed to be known about those prior research activities, framing the correct questions to ask accomplished researchers, making connections between the research of others and one’s own, accepting guidance from peers, graduate students, mentoring professors, and distinguished researchers.
Training for converting mathematical reasoning into implementable code. This is an important computational skill; the situation presents additional challenges when the mathematics is vague and unspecific in its formulation, and needs additional simplifications or boundary conditions to be implementable. Images and videos in computer vision are always helpful in this context, because they help to provide insight.
Developing persistence. This skill, possibly the most important for research and novel developments, is expected to be built around many successive failures, but with mentored patience, calm deliberation, and the search for clarity about what is not working.
Building presentation confidence—delivery. UGRs should feel comfortable speaking about topics they know about, even when they sometimes are unsure. They should get practice in making verbal mistakes, and being corrected, and learning to prepare themselves for presentations, anticipating audience questions, and being even more additionally prepared.
Building presentation confidence—visual. This is a difficult skill to learn. It is built with lots of practice, and watching the presentations of others, who are peers or more advanced and mature.
Building commitment to complete a task. UGRs learn about making commitments for short terms, they learn about daily commitments, weekly commitments, commitments for the 12 weeks, and they understand how to break daunting tasks into smaller chunks of smaller commitments.
Exposing UGRs to career possibilities in graduate school and industry. UGRs should feel they have good examples of how the career possibilities in graduate school and industry are realizable, and made real. They should have exposure to knowing where they can seek additional help for acquiring knowledge about these pathways.

The activities. At the end of each activity, we list the letters associated with the experiences that were previously described in this Viewpoint.

Immerse the UGR in a research group made up by professor and at least one Ph.D. student (B, C, E, F, G, J).
Initial two-week training in vision techniques and machine learning, a combination of lectures, tutorials, and homework (E, F).
Each year the cohort is presented with more project choices than there are students, the UGRs select their top few choices, and then we begin the task of iteration until there is a stable student to project pairing; during this period there is a lot of contact between each UGR and the possible project groups; stable pairings are achieved by the end of week three (B, C, E, F, G, J).
UGR must do a weekly presentation to a small group consisting of the mentor professor and graduate student and fellow undergraduates mentored by the same professor; the presentation is oral and visual (approximately 15 minutes) (H, I).
Social: Six lunches at Thai/Indian/Buffet restaurants, picnic, graduating Ph.D. dinner, Distinguished Visitor Lunch/dinner, banquet dinner, certificate dinner (B, C, D, G).
Field trips to three companies; during each field trip the company (involved in computer vision work) describes their products and their efforts and each UGR individually presents his/her project work for about 10 minutes (H, I, K).
Graduate school workshop. Sessions are titled “Why Grad School?,” “Why I am Going?,” “How I won an NSF Graduate Fellowship?,” “Maximizing your chance of grad school acceptance,” “Doctoral Fellowships,” presented by the Graduate Deans and award winning students (K)
Distinguished Visitor Colloquium, and Journey Talk, and group meeting where UGRs describe their summer projects (E, H, I, K).
Ph.D. student Thesis Proposal, and Final Defense (C, E, H, I, K).
Attend all-graduate students’ meeting where graduate students present their work (C, E, H, I).
Meet with the co-director each day during the summer for quick report of how overall life is progressing; this acts as release of pressure (from hardware complaints to group dynamic issues, to scheduling adjustments for weekend trips) (A, G, J).
Fall/Spring follow up work with each UGR to assist them to get industry internships, additional REU summers (at other institutions), or apply to permanent industry positions and/or graduate school (K).

At the core of all these activities lies the UGR’s immersion in the graduate environment. The UGR’s research team is formed depending on the project topic. UGRs are given a desk proximal to the graduate student on their team. The graduate student meets with the UGR at different times of the day, as the UGR makes progress or has questions to discuss. Informal short meetings with the faculty mentor occur every one to three days. All these activities lead up to the weekly presentation by the UGR. Additionally, the UGR has opportunities to meet the faculty mentors and graduate students at social events, and the weekly research meeting for the larger graduate student group.

The field of computer vision is rapidly evolving and the REU site has kept pace with the changes.

Our progress during the summer is evaluated by a professional assessment team, which provides us mid-summer feedback allowing us to adjust and adapt our strategies.

Changes Over the Years in Structure and Logistics

Our site has seen changes in many ways over the years. Initially, it offered a year-long REU; the summer was full-time research, while the Fall and Spring components involved part-time research due to full class load. The site was shared with another in-state institution, and half the UGRs were local to one institution while the other half were local to the other, so during the summer the UGRs commuted from home to their institution, and during the Fall and Spring semesters, they were able to take continued computer vision academic courses on site. The yearlong duration allowed the training in background computer vision techniques to spill over many weeks and allowed some room for easy accommodation of project topic changes. The first change came with the program becoming a single site. Additional professors from our institution were added to the team as mentors.

The next change was when the site took participants from other states. This necessitated the move to on-campus housing, the transition to focus on the summer months, the need for logistics for managing the processing of the selected out-of-state students, and widespread advertising, recruitment, and interviewing procedures.

The focus on the summer months has led to annual review of the short summer background training, inclusion of and proper scheduling of the vast variety of activities. The pre-summer activities of planning the research topics in advance has also taken greater attention.

The recent change of adding new faculty to the Center for Research in Computer Vision (CRCV) has permitted flexibility in how the 10 students are subgrouped for their weekly reporting meetings, how they are mentored each day, and has opened up new research areas within computer vision and machine learning.

Changes in Content

The field of computer vision is rapidly evolving and the REU site has kept pace with the changes. Machine learning approaches started to appear in computer vision, as they were able to contribute to object recognition solutions during the mid-1990s. Approaches such as neural networks, boosting, and support vector machines were actively competing for ascendance during the early 2000s. The advent of Deep Learning in the 2010s has slowly gained acceptance as the dominant paradigm in computer vision, and today, research in computer vision must start with a quick study of deep learning approaches and novices must acquire competence in running practical experiments with large data sets in deep learning implementation environments. Consequently, our own short course now has a strong emphasis on environments like Keras, Tensorflow, and a shift to teaching Python (away from MatLab).

Sample Topics. Looking at the topics pursued over the past 30 years indicates the student projects have evolved with the growth of computer vision. Over the six five-year periods, two topics per period are listed here.

1987–1992: Object Recognition using Multiple Sensors; Detection and Representation of Events in Motion Trajectories.
1992–1997: Visual Lipreading Using Eigensequences; Screening Mammogram Images for Abnormalities.
1997–2002: Person-on-Person Violence Detection in Video Data; Flame Recognition in Video.
2002–2007: A Vision-Based System for a UGV to Handle a Road Intersection; Scale Space Based Grammar for Hand Detection.
2007–2012: Optimizing One-Shot Recognition with Micro-Set Learning; Part-based Multiple-Person Tracking with Partial Occlusion Handling.
2012–2017: How to Take a Good Selfie?, GIS-Assisted Object Detection and Geo-spatial Localization.

Broadening Participation

UCF’s REU has a strong commitment to broaden participation among underrepresented groups. Of the 50 participating UGRs in the past 5 years, 23 are female, and 10 of the 27 males are African-American or Hispanic. This diversity in the cohort contributes to increasing the pipeline of students pursuing graduate careers.

Conclusion

After 30 years (and approximately 300 students), some patterns have emerged. Approximately half the students have proceeded to graduate school. Many of the participants have proceeded to leadership positions in their professions: becoming faculty members, starting their own companies, and rising to managerial positions in Fortune 500 Technology companies. Details about student successes are provided in the booklet at http://crcv.ucf.edu/REU/Book-let_071117.pdf

UCF’s CRCV has seen many benefits from its cultivated REU strength. UGRs have provided an opportunity to explore research directions, to develop mentoring skills among faculty (older and newer) and graduate students. CRCV-trained UGRs have populated graduate programs around the nation. Our models of evaluation and attentiveness have allowed for best practices to be tested and employed. The commitment of time, effort, and resources is expected to continue into future decades.