­Using Advanced Placement Computer Science as a Measure of Broadening Participation

The NSF-funded Broadening Participation in Computing Alliance Georgia Computes! that Barbara Ericson and I are part of is ending, as is the Massachusetts-based Alliance, CAITE (Commonwealth Alliance for IT Education), headed by Rick Adrion and Renee Fall.  The four of us have proposed a new Alliance to the NSF looking at broadening participation at the state-level, but across the United States rather than just in Georgia and Massachusetts.  There are differences between how different states are doing, and there are lessons to be learned and useful practices to transfer between different states.  Barbara has been digging into the data on the Advanced Placement exam in Computer Science Level A (AP CS) to look for data to support that claim.  The results are dramatic and surprising.

In 2011, there were 21,139 AP CS test-takers in the United States.  13,463 (63.6%) of those test-takers passed the exam (earned a 3, 4, or 5).  4,000 of those test-takers were female (19% — not the lowest it’s been), and 2302 of them passed (58% of female test-takers passed).  893 of the test-takers were Black (College Board’s demographic category) which is 4.2% of the overall pool.  283 Black test-takers passed (32%).  

That’s our baseline.  Barbara has been looking at states with similar populations overall or similar populations of Black students to Georgia and Massachusetts.  Here’s some of what she has found:

State	Population	Black Pop (%)	Test-takers/passers (%)	Black t-t (%)	Black pass (%)
GA	9.8M	2.9M (30%)	884/448 (50%)	79 (8%)	17 (22%)
MA	6.5M	0.4M (6%)	750/457 (61%)	33 (4.4%)	8 (24%)
MI	9.8M	1.4M (14%)	389/322 (83%)	2 (0.5%)	0
CA	37.6M	2.2M (5%)	3101/2224 (72%)	29 (0.9%)	15 (51%)
SC	4.6M	1.2M (26%)	214/130 (61%)	14 (6.5%)	7 (50%)
TX	25.6M	2.9M (11%)	3597/1945 (54%)	127 (3.5%)	44 (35%)

Here’s an explanation for the first row of that table.  Georgia, with a population of 9.8 million people (2.9 million Blacks, or 30% of the overall population), had 884 AP CS test-takers in 2011.  448 passed (50% of them).  79 of the test-takers self-reported as Black (8% of all test-takers, much less than the 30% Black in the overall population).  Only 17 of those 79 passed, for a pass rate of 22%.  Continuing down the table:

• Massachusetts has a much smaller Black population (by both raw population and percentage, 6%), but 4.4% of its test-takers are Black, which is closer to the population distribution than in Georgia.  The pass rate is similar.
• Michigan is as large as Georgia, with a Black population between MA and GA, but had only 2 Black test-takers in 2011, neither of which passed.
• California is huge, and has the most test-takers of any state.  The California Black population is comparable to Georgia’s. It had far fewer Black test-takers (and a much small percentage of the overall population of test-takers), but a better pass rate for Blacks (51%).
• South Carolina is the smallest state on the list by population, but is comparable to Georgia in percentage of population that is Black (26%).  It had 14 Black test-takers of its 214 (6.5%), but their pass rate was high (50%).
• Texas has the same size Black population as Georgia, though it’s only 11% of the overall population.  Texas had 127 Black AP CS test-takers in 2011 (3.5% of all test-takers), and 35% of them passed.

In every case, the percentage of Black test-takers is smaller (sometimes much smaller) than percentage of Blacks in the population.  In every case, the pass rates for Black test-takers is much smaller than for the population overall.

More importantly in terms of making progress, there are dramatic differences here.  While still smaller, TX, SC, GA, and MA’s Black test-takers percentages are closer to the overall Black population percentages than in Michigan or California.  CA, SC, and TX have much higher pass-rates for Black test-takers than in GA, MA, or MI.

I am only presenting the Black data here because this is just a blog post.  Barbara has found other surprising facts while combing through these data.  For example, no Hispanic (collapsing the College Board’s “Mexican American,” “Other Hispanic,” and “Puerto Rican” into a “Hispanic” category) female has scored a passing grade on the AP CS exam in Georgia, Michigan, Indiana, South Carolina, or Alabama in the last six years. Only one Hispanic female has passed in Massachusetts in the same time frame. In just 2011, 10 Hispanic females passed in California (for a 20% pass rate) and 40 in Texas (for a 22% pass rate).  The pass rates for different demographic groups are significantly different over the last six years.

Why are there differences?  Georgia and Texas give high school graduation credit for AP CS, but none of the rest of these states do, so whether AP CS “counts” doesn’t seem to have much explanatory power.  While there are certainly criticisms of the AP CS course and exam, which have motivated the creation of a new AP CS course and exam, it’s not obvious how the weaknesses of the course or exam would result in differences across demographic groups or different states, e.g., if the exam isn’t measuring what we’d like, why would the overall population pass at a higher rate than Black or Hispanic test-takers?  More likely is the explanation found in the Margolis et al. book Stuck in the Shallow End that explores how under-represented minorities have a lack of access to computer science classes in Los Angeles schools.  A lack of access can limit participation and skew results.

AP CS is just one way of looking at participation and performance in computing education.  It gives us a tool for shining a spotlight on differences between demographic groups and between states.  It’s the differences between states that gives us hope that we already have some answers that we might be able to share, to improve access to computing education.