Sign In

Communications of the ACM


In the Year of Disruptive Education

Peter Norvig and Sebastian Thrun

Peter Norvig (left) and Sebastian Thrun's "Introduction to Artificial Intelligence" class is often considered the tipping point for the popularity of massive open online courses.

Credit: Noah Berger / The New York Times / Redux

In retrospect, 2012 may well be remembered as the year when Internet technology enabled the popularity of MOOCsor massive open online coursesa form of disruptive or transformative education currently growing at a meteoric rate.

In just the past several months, top U.S. universities like Stanford, MIT, Harvard, Princeton, University of Michigan, and University of Pennsylvania put some of their classrooms online, giving countless students worldwide access to a higher education never before within their grasp.

One of the most successful online classessome say it was the tipping point for the MOOC movementwas "Introduction to Artificial Intelligence" taught by Peter Norvig, Google's director of research, and Sebastian Thrun, a Google vice president, last fall. More than 160,000 students signed up and 23,000 completed the course.

"There had been decades of various types of online classes," says Norvig. "It is just that now all the technology is coming together to allow online classrooms of that size on a global basis."

Those who may benefit most from MOOCs are the people in the developing world who previously had no access to consistent, quality secondary education at low cost. Indeed, of the students who took Norvig and Thrun's class, one-third were from the U.S., one-third were from Europe, and one-third were from the rest of the world, with the majority located in India and Brazil.

Meanwhile, brand-new platforms such as Coursera, edX, and Udacity are committing themselves to making the best education in the world freely available to anyone who seeks it. There are no qualificationsother than an Internet connection.

But whether they achieve the level of success within the next five years that some educators are predicting depends entirely on how well the new technology-based method of teaching clears some very high hurdles.

"There is a crisis that has to do with increasingly poor access to really high-quality learning opportunities," says Richard Baraniuk, the Victor E. Cameron Professor of Electrical and Computer Engineering at Rice University. Thirteen years ago he launched Connexions, the initiative that offers free open-source textbooks via the Web.

How does one authenticate that the person doing the work is the person who signed up for the class?

"Everyone talks about the rising cost of health care," says Baraniuk, "but it turns out that the cost of education is rising considerably faster. Student debt just crossed $1 trillion in the U.S. and it is the number one form of debt now outside of mortgages. The problem is even worse in developing world. This latest open movement around courses, around textbooks, and around certificates is extremely excitingand has a real chance of becoming a disruptive force in education."

However, three issues have risen to the top of a list of challenges for MOOC educators:

  • Business Models. No one has yet determined how to make the MOOC model self-sustaining. Educators say the current emphasis is just on delivering content; the rest, they say, will work itself out "eventually."
  • Student Evaluations. How does an instructor effectively grade students who are doing homework and labs, writing essays, creating projects, and taking exams online? It's easy when the tests are multiple choice or when the work is peer-graded, but technology still needs to catch up to evaluating the rest.

In the case of Norvig and Thrun's AI class, the tests and quizzes were all multiple choice or fill-in-the-blank and could be graded automatically.

"It takes some effort to convert an open-ended question into a fill-in-the-blank question," recalls Norvig, "but with some practice it can be done, at least for classes in technical subjects. For other, more open-ended subjects, like literature, one would have to use peer-assessment or self-assessment. Scott Klemmer of Stanford has a paper out [see the "Further Reading" list] in which he describes the successful use of self-assessment in the design course he taught."

And, just as in physical classrooms, there is a trust issue: How does one authenticate that the person doing the work is the person who signed up for the class?

  • Certifications. Because most MOOCs are not part of degree programs, what do students receive after completing a course? In most cases, it is a certificate. But is completing a free online course the same as having paid for and completed a course on campus? How potential employers will deal with this branding issue and how they will evaluate the meaning of, say, a certificate that testifies to a student's taking an online Stanford course versus getting a degree from Stanford still needs to be played out.

"Students who put classes on their resumes who have not received official credit or degrees is, indeed, an issue," says Norvig. "I think there will be a variety of accreditation schemes and we'll just have to wait and see how valuable each of them turns out to be."

"I think there's a tremendous value to what we're doing," says Daphne Koller. "We just haven't decided yet how to make it pay for itself."

Meanwhile, Carnegie Mellon University, through its Open Learning Initiative, has been providing online learning for over a decade, with a focus on systematic student assessment.

"We've concentrated more on the science of learning than on delivering courses to the masses," says Jeannette Wing, professor of computer science and department head of Carnegie Mellon's computer science department. "But now, because the media has been focusing so heavily on course delivery, we have been having internal discussions about where we should be positioning Carnegie Mellon in this space. I can't predict what decisions will be made, but I can say this is the number one topic on campusjust as it is the number one topic on many campuses right now, especially in the computer sciences."

Back to Top

Full Speed Ahead

Other universities, however, are rapidly moving forward with delivering classes online, using various delivery platforms.

In mid-April, for example, two Stanford University computer science professors, Daphne Koller and Andrew Ng, launched Coursera, a for-profit "social entrepreneurship company" that partners with a growing number of universitiesincluding Stanford, University of Michigan, Princeton, and University of Pennsylvaniaand presents online many of the universities' courses in the fields of computer science, biology, medicine, the humanities, and more.

"In October [2011], for the very first time, Stanford made available onlineand at no chargethree of its most popular computer science courses," says Koller. "More than 300,000 students registered for 'Introduction to Artificial Intelligence,' 'Introduction to Databases,' and 'Machine Learning.' Seeing the success of that experiment, my colleague, Andrew Ng, and I decided we needed to build on it but couldn't just make it a Stanford-specific effort. Great as Stanford is, there are other universities that offer amazing content currently to only a tiny sliver of the world's population. So we decided we'd spin this out of Stanford and open the opportunities to other top institutions to engage with this amazing experiment on free access to top education for everyone."

While Coursera was initially funded by a $16 million venture-capital grant, Koller and Ng are still debating how to make their efforts sustainable.

"When we launched Coursera in mid-April, we had over 200,000 users," says Koller. "Three months later, we had about 680,000and it continues to grow. So I think there's a tremendous value to what we're doing. We just haven't decided yet how to make it pay for itself."

"Even if the content is to be free, which is something we care a lot about," says Ng, "we believe it's OK to charge a modest amount for certificationperhaps $30 or $50. Given the scale of numbers we have, we think we could come up with a sustainable business model if we decide to go that route."

Another revenue-generating possibility, he says, is to charge prospective employers to facilitate introductions with Coursera students given the fact that, in IT, the cost per recruit in Silicon Valley can run as high as $10,000.

Meanwhile, in May, MIT and Harvard each contributed $30 million to edX, a non-profit joint venture by the two universities that, like Coursera, will offer free online courses to a worldwide audience. edX builds on MITx, a similar project launched by MIT this past December.

"My colleagues and I taught a prototype course on circuits and electronics this spring," says Anant Agarwal, a professor of electrical engineering and computer science at MIT who is now president of edX. "Over 150,000 online students from 152 countries signed up for the course, and we gave out 7,200 certificates. Over 300 students scored a 100."

edX is, in effect, an open source platform for universities worldwide that wish to offer courses online beginning this fall. MIT courses will be branded "MITx," Harvard courses "HarvardX," and other universities coming onboard will brand their courses "[university name]X."

In addition to MIT and Harvard's contributions, Agarwal is hoping to raise money from various foundations and philanthropies, while having similar thoughts to Coursera about the possibility of charging for placement services and/or for completion certificates.

"The question has arisen regarding the usefulness of such certificates," says Agarwal. "We have talked to a number of potential employers and have anecdotal information that, if a student applies for a job and has an edX certificate with a good grade, employers will value the certificate and will count it in the assessment of the job candidate. I mean, MITx and HarvardX courses will be taught by the same professors who teach the on-campus classes, they won't be watered down, and will be MIT- and Harvard-difficult to pass. Why shouldn't the certificates carry value?"

Connexions, one of the very first open-education projects which began in 1999, is unlike Coursera and edX in that its goal is not to offer courses but to build open source textbooks. Based at Rice University, the nonprofit platform enables authors to write textbooks that students can access free of charge. Funding comes from a number of philanthropies, including Rice and the William and Flora Hewlett Foundation.

edX is an open source platform for universities worldwide that wish to offer courses online.

"We currently have millions of users per month and our science, technology, engineering, and math content has been used about 100 million times in the last five years," says Rice's Baraniuk, who is both Connexions' founder and director.

Baraniuk marvels at the number of disruptive education models that have sprung up in just the last few months, but won't try to predict which will be the dominant model.

"All the models are extremely disruptive and are completely at odds with today's status quo," he says. "Which is why they are all so exciting. There's no question that the world of education is going to be fundamentally changed within the next 510 years. I'm convinced of that. I would also predict that the big incumbents are either going to have to transform their institutions or they're going to be rendered irrelevant. The great opportunity here is that we're going to be able to provide everyoneno matter where they are, whether they have money or no moneythe ability to be able to learn and advance themselves in their lives and their careers."

Back to Top

Further Reading

Baraniuk, R.
Open education: One perfect storm yields three revolutions, Telefonica, May 12, 2012.

Bowen, W., Chingos, M., Lack, K., and Nygren, T.
Interactive learning online at public universities: Evidence from randomized trials, Ithaka S+R, May 22, 2012.

Christensen, C., Aaron, S., and Clark, W.
Disruption in education, EduCause Review Online, Jan. 1, 2003.

Welcome to Coursera,, April 18, 2012.

Klemmer, S. and Kulkarni, C.
Learning Design Wisdom by Augmenting Physical Studio Critique With Online Self-Assessment, Stanford University technical report, July 2012.

Back to Top


Paul Hyman is a science and technology writer based in Great Neck, NY.

Back to Top


UF1Figure. Peter Norvig, left, and Sebastian Thrun's "Introduction to Artificial Intelligence" class is often considered the tipping point for the popularity of massive open online courses.

Back to top

©2012 ACM  0001-0782/12/12

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and full citation on the first page. Copyright for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specific permission and/or fee. Request permission to publish from or fax (212) 869-0481.

The Digital Library is published by the Association for Computing Machinery. Copyright © 2012 ACM, Inc.


CACM Administrator

The following letter was published in the Letters to the Editor in the March 2013 CACM (
--CACM Administrator

In the news story "In the Year of Disruptive Education" (Dec. 2012), Paul Hyman explored the challenge of how to award college credit for learning gained from free online courses offered by colleges and universities. The solution may emerge in two ways:

Credit by examination (CBE). Despite already being offered by many colleges as a way to give credit for knowledge, CBE also has a downsidethat students typically pay the same amount of tuition as if they were taking the course and that some schools limit the awarding of credit to those students who complete some period of residency at the school; and

Government-sponsored course recognition. Like many states, Ohio has developed pseudo-course designations, called Career-Technical Assurance Guides, or CTAGs. A CTAG identifies the core content of individual courses commonly offered at colleges, technical schools, and secondary schools; private and public colleges in Ohio can choose to tie one of their courses to a CTAG "virtual" course, in which case students earning credit for a tagged course at one institution carry that credit to all colleges with a similar tagged course.

It may be that states or even countries will develop CBE for virtual courses, and colleges that tag their courses will award college credit regardless of how a student gains proficiency. A college willing to reduce the cost of CBE and waive residency requirements could unilaterally implement it. Governments are usually motivated more than the colleges themselves to offer CBE at the lowest cost possible.

Christine Wolfe
Lancaster, OH

Displaying 1 comment