Sign In

Communications of the ACM

The profession of IT

The Idea Idea


View as: Print Mobile App ACM Digital Library Full Text (PDF) In the Digital Edition Share: Send by email Share on reddit Share on StumbleUpon Share on Hacker News Share on Tweeter Share on Facebook
stepped path made of light bulbs, illustration

Credit: Dan Page

Ideas pervade our professional work. We borrow them, we apply them, we solve problems with them, we create new ones, and we try to foster more of them in our teams. We express what we are doing by referring to the ideas behind our work. We hope that some of our ideas become innovations by being adopted.

We also believe that ideas launch innovations and that innovations fail for want of good ideas. As soon as we become aware of a successful innovation, we automatically look backward to try to understand where the idea for it originated. When we see failures, we immediately look to beef up our processes for stimulating imagination and creativity.

For several years I have been puzzling over why it seems that our innovation adoption rates are low even though our idea production rates are high. The overall success rate of innovation initiatives in business is around 4%.2 Yet many businesses report they have too many ideas and waste precious time and resources struggling to select the ones most likely to succeed and then work them through to adoption.1 We are idea rich, selection baffled, and adoption poor.

I have come to wonder if the connection between ideas and adoption is much weaker than we believe. What if innovation is not ideas generated, but practices adopted? What if entrepreneurs, rather than inventors, are the real innovators?

I do not pretend to have any sort of final answer to these questions. I would love to see them discussed and debated. Yes-answers would have important impacts on our professional work. For example, we would worry less about stimulating creativity and imagination, and more about developing our skills at getting our communities to adopt new practices. We would approach design not as an expression of ideas but as the framework for new practices.


An idea that changes no one's behavior is only an invention, not an innovation.


My objective in this column is to examine the ways in which ideas and practices source innovation. I will propose a position about these hypotheses, and I will draw conclusions about our professional work.

Back to Top

How Does It WorkGetting Ideas Adopted?

In discussing innovations we talk about two distinct aspects: ideas and change of practice. We often connect the two with the word "adoption"for example, when we get an idea adopted into practice.

Research is often seen as a major source of ideas and, by implication, of innovation. Dennis Tsichritizis (then the chairman of GMD in Germany) wrote in 1997 that he thought the notion of research being the source is mistaken.5 He noted four ways in which new practices can be adopted:

  • Disseminate research: publicize the idea and exhort people to practice it;
  • Create tools: build and distribute easy tools that draw people into the practice;
  • Education and training: directly teach people the new practice; and
  • Business: start new businesses to offer the new practice.

Only the first of these is explicitly concerned with generating ideas. The others, which occupy much more attention and budgets, can succeed without telling any of the adopters what ideas are behind the new practice.

So are ideas as important in the generation of innovations as we seem to think? How should we divide our effort between generating ideas and working for adoption?

If you think ideas are the key to innovation, you will put your effort into generating, analyzing, selecting, and publicizing ideas through papers, articles, books, blogs, and other venues. This accounts for proposals to improve creativity, imagination, borrowing, and recombinationall well-researched idea generators.

On the other hand, if you think adopting new practice is the key to innovation, you will put your effort into selling people on the value of doing the new practice, building credibility that it works, teaching people how to do it, furnishing tools to help them do it, providing leadership, helping people past obstacles and resistances they encounter while trying to do it, and building and executing on teams to get it done.

Bob Metcalfe, the inventor of Ethernet, calls these cases respectively the flowers and the weeds.3 He says the elegant and alluring conceptual model and mathematical analysis published in 1976 is the flower of Ethernet;4 but he spent 10 years "in the weeds" selling Ethernets to skeptical business leaders. His effort on the weeds far outweighed his effort on the flowers. Ethernet began as an idea but Metcalfe's deep conviction that Ethernet would bring value was the real driver that kept him going him through the long work of adoption.

Let's take a look at the case for each approach.

Back to Top

The Case for Ideas First

Many theories of innovation concentrate on how ideas flow from sources to products and services in the marketplace. They tell us we will get more innovation by stimulating more idea generation. They are accompanied by fascinating stories of how ideas were discovered.

The idea idea has an attractive logic. Many innovations clearly start with someone's idea (vision). If we trace back from an innovation, we can almost always find someone who mentioned the idea first. Even if the idea inventor did not participate in the work of adoption, that person often gets credit for launching the process of refining and developing the idea as it moves toward adoption.

The popular notion of "meritocracy of ideas" supports this theory. It says that innovation is the process of the best ideas winning in a competition with many ideas.

Perhaps the most popular conception of idea-source innovation is the pipeline model. According to this model, ideas invented by researchers and visionaries flow through a pipeline with stages for prototyping, product development, manufacturing, and marketing en route to becoming products or services in the marketplace. The flow in the pipeline is driven by economic pull from the marketplace and proactive push from promoters and investors. This conception was made popular by Vannevar Bush, who, as the first U.S. presidential science advisor in 1945, laid out a plan for government sponsorship of research that was eventually implemented as the National Science Foundation. Bush thought idea generation is the main driver of innovation and should be encouraged and supported by government policy. The same notion pervades the organization of many businessesseparate divisions for research, development, manufacturing, and marketing. However, many businesses today try to overcome the pipeline's inefficiencies with cross-function teams and with SCRUM and agile development.

Diffusion is another conception, almost as popular as the pipeline. First proposed by Everett Rogers in 1962, this model assumes that innovation is a change of a social system brought about by communicating the innovation idea through the normal channels of the social community, leading individual members to decide to adopt. The diffusion model does not assume a pipeline; it emphasizes the adoption decisions made by community members, and the factors that bring them to that decision. Companies that subscribe to this model organize heavily around marketing and public relations.

The diffusion and pipeline models share a common feature: they both put idea generation as the source. They differ on how the ideas move from the source to the market.

Back to Top

The Case for Practices First

Newer theories of innovation concentrate on adoption of new practice in a community. An idea that changes no one's behavior is only an invention, not an innovation.

The Prime Innovation Pattern, which is part of the new theory, says that innovators bring about changes of practice in their communities through five kinds of activities.2 The innovator (1) senses a disharmony, (2) sticks with it like a puzzle until it can be articulated, (3) discovers how the current common sense of the community generates the disharmony, (4) proposes a new common sense that would resolve the disharmony, and (5) commits to a path of action that leads to that outcome. The innovator's goal is to change the community's practice from what they are already doing to something new that resolves the disharmony. The eight practices framework is the innovator's skill set for moving through this pattern.2

In this view, there is much more to accomplishing innovation than generating and pursuing ideas. There is a lot of work to be in touch with the deep concerns of the community, and then to guide them toward new thinking and actions.

Many innovations clearly start as practices. Someone starts doing something differently, often as an improvisation. When the new way is superior to the old, others imitate it; the practice spreads. After a while, someone builds tools to facilitate the practice and enable even more people to engage with it. People in the community experience this as responding to a need or an opportunity to make things better for others.


Many ideas are therefore afterthoughts to explain innovations that have already happened.


An example is blogging, which started spontaneously when someone started keeping a daily journal on a Web page (a Web log) in the early 1980s. Others imitated the practice and it spread. Various people created blogging software to help others publish their blogs and enable newcomers to start blogs. Still others built tracking services to help people find blogs of interest to them. Although some names have been proposed as the "first bloggers," the truth is that no one really knows who was the first and no one seems to care.

The current wave of "apps" and "app stores" fits this pattern. Steve Jobs of Apple saw an opportunity to sell small pieces of software that customize the iPhone. Many others have followed suit. App developers are a thriving and growing industry, offering over one million apps by mid-2010. Most of them are responding spontaneously to perceived consumer needs and opportunities, and are not engaging in an idea-generation process.

Only later, in moments of reflection, do spontaneous innovators try to explain the new practice. They call their explanation "the idea behind the practice," even though the doers of the practice had no such idea. Many ideas are therefore afterthoughts to explain innovations that have already happened.

Back to Top

A Combined Approach

When described side by side as above, we see that idea and practice are both essential. However, that does not mean ideas always come first. It is often more fruitful to experiment first with trial practices and later distill descriptions of the best practices into ideas.

Our best answer to the original question at this point is: find a balance between cultivating ideas and cultivating practices. What is a good balance?

Birkinshaw and his colleagues figure most companies are already plenty good enough at generating new ideas; they suggest the bottlenecks to adoption are elsewhere and consume as much as 95% of the work on innovation.1 Metcalfe spent one year perfecting his idea and 10 years selling it.3

This makes the iceberg a useful analogy. The visible top part (approximately 10%) is analogous to the set of ideas describing an innovation. The invisible submerged part (approximately 90%) is analogous to the practices constituting the innovation. The practices keep the ideas afloat. The iceberg analogy has a ratio of idea work to adoption work close to Metcalfe's experience.

Too strict adherence to the idea side of innovation leads us to focus too much on the easiest part of the innovation process and to defer work on adoption until after the idea is perfected. This imbalance is called the Innovation Myth2 or the Eureka Myth.1 The imbalance leads us to underestimate the work of adoption and it compounds the selection problem. Escaping this myth will require a radical reorganization of approaches to innovation. But the escape is well worth it because it also escapes the dreadful waste of the 96% failure rate.

You can put this to work at the personal level immediately. Beware the idea idea. Put 10% of your effort into explaining the value and principles of your ideas, and 90% into fostering the new practices you advocate for your community. Get going on the work of adoption from the very beginning.

Back to Top

References

1. Birkinshaw, J., Bouquet, C., and Barsoux, J.-L. The 5 myths of innovation. MIT Sloan Management Review 52, 2 (Winter 2011), 4250.

2. Denning, P. and Dunham, R. The Innovator's Way. MIT Press, 2010.

3. Metcalfe, R. Invention is a flower; innovation is a weed. MIT Technology Review (Nov. 1999). www.technologyreview.com/web/11994/?a=f

4. Metcalfe, R. and D. Boggs. Ethernet: Distributed packet switching for local computer networks. Commun. ACM 19, 7 (July 1976), 395404; http://doi.acm.org/10.1145/360248.360253

5. Tsichritzis, D. The dynamics of innovation. In Beyond Calculation, P. Denning and B. Metcalfe, Eds. Copernicus Books, 1997, 259265.

Back to Top

Author

Peter J. Denning (pjd@nps.edu) is Distinguished Professor of Computer Science and Director of the Cebrowski Institute for information innovation at the Naval Postgraduate School in Monterey, CA, is Editor of ACM Ubiquity, and is a past president of ACM.


Copyright held by author.

The Digital Library is published by the Association for Computing Machinery. Copyright © 2012 ACM, Inc.


Comments


Anonymous

Hi!
I think that your interesting article missed some reasons for this awkward situation of high idea production rate and low innovation adoption rate.
The first one is economical issue. I have an old friend of mine with whom we were colleagues in Academy of Sciences decades ago and since that time I consider him a genius in programming. For many years he generated ideas, put them into programming products and the results were amazing. In addition to his ability for generating new ideas, he has the highest programming level and the ability to work much more than most other people. Then, many years ago, he started to work for Microsoft, and in private discussion three years ago he mentioned that now the marketing department of that company decides what is needed and what he has to work on. Do you think that the brightest people in computer world work in marketing departments and have to decide the future? Microsoft invites the best programmers from around the world (at least they declare them to be the best), fit them into already existing system of product development, and put the marketing department as the main deciding body. Knowing this, I am not surprised at all that for many years in a row I don't see anything really revolutionary and really amazing from Microsoft; marketing department can evaluate only what is already on the market.
The second reason is linked to WHERE and HOW those ideas are produced. A lot of new ideas are generated as a result of research work in numerous universities. In recent review by one of the MIT professors (not to break the CACM comments' rules, I am not mentioning his name here, but I can send it to you through a private letter, if you want) I was really amazed to read such a phrase: It should be noted that the essence of research is ideas, not implementation. To which I had to explain that the essence of any research is the cognition of the world. Ideas are the source of any research but definitely not an essence. And then I had to add that if he wanted to say that ideas are the main thing in any research and not implementation, then he was fundamentally wrong. It was right at the time of ancient Greece, but since Galileo Galilei each scientific statement must be proved by experiments and the correctness of each statement must be demonstrated in the way that can be duplicated and rechecked by everyone else. In programming it definitely means that each statement must be demonstrated in a working program. If MIT professor (and he is considered to be a bright professor) have such views on ideas and implementation, then why do you expect any outcome of his ideas? And he publishes a lot of papers every year. I don't think that his view on ideas and implementation is exceptional.
Thank you for your excellent article.
Sergey Andreyev


Anonymous

My argument is simply that generating ideas and putting them on the table is not sufficient to achieve the adoption of those ideas into practice. If your purpose, say as a university researcher, is to generate ideas and publish them in the research literature, then the work of adoption will be of no concern to you. All this means is that you have no right to expect that your work will ultimately become a standard practice within the field. If you do want your ideas to be adopted, you have more work cut out for you.

Many ideas are generated by university researchers. Many more are generated by businesses looking to make new offers in the marketplace. The biggest challenge for business is to select which of the great ideas on their table they have the time and resources to pursue. As I said, we are idea rich, selection challenged, and adoption poor.

Peter Denning


The account that made this comment no longer exists.

Your ideas deserve a wider audience: for example if true they undermine much of the justification for the monopolies granted by the patent system.

It's a pity ACM policy means this is premium subscription content.


CACM Administrator

The following letter was published in the Letters to the Editor in the June 2012 CACM (http://cacm.acm.org/magazines/2012/6/149799).
--CACM Administrator

Peter J. Denning's Viewpoint "The Idea Idea" (Mar. 2012) resonated with me due to discussions I had at Hewlett-Packard on how best to lead process improvement and innovation. New practices capable of delivering at the bleeding edge must be scouted and deployed/replicated. It is usually only after such an effort that a practice gains a theoretical basis, a two-step sequence that goes like this:

* It works (replicate/deploy/leverage); and
* This is why it works (institutionalize/educate).

During my stint, 19972000, as a member of the Software Engineering Process Group (SEPG) at Hewlett-Packard India Software Operations facilitating process improvement across all business lines, SEPG and line management concluded the best ideas originate as new practices in working teams. Teams were therefore encouraged to submit their own best practices to help identify such ideas, which were then analyzed for soundness and selected for deployment and replication. The teams were encouraged to maintain documentation using Deming/Shewart's Plan, Do, Check, Act (PDCA) at the project level and the Initiating Diagnosing Establishing Acting Leveraging (IDEAL) model at the SEPG level so they could quantify and demonstrate improvement. (PDCA and IDEAL both overlap the Prime Innovation Pattern outlined by Denning.) Selected best practices were then engineered as repeat-able processes and institutionalized through associated training and tools. What we wished to guard against was innovation for its own sake.

Ideas generated by research teams often entail too much risk to deploy as-is in industry without first undergoing feasibility studies. For one thing, they often do not scale well, as when, say, a toy programming language is used to make a (perhaps valid) point in a research paper. For another, they may be solutions looking for a problem or inventions without a contextyet.

Mallika Chellappa
Bangalore, India


Displaying all 4 comments