Research and Advances
Artificial Intelligence and Machine Learning

Knowledge Management with Patterns

Developing techniques to improve the process of converting information to knowledge.
Posted
  1. Introduction
  2. Patterns as Knowledge Media
  3. Patterns in the World
  4. Fitting a Model of Knowledge
  5. Tacit to Tacit Knowledge
  6. Tacit to Explicit Knowledge
  7. Explicit to Explicit Knowledge
  8. Explicit to Tacit Knowledge
  9. Stories, Metaphors, and Analogies
  10. Experiences: Patterns and Knowledge
  11. Time to Start Reflecting
  12. References
  13. Authors
  14. Figures
  15. Tables

If knowledge is power, then an awful lot of power is going to waste in the world. But not for want of products, tools, or techniques: for some time, many useful and practical knowledge management techniques have been obscured in the froth of successive marketing waves. As the meaning and applicability of knowledge management becomes clearer, we are finding that the tools and expertise needed to usefully manage critical business and project knowledge are within our grasp.

Knowledge management—its models, products, tools and techniques—is beyond the point at which consumers can perceive substance over market-induced expectation. The fact that an organization’s ability to marshal and exploit its knowledge as a potent tool in sustaining its competitiveness was never in doubt. From business-driven extreme programming teams to knowledge-worker cells, the potential of proactively managing the implicit and explicit knowledge that emerges from work and interaction has been widely recognized. But as knowledge management has progressively distinguished itself from the traditional fields of data and information management, its proponents have learned that knowledge is a difficult and challenging thing to manage. The discipline of knowledge management reflects this difficulty in its broad philosophical base and the range of approaches, from technological ones based on information management infrastructure through to sociotechnical ones that measure and develop knowledge capital [7, 12].

Knowledge is a personal thing. We take the view that knowledge comes from a situated, personal and internalized experience of the world. Far from being reducible to information schemas or templates, knowledge emerges from the combination of personal skill, expertise, perception, history, and constructive memory. In its most basic form, knowledge is internalized know-how, the ability to tacitly know—in any particular context—what needs doing and how it should be done. It is developed whenever the individual applies his or her concepts, interpretations, and assumed conceptual relationships to the set of commonly observed phenomena in the world.

So while we can communicate data in the form of events and messages (information) formed using syntactic and semantic conventions, knowledge is a far more elusive property that does not quantify easily, especially in our increasingly complex, heterogeneous work and personal environments. What one person perceives as knowledge may be bland information in another’s eyes. So if we cannot reify knowledge objectively in databases or build tools that handle knowledge directly, a promising alternative is to capture and manage information in such a way that it supports how knowledge is made explicit and transferred. In doing so, we focus on devising techniques to improve the process where information is converted to knowledge. Here, we explore the use of patterns [1, 3] as a powerful mechanism for this information-to-knowledge conversion.

Back to Top

Patterns as Knowledge Media

To illustrate the potential of using patterns for knowledge management, consider the pattern Sell Now, Resource Later (see the table here), an example of a pattern from a pattern language that might describe the corporate culture of a consulting or professional services firm. It illustrates some of the key features that differentiate the patterns approach from other knowledge management or design transfer techniques. We will contrast some of these later, but for now, look through the syntax and structure of Sell Now, Resource Later as described in the table to consider the intent of the pattern. It describes in no uncertain terms the company’s resource scheduling strategy and operational policy, and does this in an open way, allowing the reader to deconstruct this knowledge to its component parts and then reconstruct the solution’s rationale. The elements of the pattern (context, problem, forces, solution, rationale, resulting context, related patterns) allow its readers to draw together a judgment as to its usefulness, appropriateness, and applicability in different contexts. Sell Now, Resource Later makes a tacit optimistic resource allocation strategy for the consulting organization explicit and accessible.

Organizational benefits. For the organization, this pattern’s strategic and operational value is clear. Supervisors find great value in being able to convey useful resource allocation strategies to first-time managers or new employees in a form that is informal and easy to comprehend. As a technique for expressing knowledge, the process of identifying patterns serves to make explicit and validate tacit knowledge, so that it can be subsequently reapplied and combined with other pieces of knowledge. Over time, this extends the boundaries of the organization’s knowledge, while encouraging ineffective ‘knowledge’ to be identified. As a tool for cultural reinforcement, the pattern attaches a label (Sell Now, Resource Later) to an organization’s strategy that has been proven and should be followed. The creation of a company-specific vocabulary of pattern names and the association of each with a desired behavior provides a systematic counterpoint to the management of company culture than ad hoc or opportunistic ones. The applicability of patterns as a technique for knowledge management rests on two key characteristics—knowledge capture and knowledge communication, and it is these benefits that we believe patterns can bring to the discipline of knowledge management.

Back to Top

Patterns in the World

Because of the personal and perceptual dimensions of knowledge, the highest return-on-investment in knowledge management effort is frequently realized by concentrating on:

  • Presenting and structuring information in such a way that it supports reification of knowledge and knowledge-enacting behaviors in people rather than artifacts;
  • Building conceptual frameworks that allow the sharing of ideas and abstractions between people working in the same conceptual spaces; and
  • Identifying individuals or groups who possess critical knowledge, and then supporting these people appropriately with the right techniques and resources.

Patterns are an excellent candidate technique to meet these requirements—they provide a flexible but structured informational template, they use simple but expressive everyday statements, they are deliberate and instructive while providing transparent reasoning, and they can convey business structure, cultural values, and strategies at many levels of an organization simultaneously.

The impact of patterns. The pattern languages and catalogs of Gamma [4], Buschmann [2], Rising [9] and other authors have altered the way software developers speak and think about design. Patterns like Observer, Visitor, and Iterator [4] have lifted the level of abstraction in the design discourse, ensuring rapid exchange and transfer of proven solutions between designers. The application of patterns to the problems of structuring software teams is less well-known but equally well-regarded—Coplien and Harrison’s organizational pattern language encapsulates extensively researched knowledge of how the world’s most productive and healthy software teams operate.

The object-oriented world and other domains where patterns have been successfully applied [10] demonstrate the capture, transfer, and management of design knowledge. The next context for pattern adoption—organizations and global business communities—presents a vast opportunity. But to achieve the transition from a system design practice to an organizational and business practice, patterns need to fit the ways that organizational knowledge is understood, valued, and managed. To be acceptable, useful, and usable, patterns and the knowledge-sharing they enable must fit comfortably with organizational and business models of knowledge.

Back to Top

Fitting a Model of Knowledge

The concepts of tacit and explicit knowledge [8] are central to knowledge management theory and practice. Nonaka and Takeuchi [7] present a model for the interplay between these two phases of knowledge, as depicted in the figure on the left.

Back to Top

Tacit to Tacit Knowledge

Tacit knowledge underpins classic expert behavior—the uncharted, unpredictable combination of skill, information, expertise, experience, and judgment that work together in a seemingly ad hoc way in the minds and actions of an expert surgeon, lawyer, executive, architect, designer, navigator, writer, consultant, or programmer. It is common for such experts to be unable to explain their methods or rationalize their actions. Tacit knowledge of this kind is both critical to expert performance and impossible to predictably manage, and is typically only passed on by craft-like mentoring of apprentices over many years. Some professions—for example, law, medicine, and architecture—prefer to keep it this way, and have a long history of knowledge preservation and even protectionism. Others, like the engineering disciplines, are more altruistic about knowledge exchange.

Back to Top

Tacit to Explicit Knowledge

Mining patterns. A valuable insight derived from knowledge management is to make tacit knowledge explicit, allowing it to be analyzed and critiqued, learned, shared, and combined. Several aspects of the patterns approach support this phase transition. To begin, the pattern is expressed in written language, where it may be described at various levels of granularity and generality, depending on how well the pattern author can express the pattern’s context, problem, forces, and solution. The writing, review, and refinement of patterns follows a model of incremental or piecemeal growth—patterns under construction typically evolve as fragments of tacit knowledge expressed in pattern form. Pattern authors consistently report this is not an easy thing to do, because it involves explicating what is unsaid and assumed to be explicit. Recognizing this, the software design pattern community uses a process that supports the tacit-explicit transition and provides an exemplar for knowledge mining in other contexts. The process is as follows:

  • A practitioner identifies a potential pattern—it should not have been described before, it should be a result of the author’s direct experience, and it should solve a specific and recurring problem with a proven solution. Because the problem is recurring, it should also have been experienced by others, and in a similar way the solution should have experienced previous use (though it may not have been recognized as a pattern at that time);
  • The author drafts the pattern(s)—this usually involves concerted introspection as the author grapples with the problem being solved, the forces in the problem context that demarcate the problem and make it unique, and the clear expression of the known solution;
  • Each drafted pattern is related to other patterns, both in the author’s pattern language and in other languages. This phase of pattern mining is possibly the most difficult, and is rarely done well during the first attempt.

Once a knowledge fragment has been expressed as a pattern, it has a name that allows it to be referenced, a description of the context that clarifies when the pattern may be applied, a statement of the problem and forces that exist in the problem context, the solution itself, and the consequences of applying the pattern’s solution—all of which add precision to the pattern’s encapsulated knowledge. While the solution is an important part of a pattern, it is these other components that situate and orient the solution, distinguishing the pattern from a simple expression of a recommendation, a heuristic, or a business rule.

Distilling tacit knowledge. Software pattern authors are encouraged to draw on the help of the patterns community as they write. Since 1994, pattern-writing conferences have been held annually in the U.S., Germany, Australia, Japan, South America, and Scandanavia for the sole purpose of reviewing patterns. When a pattern author submits a draft, the conference committee allocates a “shepherd”—a more experienced pattern author—to work closely with the author in refining the draft patterns. What follows is an exercise in literary criticism, where the shepherd guides but does not control or dictate the emerging work. Once the draft has been shepherded, typically over a six- to eight-week period, the shepherd makes a decision in conjunction with a program committee member to accept the work into a writer’s workshop, a group review protocol similar to that used for many years by poets.

At the conference, the writer’s workshop inverts the conventions of conference paper presentations. The author initially becomes a “fly on the wall”: not speaking a word, but listening as the workshop attendees, many of whom are authors themselves, discuss their understanding of the pattern. This allows the author a rare opportunity to hear how others have understood and interpreted the pattern. Misunderstandings become immediately clear during this time. The workshop concludes when the author reenters the circle and has the opportunity to request clarifications of the reviewers.

Many authors find a writer’s workshop to be a rich and rewarding experience that provides very high- quality review and feedback. This intensive process can result in a distilled and highly evolved expression of an important piece of domain knowledge.

Back to Top

Explicit to Explicit Knowledge

Synthesizing knowledge. New explicit knowledge may be created by combining existing pieces of explicit knowledge. While combination is important, it does not typically yield the powerful steps of insight that accompany articulation (tacit-explicit conversion). The cost of externalizing fragments of tacit knowledge in pattern form begins to be repaid when the pattern is published and used widely, either in conventional book form or in a Web-based repository. Patterns can be combined with other patterns or with other knowledge representations and models, such as business processes, narratives, and personal development activities. The Related Patterns section of Sell Now, Resource Later demon-strates its links to other relevant patterns, situating itself in the context of previous work and collective knowledge. Because pattern theory has developed to accommodate linking and combining solutions through the pattern language, it is tailored toward supporting combination in a way that conventional documentation does not.

Back to Top

Explicit to Tacit Knowledge

New tacit knowledge. The knowledge spiral completes when the refined or synthesized explicit knowledge becomes internalized back into tacit form. This occurs as individual patterns become familiar in an organizational context, as experienced and novice business people alike discuss their strategies, designs, concepts, and actions with a liberal sprinkling of pattern names. The best example of this can be found in the design dialogues of OO software designers, in which the names of architectural and design patterns become tacitly and fluently interspersed.

In this phase, pattern languages become significant as people apply patterns tacitly. In fact, the full benefits of patterns are not realized until a set of related patterns are applied together, in concert. A pattern language allows an individual, team, or organization to navigate through complex, multidimensional problems by using patterns that link to each other via their initial and final contexts. This allows both the isolated local problem (resolved by an individual pattern) and the interrelated strategic problem (resolved by the pattern language) to be addressed at the same time. A well-constructed pattern language that represents a knowledge whole is far greater than the sum of its constituent parts, the individual patterns.

Back to Top

Stories, Metaphors, and Analogies

The use of stories for sharing experiences and disseminating lessons learned is another well-known knowledge management practice. These can vary from unstructured stories to structured narrative techniques specifically intended to build organizational learning such as learning histories [5]. Metaphors and analogies [7] are also used to share concepts. Metaphors combine ideas into expressive imagery, while analogies are useful in helping us form new ideas by comparing them with familiar ones.

Patterns complement these approaches. The key differentiator of patterns is their self-descriptive and reflective nature, and the ability to generalize instances of experience into distilled core knowledge that cuts across specific uses and individual instances. While stories, metaphors, and analogies are concrete examples of knowledge in action, patterns can capture and generalize the essence of this knowledge. These complementary roles are recognized in some pattern templates that include a section for known uses, the examples of how the pattern was used in a specific instance, perhaps phrased as story, metaphor, or analogy at the author’s discretion.

Back to Top

Experiences: Patterns and Knowledge

Observers consistently report advantages of using and mining patterns in organizations—patterns effectively share information, highlight design rationale, capture competencies and sound practice, and provide a standardized design vocabulary.

The limitations of patterns in knowledge explication reported to date relate to the difficulty of introducing change, generating support for patterns, writing good patterns, indexing patterns to increase knowledge about the applicability of patterns, and quantifying the impact of patterns. Patterns also face the difficulties associated with introducing change into organizations [6]. These problems are consistent with other knowledge management initiatives, and to a large extent are a consequence of the inherently ‘wicked problem’ of knowledge [11]. Patterns are certainly no panacea. While patterns make good projects better, they cannot resurrect bad projects or substitute for effective and timely project management.

The sheer number of patterns produced, both within a given organization and across domains and organizations, may create problems of assimilation. Some efforts [9] address this, but the challenge is significant, given that patterns can cross multiple domains, and people bring unique perspectives to the knowledge (patterns) they seek.


Observers consistently report advantages of using and mining patterns in organizations—patterns effectively share information, highlight design rationale, capture competencies and sound practice, and provide a standardized design vocabulary.


While the concept of a pattern is reasonably straightforward and expressed in a readable fashion, writing good patterns is surprisingly difficult. Significant effort and time need to be invested in prospective pattern authors to bring them to a level of authoring maturity that ensures useful and effective patterns. Again, there seems to be a confluence here with the general problem of understanding and expressing knowledge.

Back to Top

Time to Start Reflecting

Patterns have great potential to be a rich knowledge extraction technique that is intuitive, straightforward, and offers a good return on investment over medium- and long-term periods. The success of patterns to date has largely occurred within design-intensive software and technology domains. We have some evidence that patterns are useful in more general business contexts, and this article has explored the basis of our beliefs that patterns can be a valuable general-purpose knowledge management tool.

Patterns are in their infancy, but the future looks bright. It is likely that patterns will gain a foothold in other design-intensive industries in the same way as has happened in OO software engineering. Such expansion would represent a form of linear growth, from one industrial domain to another. But the really big opportunity for patterns and pattern languages, an opportunity for exponential growth, comes to light when we perceive patterns as an important part of a larger challenge—that of valuing, exploiting, and managing the knowledge available all around us, in both tacit and explicit forms, ripe for the picking.

Back to Top

Back to Top

Back to Top

Figures

UF1 Figure. How patterns fit with the spiral of knowledge.

Back to Top

Tables

UT1 Table. A pattern that captures a consulting company’s resourcing policy.

Back to top

    1. Alexander, C. A Pattern Language. Oxford University Press, New York, 1977.

    2. Buschmann, F. and Meunier, R. Patterns of Software Architecture: A System of Patterns. Addison-Wesley, Reading, MA., 1995.

    3. Coplien, J.O. Software Patterns. Lucent Technologies, Bell Labs Innovations, New York, 1996.

    4. Gamma, E., Helm, R., Johnson, R. and Vlissides, J. Design Patterns: Elements of Reusable Object-Oriented Software Architecture. Addison Wesley, Reading, MA, 1995.

    5. Kleiner, A. and Roth, G. How to make experience your company's best teacher. Harvard Business Review 75, 5 (Sept.–Oct. 1997).

    6. Manns, M.L. and Rising, L. Fear Less and Other Patterns for Introducing New Ideas into Organizations. Addison-Wesley, Reading, MA, 2003.

    7. Nonaka, I. and Takeuchi, H. The Knowledge-Creating Company. Oxford University Press, 1995.

    8. Polanyi, M. Tacit Dimension. Peter Smith, 1983.

    9. Rising, L. The Pattern Almanac 2000. Addison-Wesley, Reading, MA, 2000.

    10. Rising, L., Ed. The Patterns Handbook: Techniques, Strategies, and Applications. SIGS Books and Multimedia, 1998.

    11. Rittel, H.J. and Weber, M.M. Planning problems are wicked problems. In N. Cross, Ed. Developments in Design Methodology, Wiley, 1984, 135–144.

    12. Sveiby, K.E. The New Organizational Wealth: Managing and Measuring Knowledge-based Assets. Berrett-Koehler Publishers, San Francisco, 1997.

Join the Discussion (0)

Become a Member or Sign In to Post a Comment

The Latest from CACM

Shape the Future of Computing

ACM encourages its members to take a direct hand in shaping the future of the association. There are more ways than ever to get involved.

Get Involved

Communications of the ACM (CACM) is now a fully Open Access publication.

By opening CACM to the world, we hope to increase engagement among the broader computer science community and encourage non-members to discover the rich resources ACM has to offer.

Learn More