We got away without software process for quite a long time, and produced a lot of software, some of it pretty good, too. The fact is, process is often strongly resisted by the people who are expected to use it. It is often implemented less than optimally in many establishments. There are certainly many improvements that could be made to both our approach to software process and the process our approach produces. But is all the hesitancy, reservations, resistance, and outright rebellion simply the work of recidivists? Is it just stepping on the cowboy programmers’ inalienable right to program whatever and however the heck they please? What is it they object to? "Your process is stifling my creativity!" is the cry. Do they have a point?
The cornerstone of the software process faith is that process engenders repeatability. "Repeatable" is even the name of the first level above bare existence in the SEI model. But what is repeatability? And is it valuable?
This scenario may help explain: a couple of years ago I was facilitating a management workshop for a systems engineering group. There were 14 senior managers and their VP in the room. These managers were, in their engineering youth, the people responsible for making the first cellular telephone systems operational. A more "can do" group I cannot imagine. With the exception of one person, they did not have a process-oriented bone in their collective and corporate body. Their idea of process was "just do it." The personality profiling we did with them during the workshop supported this view. Collectively they had both the right approach and the right personalities to break open a new market, make things happen, fly in the face of accepted wisdom, accomplish things that had never been done, do things others said couldn’t be done—boldly go where no person had gone before. But process? You could have tortured these people and they would not have admitted that process was a good idea. The solitary process-oriented individual was a voice crying in the wilderness.
I was giving a pitch on the SEI/CMM model, and it received the predictable response. Jim was one of the more challenging of a challenging group.
"SEI Level 2 is ‘repeatable’, right?" Jim pointed out. "Well, look at our title. We’re the Advanced Products Group. Advanced. Everything we build is new. Different. We don’t want to repeat anything. Repeatability is of no use to us!"
He had a point.
Luckily there was a counterpoint—the implementation of the SEI model and the processes used must support the business practice and goal. Their business practice and goal was to be creative and innovative. Therefore, at SEI Level 2, they should be able to be repeatably creative and innovative. This point kept them quiet for a while, though they were by no means converts to process, and remain unconvinced to this day. However, it got me thinking. The nature of process for a creative group producing something for the first time should be different than for, say, a product group producing the fifth in a series of system upgrades.
Types of Teams
There are basically four types of project teams, and each of them has specific needs and goals. Since what each team is trying to accomplish is different, one would presume the nature of its process must be different if it is to be successful. One size probably does not fit all. The four types of teams are: tactical (its job is to do something); problem-solving (its job is to fix something); creative (its job is to build something); learning (its job is to learn something).
We can map these on to the Five Orders of Ignorance ("The Business of Software Communications, Oct. 2000, p. 17).
Tactical team. Key issue: follow a plan; key need: defined roles and processes (Orders of Ignorance covered—0OI).
Problem-solving team. Key issue: solve a problem; key need: defined roles and trust (Orders of Ignorance covered—0OI, 1OI).
Creative team. Key issue: build something new; key need: freedom from restrictions (Orders of Ignorance covered—1OI, 2OI).
Learning team. Key issue: construct a model of understanding; key need: consistent, shared models and language (Orders of Ignorance covered—2OI, 3OI).
A tactical team, working to execute a plan, requires—above all else—well-defined roles and processes for the team members. This is because they deal mostly with known quantities to produce known or expected results. For a highly tactical project or team, there are no unknowns. In the software world, porting operations tend to fall into this area. Activities such as configuration management within more creative projects also tend to be highly tactical. The issue is not discovering new knowledge, but applying old knowledge. There is a pressing need or advantage not to do things in a different way. This is almost 100% 0th Order Ignorance (0OI). We are dealing only with answers.
A problem-solving team also uses 0OI, but tends to work more in the area of 1st Order Ignorance (1OI). If the team doesn’t have the answer then it generally has the question; it is a matter of finding the answer, and implementing the solution. Perhaps some of its work tends to be slightly creative (such as how it gets the questions answered), but mostly it will be data collection and application. The problem-solving team shares with the tactical team the need for good role definition, but it is not as strong. The reason is that the answers to the questions might well change the roles and processes, so too-rigid process definition might be harmful. An element of trust is needed, particularly that each team member will solve the problem assigned to him or her. A software project tracking down and debugging network problems might be a good example of this kind of team.
The creative team may require explicit lack of process. Well-defined process enables known activities, but usually restricts unknown activities. For truly creative activities, that is, activities creating something never created before, we honestly cannot have a process as it is generally viewed. It is not that there isn’t a process for creation, but such a process cannot be rigidly defined. Certainly, whatever process is imposed should allow a lot of latitude in application. That software processes often do not allow this "creative space" is one of the major criticisms of process as it is usually implemented. One of the definitive creative teams in the computer world was the team that created the original IBM PC. Its need for freedom from IBM’s corporate culture (and process) led the team leader, Don Estridge, to relocate the team as far away from Armonk as he could [2]. Even in the most prosaic software development, there is always some element of creativity that requires this freedom. The most creative of knowledge acquisition always addresses the 2nd Order Ignorance (2OI), as we find out things we didn’t know that we didn’t know.
The learning team is a research group. The output is some intellectual model of whatever has been, is being, or must be, learned. The key need for this is the construction of shared mental models and cognitive languages between team members. Constructing these models has strong elements of creativity, but does not necessarily produce a product; often the output is a better idea of what might work. For true research groups, most of the effort is spent in finding approaches or processes that will unveil what is not known. Therefore, this kind of team is focusing on the 2OI3OI levels.
The challenge for most software teams is that all types of processing are necessary. Usually a software team is not just one of these. It may be a learning team in gathering requirements, experimenting with different ways of interacting with the customer or environment. Then it may be a creative team in designing the system, tactical in implementing it, and problem-solving in debugging it. This is why defining software processes is so difficult, and why rigid processes do not work well.
The approach to defining process must take these issues into account. The peril of not doing so is that the process will force the wrong answer.
Putting This All Together
We can summarize these points as follows:
- We can only have a well-defined process for those (0OI and 1OI) things that are well defined and we have done before.
- We cannot have a well-defined process for things (2OI and 3OI) we do not know how to do.
- Defined process only really supports tactical and problem-solving activities, and may be harmful in creative and learning activities.
- Almost all software projects have elements of tactical implementation that can benefit from well-defined development procedures, creativity, and learning hampered by restrictive process.
- Non-restrictive processes (guidelines) are necessarily vague and therefore are not much use.
- The ideal process applies at the wholly predictable and mechanical level.
- The purpose of process is to liberate us from the mundane.
Therefore, we can assert that the purpose of process is to take those things we already know how to do successfully, and to package them so they are entirely predictable and invariably successful. The process serves to free up our brain from reprocessing those things that really do not need to be reprocessed.
What Does This Leave Us Doing?
The freeing of brain cycles from the repetition of the more predictable and mechanistic activities of our projects gives us the bandwidth to work on the unknown aspects of building systems. These are the unstructured, unpredictable, undefined, variable, new, and, yes, creative activities for which we cannot have a well-defined process. In OI terms, the job of dumb process is to take care of 0OI and some of 1OI. Our job as clever and inventive developers is to take care of 2OI and 3OI. The fact that software developers have a high growth need strength or need for personal development (read: learn new things) [1] is evidence of this.
To be free to create, of course, is exactly what engineers and developers want. The fact that they often view process as preventing them from doing this is probably the most legitimate criticism of process as it is often done.
Join the Discussion (0)
Become a Member or Sign In to Post a Comment