Research and Advances
Computing Applications

The Whitewater Process: Software Product Development in Small IT Businesses

Small software development companies need it to ensure they stay on course and are able to respond to the market's ebbs and flows.
Posted
  1. Introduction
  2. Development Processes
  3. Product Development Analysis
  4. Conclusion
  5. References
  6. Authors
  7. Figures
  8. Sidebar: How the Study Was Done

At the core of the engine that drives historical economic growth are the small businesses that employ 50% of all workers in the U.S. [6]. According to the U.S. Department of Labor, 80% of IT workers are in small companies, and the economy’s IT sector is expected to grow 68% in the period 2002–2012 [5]. These companies use nontraditional development methods, yet few scholars have sought to examine or even identify their characteristic operations. Here, we examine the software development process and methodologies used at three small IT businesses (SITBs) in Florida.

It would be convenient to view a small business simply as a large business with fewer people, but a business’s size directly affects its structure [3]. A shortage of slack resources (such as software engineers) is a common SITB problem. Just one unexpected new customer can require doubling the development staff, and just one delayed sale can leave the same staff idle. A small portfolio of active projects can lead to volatile staff requirements and uncertain revenue streams. One way to level demand on the staff is to develop software products, instead of relying solely on custom development contracts. Small businesses can leverage limited development staff by building a product, then deploying it repeatedly for multiple customers.

Embracing a product-oriented development strategy also brings new challenges to a development organization. The most notable is the difficulty in determining product requirements. Products must be designed for the needs of an amorphous market, but it is more difficult to define the needs of a market than it is to gather requirements for a particular customer. Furthermore, a product must not only fulfill the market’s needs, it must also be favorable compared to competitive products.

Many techniques used by large businesses to probe the market are resource-intensive. For example, current users may be enticed to provide feedback via user-group meetings held at attractive resort destinations. Prospective users may be reached via elaborate trade show presentations where key prospects are invited to confidential briefings. Large companies may employ resident experts who track the competition, as well as those who follow technology trends and forecast changing market requirements.

SITBs lack the economies of scale to support these market-probe techniques. In large companies the cost of a trade-show booth can be amortized across many customers. Small companies may be unable to afford the market-planning activities used by their larger counterparts. Similarly, economies of scale in large companies help justify staffing full-time marketing and technology specialists. In contrast, small companies have fewer people to cover the same range of activities. Small-company employees find themselves stretched to handle a broader range of issues and lack time to develop in-depth expertise in individual issue areas.

The implications of these factors can be far reaching. With only a small marketing staff, the development and operations team receives less guidance on product requirements. With fewer specialists, the SITB loses a source of standards and guidelines that would help developers manage unfamiliar issues. As a result, many SITBs have relatively streamlined structures compared to large companies [3].

Our study examined three small software businesses operating in loosely structured, seemingly chaotic, markets (see the sidebar “How the Study Was Done” for business descriptions). We found that, despite their differences, each had independently developed similar techniques for managing software development. While these techniques do not necessarily represent universal answers for every business, they do include insights that may be useful for other small businesses.

Back to Top

Development Processes

IT researchers have studied many development processes designed to manage the tension between control and flexibility. Examples include the waterfall development process, the spiral development methodology, and rapid prototyping [1, 2, 4]. The most widely used is the waterfall development process (see Figure 1). The waterfall name refers to the stair-step depiction of the process through distinct stages. However, a more evocative description might be called the “canal” approach in which the project transitions from one stage to the next in controlled steps, like a ship transiting the locks of a canal. This stately procession is the antithesis of the loosely structured chaos in many small businesses. While the spiral development methodology and rapid prototyping represent more flexible processes, neither captures the freewheeling nature of small business development. Small businesses proceed in a more chaotic fashion, like a lone whitewater kayaker shooting the rapids of a river.

The image of a heroic kayaker dodging obstacles through instinct and quick reflexes is intriguing, but the history of IT shows that chaotic processes lead to system disasters. The goal of our study was to help us understand the underlying structure behind the seemingly freewheeling development approaches used by SITBs. That’s why we focused our search for the underlying “whitewater” process that gives form to apparent chaos.

Although the executives we interviewed were in three different companies, the companies themselves shared several characteristics. In order to provide a context for our study, the following outlines the environment in which the interviewed companies operate:

  • Product complexity. Each company offers full-featured products, though these products are less complex than comparable enterprise-class systems offered by large companies;
  • Modular architecture. The products use a modular approach, allowing for extensibility;
  • Small teams. The interviewed companies had software teams of fewer than 10 members, so coordination among them was relatively easy. The coordination costs are a special issue during software integration and release. As a result, smaller teams may find it easier to justify more frequent releases;
  • Internet delivery. The products and their documentation can be compressed and downloaded over the Internet; new releases incur no packaging or printing costs;
  • Limited customer base. The three companies had no more than a few hundred customers for each product in their portfolios; when problems or opportunities arose it was relatively easy for each of them to reach its entire customer base;
  • Single product path. Each release is a linear progression along a single product upgrade path; SITBs lack the resources to support special versions and parallel product directions; and
  • Customer characteristics. The target customer is a small entity lacking IT experts; hands-on support is critical; in fact, the willingness of an SITB to provide personalized installation, training, and follow-up is often a key selling point.

Although support needs are demanding, small business customers can be more tolerant in terms of feature requirements than their larger counterparts. These customers compromise on specific features, as long as overall functionality is acceptable. Moreover, infrequent bugs are not a problem, as long as they are resolved quickly.

Back to Top

Product Development Analysis

At first glance, the companies in the study appeared to be using unstructured development processes. However, comparing the three companies revealed common structures that build on small business strengths to provide an experientially based development process. We identified five components of that process—inspiration and evaluation; micro-releases; delivery and high-touch support; feedback and control; and technology platform (see Figure 2)—discussed in the following sections, which include representative quotes from the interviews:

Inspiration and evaluation. The process begins with inspiration, usually from customers, employees, and marketplace scanning. In evaluating new ideas, the customer is the primary focus; however, the company does not automatically accede to customer demands.

“[we] also develop software from ideas originated [here]…but we will still attempt feedback from…a potential customer…”—LegalSoft IT director.

Resources are inadequate to support multiple product variations. When resources are assigned to custom development, core product development stalls. Although new features may be related to a particular customer’s needs, they must also add to overall product attractiveness.

“Oftentimes a request…is so far off where we want the product to go that we will say no thanks…[and] redirect [it] to our partner.”—iManager CEO.

The line between brainstorming and evaluating ideas is often unclear. The companies prioritize their investments in development tasks based on employee and customer judgment, not on analytical tools. In this qualitative decision-making environment, executives gather input from stakeholders, synthesizing it to choose a direction based on their own judgement.


Small businesses proceed in a more chaotic fashion, like a lone whitewater kayaker shooting the rapids of a river.


This informal decision-making approach may seem risky, but the three companies in the study take two steps to minimize risk:

  • Limit the size of each development increment; since the amount of development in a release is small, it can be less costly to try out a concept than to conduct a market analysis; and
  • Look for lead customers to partially fund development.

Micro-releases. In this part of the process, features are released to the market as soon as they are ready. One company turns out weekly releases.

“Almost every week there are point releases. In more traditional software development methodologies of a decade ago these would never be released to the customer. [Instead] they would be bundled up over a cumulative period of a year, and then a major release would be sent out to the customer.”—iManager CEO.

The effects of a micro-release ripple through a business, allowing it to quickly monetize its development investment. Decision risk is minimized since the development cost of any specific decision is limited. The development process is simplified since any given release involves few changes. This means the likelihood of bugs is smaller, integration is easier, and tracking down problems is easier. Micro-releases also provide a marketing advantage.

“[The] ability…to turn around features quickly and get them into the market is critical, because the customer is not willing to wait for two years.”—iManager CEO.

At any given time, the company is only weeks away from a new development cycle, and unforeseen requirements may quickly be added to the next production cycle. Furthermore, micro-releases allow the company to demonstrate continuous product growth. Although the smaller SITB team cannot produce the absolute number of features a large development team might produce, it can deliver a more current set of features.

However, the companies studied do have occasional development cycles of longer duration. At times they must incorporate more complex features or refactor the software in order to realign the software’s structure with its evolving purpose.

Delivery and high-touch support. When micro-release development is complete, the SITBs are committed to providing their customers with a high level of support.

“…for the most part the salesperson will take the CD with them and assist them in the installation and setup.”—InsureCo CTO.

“…we either deploy them through a thin-client [Citrix] or provide installation support over the phone.”—LegalSoft IT director.

Although a small customer service staff may track market and production issues, the developers themselves usually provide the actual support, assisting sales presentations, managing installations, training customers, answering questions, and fixing problems.

Resource constraints and a limited customer base make developer-based support attractive but greatly improve planning. As developers support customers they see the product in use and become aware of issues and opportunities.

Feedback and control. Developers in the companies in the study apparently have considerable flexibility in working from broad development requirements, but this flexibility is actually bounded by several factors. For example, development includes three types of review sessions:

Current customers. “The development is iterative, with numerous demos with the stakeholders for feedback and direction.”—LegalSoft IT director;

Potential customers. “[If] we develop a product that clients have expressed interest in…we might solicit their feedback…”—LegalSoft IT director; and

Customer surrogates. “…a test group…often includes people from marketing and customer support…”—InsureCo CTO.

At the end of every micro-release, developers must train and support new customers on the just-implemented changes. This gives the developers quick feedback on any new changes, letting them know if adjustments are required. Developers have an incentive to produce high-quality software, since they know they will be answering directly to customers for any defects they might introduce.

Technology platform. SITBs provide the most possible features while minimizing the cost of their development. They focus on a single development environment and are not usually early adopters of technology.

“[Customers] think in terms of business problems…not necessarily the technology to solve [the problems].”—iManager CEO.

SITBs concentrate resources on new features rather than on duplicating features across platforms.

“[Concerning emerging technologies] we…adopted a wait-and-see attitude…key factors…are a limited training budget and finding ways to maximize the current skill set of our developers.”—LegalSoft IT director.

Sending developers to a week-long training session means a week of cost with no offsetting revenue. Similarly, betting on a new technology may mean a complete write-off if the technology fails.

“If you add that capability to the product, are you pushing yourself out of the target market?”—iManager CEO.

Product prices are market-based; any cost increase comes directly from the SITB’s margin.

New technology finds its way into the platform when required for the critical functionality to make a product viable. Environmental forces may also force the introduction of new technology. For example, traditional Microsoft ASP code may be satisfactory for a product, but marketplace expectations could force a move to a .NET solution.

Back to Top

Conclusion

Our study of software product development processes in three SITBs examined the apparently chaotic SITB software product development process. Environmental forces (such as resource constraints and the need to monetize development costs) affect SITB processes. We found that what at first appeared to be chaotic processes in the companies were experientially based processes that tie the development team directly to the needs of the market.

Whereas some large software development companies rely on specialists to manage market requirements, the three SITBs use a more integrated approach. The developers themselves are immersed in their customers’ world through product-support processes. Short cycle times reduce risk and allow teams to react quickly to new information; as a result, the product is continuously aligned with changing customer demands. Furthermore, Internet-based delivery enables the team to introduce changes quickly.

This whitewater process seems to be effective in helping small businesses see shifting market requirements and react to the changes resulting from them. It may be tempting to apply it to larger products, but the process is based on small teams developing less-complex products for limited numbers of customers. As products become more complex and teams grow larger, individual developers will become more specialized, but specialized developers may lack the broad knowledge required to interact with customers. Furthermore, operating in relatively small markets enables whitewater developers to experience the entire market directly. As the market for these products grows, individual customer interactions may be less representative of the market as a whole. A developer in a large market must address the risk of overreacting to the needs of a single idiosyncratic customer.

Back to Top

Back to Top

Back to Top

Figures

F1 Figure 1. Traditional waterfall software development process.

F2 Figure 2. Whitewater software development process.

Back to Top

    1. Baskerville, R. and Stage, J. Controlling prototype development through risk analysis. MIS Quarterly 20, 4 (Dec. 1996), 481–504.

    2. Boehm, B. A spiral model of software development and enhancement. IEEE Computer 21, 5 (May 1988), 61–72.

    3. Dean, T., Brown, R., and Bamford, C. Differences in large- and small-firm responses to environmental context: Strategic implications from a comparative analysis of business formations. Strategic Management Journal 19, 8 (Aug. 1998), 709–728.

    4. McConnell, S. Rapid Development. Microsoft Press, Redmond, WA, 1996.

    5. U.S. Department of Labor. High Growth Industry Profile: Information Technology. Washington, D.C., June 2005; www.doleta.gov/BRG/Indprof/IT_profile.cfm.

    6. U.S. Small Business Administration. Office of Advocacy, Washington, D.C., 2005; www.sba.gov/advo/research/profiles/05us.pdf.

Join the Discussion (0)

Become a Member or Sign In to Post a Comment

The Latest from CACM

Shape the Future of Computing

ACM encourages its members to take a direct hand in shaping the future of the association. There are more ways than ever to get involved.

Get Involved

Communications of the ACM (CACM) is now a fully Open Access publication.

By opening CACM to the world, we hope to increase engagement among the broader computer science community and encourage non-members to discover the rich resources ACM has to offer.

Learn More