It is commonly acknowledged that success in IT projects is difficult to achieve. A recent industry survey observed that only 34% of IT projects were considered successful.9 Of the several potential factors contributing to this hard-to-achieve success, user involvement was noted as the most important one. Consistent with this notion, both researchers and practitioners have viewed user participation as an important way of improving software quality and increasing user satisfaction and acceptance.6 Users/customers are often encouraged to participate and directly communicate with developers in the software development process. On the other hand, empirical evidence shows that user participation in the development process can negatively influence project performance since it could make the process more difficult, lengthy, and less effective.4 Such contradictory findings raise the question of when user participation is actually helpful and when it might negatively impact project performance.
Previous research tells only one side of the story since it has examined user participation or project performance by focusing primarily on user viewpoints. The findings give an incomplete picture since they have not thoroughly addressed developer viewpoints. Clearly, there is a need to investigate software project success from the perspective of developers given that not only are they at the core of development process but they also represent the largest single cost in software development. Developer satisfaction is imperative for systems development success.8 Dissatisfied developers would adversely affect the quality of software as well as the productivity of development teams. A high rate of developer turnover in projects (due to dissatisfaction) could lead to increasing costs for development firms as well as high user/customer dissatisfaction.1 Due to the differences in background and circumstance, developers and users often share different and sometimes conflicting interests during software development. Researchers have identified a large gap in perceptions and definitions of project success between developers and the software industry (for example, users/customers). For example, developers tend to be achievement-oriented and are intrinsically motivated to develop excellent software, while users/ customers emphasis more on meeting a schedule or maintaining a budget.5 Thus, the potential conflicting interests between users and developers might negatively affect the software development performance.2
This article addresses the question of the relative effectiveness of user participation by empirically examining the perceived software project performance (for example, satisfaction) from both user and developer perspectives simultaneously. We used survey data from 117 software development projects and 746 respondents at a large Fortune 100 manufacturing firm during a four-year time period to investigate the impact of user participation on the satisfaction of both developers and users. Our findings offer insights into the impact of user participation on generating higher levels of developer and user satisfaction and, at the same time, minimizing the perception gap between users and developers on project performance. In addition, we also study the role of software complexity (for example, whether projects involve new software development as opposed to maintenance of existing software) in user participation and its effect on satisfaction.
Questionnaire data was collected from 453 software developers and 293 users/customers working on 117 software projects (for details of the data collection process, see the sidebar “How the Survey was Conducted“). The average number of developer survey respondents per project (for example, team) was four and the average number of user/customer respondents per project was three. Our analysis was performed at project level, and satisfaction scores for developers and users were averaged for each project. Of the 117 software development projects, 45 (39% of our sample) were maintenance projects and 72 (61%) were new development projects. The average software development time of the 117 software projects was 126 days. Figure 1 outlines the project characteristics in our sample. The most common business domain for our software projects was the manufacturing and supply chain (41 projects, 35%) and most of the projects were Web-based applications (78 projects, 66%).
The Level of User Participation
In our data, software project size was measured in function points,3 and participation time was measured in hours (gathered from time sheets). We adopted a normalized measure of user participation with size as a normalizing factor since user participation time is very likely to be closely related to the project size. In an average project, the user(s) generally participated in the following activities: project scoping and prioritization of requirements; responding and providing inputs to product prototypes created by development teams (these prototypes were primarily addressing user interface issues and communication functionality); and participating in design meetings and providing inputs on product features as well as design aspects that were important to the user group. For certain projects in our sample, the ratio of user participation hours to project size was zero because user participation was minimal and limited to planning and requirements gathering meetings. The levels of user participation—low, moderate, high—were computed based on the median value (0.462) of the ratio of nonzero user participation hours to the project size. Projects with nonzero participation level less than the median (0 < participation level ≤ 0.462) were categorized as moderate participation. Projects with nonzero participation greater than the median (> 0.462) were categorized as high participation. Of the 72 new development projects, 19 (26%) had low (very minimal), 26 (36%) had moderate, and 27 (38%) had high user participation. On the other hand, of the 45 maintenance projects, 15 (33%) had low, 16 (36%) had moderate, and 14 (31%) had high user participation in the development process.
New Development Project Satisfaction
The results of developer and user satisfaction on user participation in new development projects (left) and the absolute difference between these two levels of satisfaction (right) are plotted in Figure 2. Our study of the relationship among user participation, development project type, and satisfaction was not confounded with other factors such as project size, development time schedule, performance, and scope change. We find that the effect of user participation is statistically significant even after controlling for the aforementioned factors (p < 0.05).
The results of our analyses show that for new product development efforts, an “X-shaped” effect occurs as the level of user participation increased. Developer satisfaction improved considerably with increasing user participation (p < 0.001). Users, on the other hand, felt less satisfied when they increasingly engaged in development processes. From developer viewpoints, these findings partially support arguments from existing literature that user participation has a positive impact on project satisfaction.8 Further, while we find significant results on developer satisfaction, we do not observe a relationship between user satisfaction and participation (see Figure 2).
An X-shaped effect suggests that users were most satisfied while developers were significantly less satisfied in the low (minimal) user-participation situation. The perception gap between user and developer on project performance appeared to be the largest in this level (see Figure 2). Developers, under such circumstances, were likely to find greater difficulty in resolving requirements and product feature ambiguities. This often led to increased frustration from developers, thereby making them dissatisfied. From the users’ viewpoint, if users did not participate in any of the development activities, they are less likely to expect or demand as much as if they had been heavily involved in the development process.
At moderate levels of user participation, we observe that the gap in perceived satisfaction between developers and users is comparatively minimal. Even though the satisfaction levels were not at their highest, developers and users seemed to reach a common ground when users engage only moderately in the development process. In our sample of new development projects, both developers and users shared very similar levels of (relatively high) satisfaction when users participated for an average of 3.88 workdays for a 100 function point project, implying 28 workdays of customer effort for an average new development project (711 function points, corresponding to 127 calendar days). In other words, users invested approximately 22% as much as the overall project development time.
Users who intensively participated during development felt the least satisfied among the three groups of participation. In contrast, developer satisfaction reached the highest point at this high-participation level. In our sample, a project in this group (high participation), exhibited an average of 105 workdays of customer participation, in comparison to 28-workday effort for a moderate participation project, for an average of 127-workday project. That is, users participated 83% of the overall project development time. Software developers might be grateful for such intensive participation since new development projects often tend to be complex and require increased communication and coordination. However, when users participate to such extreme levels, their expectations of project outcomes could also be exceptionally high. These users are then very unlikely to be satisfied if actual project outcomes do not perform to their high expectations.9
Maintenance Project Satisfaction
Both software developers and users were unlikely to be fully satisfied with low user-participation in maintenance projects (see Figure 3). Surprisingly, as the level of user participation increased, we observed an “inverted U-shaped” effect with respect to both satisfaction levels, in which both satisfaction levels were highest at moderate levels of participation. This effect may be due to the fact that software maintenance is a task that is fairly difficult to execute and manage effectively due to added complexity of understanding prior written code and prior corrective repairs on the baseline software code. The quality of the current software drives the software maintenance outcome.3 In such cases, software developers might be frustrated by the significant amount of time spent on studying the original software code, especially in the case of poorly developed software.
Developers often operate under constraints of unclear software requirements. When users are involved in such maintenance projects (moderate participation), they might be able to lower developers’ frustration and enhance developers’ overall software comprehension since users tend to know more about the features in the original software. In our sample, an average project with moderate user participation involved approximately 19 workdays of customer participation (calculated using a baseline rate of 5.66 hours per workday for a project with 503 function points). That is, users participated approximately 15% as much as the overall project development time.
In contrast, at high levels of user participation, interactions between users and developers can generate unnecessary conflicts and increased reworking of software features. In our data, users and developers were relatively less satisfied when users participated for approximately 75 workdays (60% of the overall project development time) for an average project (with 503 function points that takes 126 calendar days). As we discussed previously, users tend to expect more when they engage more in projects. At high expectation levels, increased user participation in the software maintenance process can lead to unattainable expectations. In other words, users might feel less satisfied with the actual outcomes given their high expectations of the developers and high participation in the project. Even if the original software is well-developed, the maintenance task is relatively less complicated, and the overall maintenance project is completed with fairly minimal changes in scope, the high expectations of users might remain unsatisfied. At the same time, software developers are likely to be unhappy with overdemanding customers/users. Increased conflicts are also likely to result from frequent and overscrupulous changes suggested by customers on an ongoing basis.
Lessons Learned
No matter how technically good a piece of software is, it is still poor software if its users perceive it as such. Therefore, a level of high user satisfaction is essential to software project success. However, previous research has identified a perception gap between users and developers on project performance. Users/customers and developers often share different views on project satisfaction and perceived project success. By simultaneously examining developer and user satisfaction, we hope to provide a more complete understanding of the perception gap and the relationship between perceived project performance and the level of user participation. Our analysis is based on time sheets and survey data from 117 projects and 746 responses at a Fortune 100 manufacturing firm.
For the sample of new development projects, an X-shaped effect arises from increasing the level of user participation. Even though developers were most satisfied with high user participation, users were most happy by engaging minimal time (low participation) in development process. The perception gap was fairly large at low as well as high participation levels. In contrast, both users and developers shared very similar perceptions of project performance (for example, smallest perception gap) only when users were moderately engaged in the development process. On the other hand, for maintenance projects, an inverted U-shaped effect appears with increasing user-participation levels. Even though the perception gap is comparable across three levels of participation, both users and developers were most satisfied with moderate levels of user participation (for example, participated approximately 20% of the overall project development time). In general, extremely high participation (for example, invested more than 60%) is not welcomed by both parties. Our evidence supports the argument that potential conflicts arising from greater user participation may play a vital role on perceived satisfaction of software developers and users.
Our findings have important implications for managers trying to balance the challenges of managing new software development and making enhancements to existing software, while engaging the user actively in the software development processes. Managers should be aware of the potential trade offs between achieving utmost satisfaction and minimizing potential user-developer conflicts (for example, perception gap). In a new development project, developers have a poor understanding of product requirements, and are often unable to make wise product decisions because of requirements ambiguity. Ideally, they would benefit significantly from user inputs in these projects. However, given the novelty of the technology involved and their uncertainty about “what the end product should look like,” users too are likely to find it difficult to express their needs and suggest design guidelines. In these projects, project managers and leaders can play a strong role in clarifying the expectations of the customers/users as well as the software development firm.8 While more participation is always likely to be viewed as beneficial to the development group, managers might find that users might be unable to contribute in a manner desired by developers.
Finally, in maintenance projects, users are likely to have clearer expectations from the end-product and comparatively knowledgeable about desirable product functionality. Developers would greatly benefit from seeking these inputs and minimizing the ambiguity in product requirements and design choices. Managers could act as facilitators, helping bridge this gap in knowledge between the two groups. However, with excessive participation and micromanaging of tasks, both groups could develop conflicts and managers might need to ensure that non-productive processes are tempered.
Join the Discussion (0)
Become a Member or Sign In to Post a Comment