Why Do Projects Fail?

Why Do Projects Fail?

Stories of spectacular failures abound. A project that was budgeted at $10 million is finally killed when it passes $100 million. A system that was due at the end of July is delivered at the end of September two years later. Why do these and less extreme but equally frustrating failures haunt the industry?

The most commonly cited culprit is bad estimating. It is the conventional wisdom that technical people cannot estimate the cost of lunch, even given a menu to work from. There is only one problem with this explanation: It is a myth. Of course, there are poor estimators, but even if the entire estimate were off by 100 percent, the total cost would do no more than double. This is not desirable, but neither is it comparable to the industry's celebrated overrunsthe ones that we speak of in hushed tones, thankful that we were not involved.

In fact, most people can estimate reasonably accurately, and, in the sweep of a large project, the optimism of some estimators is fairly well balanced by the pessimism of others. In fact, there are three recurring reasons that projects fail: scope changes, work breakdown, and technology.

Scope Changes

Scope changes are probably the major cause of failures. The problem with scope changes is not simply that they add cost, but that they add cost out of proportion to their apparent effort. Consider, for example, a project estimated at $1 million that grows into a project that, had it been planned that way from the start, would have been estimated at $2 million. It is tempting to believe that the final cost of the project will be about $2 million. In fact, it will be closer to $4 or $5 million.

The problem is that scope changes disrupt planning and development that has already been completed. In the simplest case, file or database structures, along with screen and report layouts, must be changed, which involves rework. Even worse, it often happens that the design of the $1 million project is not appropriate for the larger project. The first response of developers is to try to finagle the design to accommodate the change. This is reasonable; nobody would agree to redoing the design simply to handle a small scope change. But the result is that the design is no longer clean, and when the next scope change appears, the distortions will expand and will continue to expand with each new change until they have gradually infiltrated throughout the entire system. At some point, it will become clear that the design is inadequate and needs to be redone. In effect, the project now starts back at the beginning.

Scope changes occur for three reasons, the most dangerous of which is that the scope was not clearly established at the start of the project (see "Defining the Scope" in Chapter 3). People quite reasonably will request new features without concern about scope because they believe these features are well within what the project was intended to accomplish. Furthermore, without a clear definition of the scope, the project manager has no criteria for challenging requests. So the project balloons far beyond its original concept.

The second reason for scope changes is poor management of scope (see "Managing Scope Changes" in Chapter 5). Changes get smuggled in or innocently added by cooperative team members, so that the project manager is not even aware of them, and those that are known are not subjected to any formal scope change mechanism. The result is that scope changes inundate the project and destroy any hope of meeting the plan.

The third reason that a project's scope changes is because business conditions change. For example, in the middle of a project, the client company acquires a competitor, and the changes in business that result cause a massive upheaval in the project. It is clearly unfair to blame the project; if it had been finished, a new project to enhance the systems would have been part of the costs of acquisition. But because the project is in progress, with a published schedule and budget, it is all too frequently blamed when changing business requirements derail it.

Work Breakdown

Projects are always broken down into small, manageable chunks of work activities that can be assigned to specific people and tracked for completion. Unfortunately, two problems with breaking down the project are that activities can be missed and that they can be glossed over.

If an activity is missed in planning, it is not part of the estimates, does not appear on the schedule, and has no effect on the plan. Therefore, when it does appear, it can cause chaos. Sometimes these activities are relatively minor and can be added, albeit at some cost. For example, if the plan does not include user training, the cost will be that somebody has to develop training materials and deliver the training. However, if the omission is not found until the system is ready for release, the time required for preparing for the training and scheduling the attendees could add one or two months to the release date.

Even worse are missed activities that delay the entire project team. For example, new development hardware is required, but the plan overlooks the effort needed to set up the network. In this case, not only is there a delay while the network is being designed and implemented, but the entire team is sitting around doing very little other than charging costs to the project.

Worse yet are missed repetitive activities. As an example, assume that a project will require analysts to build unit test plans and developers to conduct the tests for each program. If the estimates have failed to include time for building the plans, conducting the tests, and revising code based on the results, the project time and costs will mushroom as one or two weeks is added to each program unit.

Some activities are mentioned but glossed over. One activity that is vulnerable to this is integration. If the plan contains just one activity for integration, it is a signal that the project manager has not thought through the steps needed to integrate programs into a system. The problem here is not only that the actual integration will take far longer than the time that was cursorily estimated, but that the entire project will not be conducted with integration in mind. For example, if programs that should be integrated last are the first to be developed, then when the integration team is ready for them, substantial changes will probably be required and the programmers responsible may no longer even be on the project.

Technology

Technology was not a problem in systems projects as recently as ten years ago. The only serious technology came from a few large manufacturers, and if a systems manager was so bold as to purchase hardware or software components from someone else, each manufacturer was so dominant that its standards were unchallenged.

Today, the reverse is true. A typical project development environment will include tools for coding, linking, debugging, accessing databases, laying out graphics screens, developing reports, handling inter- and intraprocess communications, managing code and document changes, generating code from CASE tools, and numerous other development activities. All of these tools must fit within a particular operating system, all come with a multiplicity of options and versions, and all are provided by different vendors. Those companies that select a development environment and stick with it for a reasonable period will find that they are rewarded by increased productivity and predictable projects. Unfortunately, many companies become sold on the idea that their development environment is archaic and that this one tool will be the silver bullet that will solve their problems. The result is twofold. First, productivity suffers because developers never develop deep expertise in an environment. Second, projects suffer because, inevitably, the new pieces do not work with some of the older ones or with one another, and simply trying to get the environment functioning takes months.

Summary

Projects do not have to fail. When project and corporate managers and development teams take seriously the planning and running of projects, projects usually succeed. This book is intended for those who prefer to work quietly on successful projects and are happy to leave the disasters to others.