The Codist

These type of projects normally begin with a simple request to do something straightforward and complete. Over time more and more requests start to pile up until the application becomes self-aware and eventually swallows the entire department or company. Each request is considered just a minor point, and normally management doesn’t understand why it should take longer. Often these projects descend into death marches or over time become Sisyphus projects.

Example

One of my friends a couple of jobs ago got involved in what seemed to be a simple project - write a brand new web app to manage the company's sales office hierarchy. The usual kind of thing: division, region, district, and office. The existing application ran on the company's AS/400 and involved a lot of manual and just-in-time data entry (i.e. if an office moved locations you had to perform certain updates on the exact day it moved or all the commission reports got screwed up). The old code kept no history and allowed no future changes so it was a pain to manage. A new web app was just the ticket to make things easier.

First thing in the company methodology was to determine the business needs. The application was needed immediately so the plan actually moved directly to writing the requirements document. No coding was to be done until this magical document was complete. However it was scheduled to be completed one month after the whole project completion date so coding actually started simultaneously.

One of the main complexities was that the existing database (the one with no sense of time) was tied into virtually every system the company had on the AS/400 and couldn't be (safely) modified, so the decision was made to build another database which would actually maintain the hierarchy, and a convertor which would update the old database at "exactly" the right time. Naturally the original database was not really relational and the convertor would have to written in RPG, not the 4GL system the AS/400 people used, and if anything went wrong, all the company's agents and the company itself might get a complete financial bath. Lovely recipe for success.

One of the best development issues was designing the user interface, which the business analyst ( the one writing the requirements document) decided that they had to design the user interface so it could be put in the requirements, and no one else was knowledgeable enough of the users (which were a couple people in the whole company whose entire job was managing the hierarchy) to do it. So there was a non-experienced designer designing an interface for an application with no requirements for a single user. Months passed.

Of course while the code was being written, the requirements were being debated by every department with a stake in the company. Apparently the rules of when an office (or actually any level) moved were not so simple. Various reports used different dates for the same function; sometimes departments wanted to shift sales to a different office or district based on no apparent consistent criteria, such as punishing someone for moving, or rewarding a higher up with extra income they didn't earn. Months and months of wrangling occurred, continuously changing the requirements (and breaking the code over and over again). Other departments decided it would be nice to have some of their functions appear in the new system. Yet other departments wanted nothing to do with the new system and demanded the ability to override the new database in the old.

Soon more and more of the 12 person java development team got sucked into the coding. Of course the deadline passed with no application (despite the demand it had to be in use by that date) and no requirements. Half a year after the deadline (and over a year after the start) the requirements document is complete, but the requirements themselves are still changing. The code is now a monster and yet is no where near complete.

So far they still manage the hierarchy the old way.

Solutions

Like the ten-foot-pole disaster this type of engineering quicksand is difficult for the development team to avoid or correct. It require a comprehensive top to bottom change of attitude by the entire development and business hierarchy to effectively manage IT projects and the business that depends on them. Failures of this type are always the fault of the organization and are usually a sign of other business management issues. In this company there were so many problems that led to this project's ongoing failure they cannot be described adequately. Sadly many companies have the exact same problems (although to be honest I never worked for a company with more troubles).

It takes strong IT management, and business management with an understanding of IT challenges, to avoid this sinkhole effect. I am not an advocate of Big Upfront Requirements and Design, which is often the result of project failures, based on the (faulty) assumption that more control up front will assure less trouble later. This never works in IT projects. Requirements (or more simply, what problem or need is the software trying to solve or provide) are never completely known and always (in my experience) are dragged out of the customer only after some part of the application is usable or visible. People only react to what they can see or touch; asking someone "what do you need" is pointless. Thus the only way to ensure a project succeeds (i.e. delivers what is needed, as opposed to what is requested) is to build it iteratively. Build a little, try a little, feedback a little, repeat.

The funny thing is by building a feedback look into your project methodology (a very "engineering" sort of solution) you wind up making customers, developers and management far happier. It doesn't mean you wind up with the kitchen sink, for your customers (and managers) get better feedback and see how their requests change the application so everyone learns how better to ask for things. In the end, it's the improved communication that ultimately improves the project success rate.