These type of projects normally begin with a simple request to do something straightforward and complete. Over time more and more requests start to pile up until the application becomes self-aware and eventually swallows the entire department or company. Each request is considered just a minor point, and normally management doesn’t understand why it should take longer. Often these projects descend into death marches or over time become Sisyphus projects.

Example

One of my friends a couple of jobs ago got involved in what seemed to be a simple project - write a brand new web app to manage the company's sales office hierarchy. The usual kind of thing: division, region, district, and office. The existing application ran on the company's AS/400 and involved a lot of manual and just-in-time data entry (i.e. if an office moved locations you had to perform certain updates on the exact day it moved or all the commission reports got screwed up). The old code kept no history and allowed no future changes so it was a pain to manage. A new web app was just the ticket to make things easier.

First thing in the company methodology was to determine the business needs. The application was needed immediately so the plan actually moved directly to writing the requirements document. No coding was to be done until this magical document was complete. However it was scheduled to be completed one month after the whole project completion date so coding actually started simultaneously.

One of the main complexities was that the existing database (the one with no sense of time) was tied into virtually every system the company had on the AS/400 and couldn't be (safely) modified, so the decision was made to build another database which would actually maintain the hierarchy, and a convertor which would update the old database at "exactly" the right time. Naturally the original database was not really relational and the convertor would have to written in RPG, not the 4GL system the AS/400 people used, and if anything went wrong, all the company's agents and the company itself might get a complete financial bath. Lovely recipe for success.

One of the best development issues was designing the user interface, which the business analyst ( the one writing the requirements document) decided that they had to design the user interface so it could be put in the requirements, and no one else was knowledgeable enough of the users (which were a couple people in the whole company whose entire job was managing the hierarchy) to do it. So there was a non-experienced designer designing an interface for an application with no requirements for a single user. Months passed.

Of course while the code was being written, the requirements were being debated by every department with a stake in the company. Apparently the rules of when an office (or actually any level) moved were not so simple. Various reports used different dates for the same function; sometimes departments wanted to shift sales to a different office or district based on no apparent consistent criteria, such as punishing someone for moving, or rewarding a higher up with extra income they didn't earn. Months and months of wrangling occurred, continuously changing the requirements (and breaking the code over and over again). Other departments decided it would be nice to have some of their functions appear in the new system. Yet other departments wanted nothing to do with the new system and demanded the ability to override the new database in the old.

Soon more and more of the 12 person java development team got sucked into the coding. Of course the deadline passed with no application (despite the demand it had to be in use by that date) and no requirements. Half a year after the deadline (and over a year after the start) the requirements document is complete, but the requirements themselves are still changing. The code is now a monster and yet is no where near complete.

So far they still manage the hierarchy the old way.

Solutions

Like the ten-foot-pole disaster this type of engineering quicksand is difficult for the development team to avoid or correct. It require a comprehensive top to bottom change of attitude by the entire development and business hierarchy to effectively manage IT projects and the business that depends on them. Failures of this type are always the fault of the organization and are usually a sign of other business management issues. In this company there were so many problems that led to this project's ongoing failure they cannot be described adequately. Sadly many companies have the exact same problems (although to be honest I never worked for a company with more troubles).

It takes strong IT management, and business management with an understanding of IT challenges, to avoid this sinkhole effect. I am not an advocate of Big Upfront Requirements and Design, which is often the result of project failures, based on the (faulty) assumption that more control up front will assure less trouble later. This never works in IT projects. Requirements (or more simply, what problem or need is the software trying to solve or provide) are never completely known and always (in my experience) are dragged out of the customer only after some part of the application is usable or visible. People only react to what they can see or touch; asking someone "what do you need" is pointless. Thus the only way to ensure a project succeeds (i.e. delivers what is needed, as opposed to what is requested) is to build it iteratively. Build a little, try a little, feedback a little, repeat.

The funny thing is by building a feedback look into your project methodology (a very "engineering" sort of solution) you wind up making customers, developers and management far happier. It doesn't mean you wind up with the kitchen sink, for your customers (and managers) get better feedback and see how their requests change the application so everyone learns how better to ask for things. In the end, it's the improved communication that ultimately improves the project success rate.

My Tags: project management programming

Disaster #3: The Sisyphus Project

Description

Sisyphus was an ancient greek mythical character who was eternally condemned to roll a stone up a great hill, only to have it always roll back down so he had to start over. In this type of disaster the project can only be worked on with great effort, long hours, difficult work and usually massive stress and pressure from management. Once a version is delivered it begins again. These type of projects usually burn out developers leading to many new additions not familiar with the beast, so it actually gets more difficult over time.

Example

I've been fortunate to have never experienced this type of project, so I don't have any concrete examples. I do know that this is not an uncommon situation in many companies and there are usually two underlying causes: a lack of experienced technology leadership and management ignorant of the realities of software development.

Software development today is actually much more difficult than it was when I started 25 years ago; the number of technologies required to build and maintain a modern application is almost too much for anyone to comprehend, must less architect, design, test and ensure interoperability with other systems. Simply choosing a set of tools and technologies to use is a challenge since no one can have experience in everything (despite what you see in job ads). If the technology leaders (architects, project managers, project leads) have only limited experience with or even refuse to acknowledge better alternative choices then the base of an application may suffer for the rest of its lifetime (and the programmers with it). Discerning potential problems in technology choices before they become a real problem requires a willingness and time to experiment or prototype before embarking on a big project. Jumping into a large project with unknown technology is begging for a disaster.

Top-level management often has no real clue on how technology choices may affect the cost, time and performance of a large project, preferring to listen to sales pitches from vendors instead of their own technology leaders. Then when a project starts to have difficulty they may assume their team is at fault, as the vendor assured them "it was easy". Now they pressure the team to deliver and the ball starts rolling uphill.

One company I worked for embarked on a new portal strategy for its customer and sales force facing sites without any real idea why, other than a desire to join the crowd. Porting the customer site (already written in a mishmash of technologies and styles) to the portal took much longer than anyone predicted and created am even worse mashup of code which was difficult for anyone to properly test or sometimes simply get to run. The sales force application portal was always threatened but no one ever wanted to start it. See below.

Disaster #4: The Ten-Foot-Pole

Description

Some projects are so important and far-reaching, they simply cannot be started at the present time due to manpower shortages, business reasons, cost, or a surplus of cosmic ray particles. Actually any reason is sufficient to avoid having to do this project even though not doing the project may cause even greater problems. In most businesses it’s far easier to justify painful existing processes or systems than to step forward and tackle a replacement. Due to its enormous importance the project never goes away but remains just out of reach.

Example

The best example of this was a smaller local defense contractor which asked the consulting firm I worked for at the time to come in and see what we could do for them. Some of their managers discussed with us their problems in development for the web (this was late 90's) since no one had any experience with it. They knew many of their systems and applications were too inflexible and hard to use and wanted some (easy) was to fix them.

This all seemed reasonable to us in the early heady days of the web until we asked about their database systems. They mentioned they were still using databases designed in the early seventies and had coupled the applications directly to them. When asked why they hadn't updated the database system in nearly 25 years they replied that with a project-based accounting system, someone would have to pay for it out of their project budget, and then be held responsible for any problems it would cause (likely to be huge since it touched everything the company did). There was no management responsible for the company's entire technology environment so no one had enough authority to even look at the problem.

Given the tremendous coupling of the company's applications to the neolithic database system we found nothing we could do to help them and nothing was ever done. One can argue that "if it isn't broken, don't fix it" but often the problem is not that the current system is unusable, it's that it's inflexible and potentially unmaintainable. In this situation the company was unable to improve their productivity as the old applications were difficult to use, brittle, and especially hard to modify as business needs changed. So changing technologies was painful, but keeping the old technology was equally painful but at least familiar. The company is still in business but losing money (a rarity for an longtime defense contractor).

Solutions

Both of these project disasters are complex problems usually with the decision-makers in an organization. Unless you are one of them it's difficult to fix the problems from below. If you are one of them the best solutions are:

• Educate Yourself - read unbiased reports, news and even technology sites to gain an understand of what people are doing, and how effective the solutions are
• Never Believe Vendors - don't let vendors convince you of anything unless you can get independent verification. A consulting firm salesman I once worked with told me "His job was to lie to customers, and mine was to make him look good".
• Hire Broadly Experienced Technology Leaders - find people who are willing to learn and use anything, have worked with many technologies, preferrably hands-on, and especially are willing to speak their mind.

My Tags: project management programming disasters

Software project disasters come in all shapes and sizes; in my 25 years in this industry I have seen almost everything. Here are a few disasters described and illuminated for your reading and gagging pleasure. I doubt anyone involved in the software industry hasn’t experienced several of these types. There are 6 in the pipeline, I may think up more.

• The Train Wreck
• The Death March
• The Sisyphus Project
• The Ten-Foot-Pole
• The Kitchen Sink
• The Painless Upgrade

Any similarities to the The DailyWTF are purely a figment of this industry.

In the next few articles I will write up the disaster types with a description, an example, and some thoughts on how to avoid the disaster.

The Train Wreck

Description A train wreck is a project where everyone assumes things are going well, progress is being made, and it appears that everything will turn out well. In reality, the train is barreling down the tracks at good speed until it hits the bridge that is out and then crashes spectacularly into the valley. No one is actually driving the train and all anyone sees is out the side windows, so the illusion of progress is great.

Example The best example of this common project type I will call "Country Doctor". A team of entrepreneurs had the brilliant idea of selling a new, modern medical clinic management package for a specific specialty. Aided by a new government mandate (HIPAA) requiring everyone update their software, and having a killer feature no one else had (that would save clinics lots of money), it was a sure fire hit at trade shows and sales calls. People not only paid in advanced but many offered to invest in the startup as well. Now all that was needed was the software.

One of the founders knew a project manger in another state that they had worked with, and this person happened to know an architect who could design the application, so a development office was opened in the most expensive real estate area in the country. The architect came up with a object model, but had no interest in UI design or managing development so the project manager hired a large group of contractors to work on the UI and database pieces along with the business logic. Since the application was way too secret to show to anyone, the application was sliced vertically (from database to UI) and each contractor was only allowed to see the source of their slice. Any contractor who tried to look at source they were not working on was fired. With no one to look over the entire design massive duplication and outright conflicts were everywhere. But the reports coming from the development office to the business team was always positive and a few bits of the app were visible so everyone was excited about progress.

Of course the business team knew nothing of development, the sales team sold the sizzle without having to try, and each developer got closer and closer to delivery. Then came the start of beta, when a few lucky clinics would start use this wondrous package. First they hired a few industry experts to make sure no detail was missed. Bad idea, the first time anyone who actually knew medical accounting saw the package they realized it was completely unusable. Every screen worked differently, features appeared in no particular order, important features (like paying bills) were not even working; it was an unusable mess. Now the entrepreneurs had a big problem, as the HIPAA deadline was approaching and all these clinics had given money by the handful and disasters would happen if the software wasn't ready. So they hired the company I worked for to "fix" the code.

Our first inkling that something was wrong was getting the source. Apparently no one had ever compiled the entire application from source, due to the "security"; they simply compiled a slice and added it to the existing binary. After a month of work we managed to get something running but the horror of the sliced up application was apparent and no amount of work would every make it usable, other than a complete rewrite. But our company was desperate for money so we kept working anyway. Eventually we tried to get paid for all our work but they ran out of money, closed the doors, and ran for the hills. I bet the lawsuits are still chasing after them.

Solution Train wrecks survive due to a lack of open communication among all the participants, a fear of honest status reporting, and most principally, a lack of visible progress. My example had even more problems that I can easily document but the inability of the company founders to have an independent eye on the progress of the application (no domain experts until the end) made it far too easy for the train to move to the crash.

Modern iterative schemes break up a project into smaller repeatable units, where the application is always functional at the end of the iteration, and can readily be examined. It's hard to have a train wreck where the train stops every mile while everyone looks around. There is no excuse for anyone to be "surprised" at the delivery date. It is also imperative that non-technical management have knowledgeable, trustworthy people able to explain what is happening in language they can understand, and that they listen.

My Tags: project management programming