Coprolitic Programming

While I wile away my time writing a major post (The Future Of Programming), I'd like to approach a scary secret of our industry: Coprolitic Programming.

Coprolite: Fossilized dung

Coprolitic Programming: Working on systems of fossilized crap

Everyone has known or has worked on software systems or applications built on ancient code bases which are difficult to work on, require large teams to keep running, defy extension or interconnection, and generally scare the hell out of everyone involved. Over the years they become less and less understandable as more and more engineers add their changes (you could call this mineralization). Management types lose hope or refuse to consider replacing these systems due to perceived expense or difficulty: better the rocky turd you know than the fresh one you don't.

Now mind you some of these systems continue to work and produce value despite the difficulties; even a fossilized bit of animal remnant can make a fine brick or tool. The Sabre Reservation System (written in the 60's) ran until recently as the backbone of a large portion of the airline reservation universe and managed some 8,000 transactions per second with little down time. Yet making changes was a major issue as the system was written in a macro assembler, used what amounted to physical disk sectors as a database, and required major rings of protocol conversions to survive into the internet era. After something like 10 years of work the system now runs a nice modern architecture.

I remember being at a meeting at a local defense contractor in 1998, who pointed out that many of their database systems were written in the 70's and never updated, as this would require someone to take responsibility (both financial and managerial) and no one wanted to tackle this. Trying to add web support to these coprolite DBMS's was too difficult for them to do and they were looking for someone to do it for them. We declined. Software development is hard enough without the archeological aspect.

Programmers who toil on these types of petrified projects rarely enjoy the experience. Most either burn out or are reduced to a dull-witted existence where fixing a bug a week justifies success. The experience of working in what amounts to a large specimen of something pungent you would avoid if it were fresh, devoid of modern software practice, languages, algorithms and knowledge makes it virtually impossible to find new jobs or start new projects. The only option is a continued death march into retirement until their brains start to take on the hardness of their project. Generations of management come and go but the project endures.

Man that was a depressing paragraph. Yet I have seen projects of this type and seen what it does to people. I am sure we all have. It's the ugly backwater in our industry. There are running Cobol programs out there almost as old as I am. Yes, they still work. People still work on them. 5% to 10% of the population takes antidepressants; a coincidence? Hardly.

The arguments against rewriting or replacing coprolite systems usually fall on favorite management topics: it's too expensive, we'd rather have the system with the 4,032 known bugs in which the customers have found 17,345 workarounds than start the bug count over again, we would have to hire new programmers since all of ours are fossilized, and my favorite: "it's too hard". From a business point of view it's easier to justify the large staff you already have supporting the frozen dung monster than to justify a new project. The long term danger of a system that can't be modified to meet a changing world is hard to quantify and easy to push off on the next generation of decision makers.

So legions of coprolite programmers continue to toil on systems which future archeologists will struggle to understand how our civilization survived with them (if indeed it does). It's a sad truth that software being the most malleable of technologies can none the less become as fossilized as dinosaur skeletons and their poop.

The next time you think to sigh, "this code is such a pile of crap" imagine what your grandchildren will say some day when they work on it.