The Big Bang And Other Stupidities

At the place I am currently contracting they are in the midst of a giant replacement of everything project. My part of it, an iOS app, is mostly done.

The bulk of the dozen or so applications are either incomplete, untested, or not even started; yet the deadline is now set to the end of this year.

Why do people do this? After 3 decades as a programmer I can tell what's possible and what has a chance of some imaginary number of happening.

Big Bangs are great for starting universes but are a terrible yet oddly common technique for developing software. Somehow people get the idea that you can do anything you put your mind to, including flying to the moon by flapping your hands, which is more likely than replacing a huge collection of systems all at once with all new code.

I've seen this way too many times to think it will ever be unpopular. They always fail, sometimes miserably, occasionally by limping along for years. Yet hope springs eternal.

Having worked as a programmer for so long, one thing I know is that software is never easy to write, and the more you write at once the harder it is, and the more you write that has to work together with other new software the less likely any of it will ever work. Trying to test one piece is difficult; trying to test all of the pieces at once makes it impossible to ever know what might be working and what might be broken. You test a report app and something is wrong. Is it the report app, the report API, the database, the data, the data collector API or a data collector app? How do you understand the problem if there is nothing solid you can built a test plan on? Trying to trace a problem through half a dozen systems is mostly impossible. Even if you fix what you think is wrong, how will you ever know you didn't just break the other 5 apps?

Big Bangs are eternally popular, who can resist the idea of all new software, full of amazing new features? Even better, imagine all the old data converted to use with all the new whiz-bang features! No more terrible old software for us!

At my previous employer, where I worked for a highly functional mobile team, the company a decade earlier had overspent for a foreign company, and then embarked on a grand adventure to build a new system to replace both company's systems with a new modern system. After years of failure which sapped all the development time, the existing systems for both companies were years behind the competition, and the new Uber System was still barely functional and slow. Our parent company finally gave up and turned us into a paper brand and got rid of most of the employees. Of course the existing system (which we had to call from our mobile API) was terrible but if all that energy had been put into updating it over the same time frame, we might still be cooking and I wouldn't be watching it happen again somewhere else.

Writing code in small chunks makes it far more likely to be testable and predictable; this is a fractal in that small changes can happen at every scale. While I despise working on terrible code (the iOS app I am working on was written by an army of outsourced people for more money than I want to know about before I got there, and was leper quality) it's never a good idea to go whole hog on replacing everything. One app in a large collection maybe, but too many moving parts makes for impossibly long development times and possibly impossible in general.

If you have a solid API you could rewrite an app that calls it and know you can always test the old app and the new one and ensure that nothing is different. But if the API is changing as well, you have nothing to base testing on. You wind up driving a car that's still being assembled on a road full of bulldozers being assembled by toddlers.

Sometimes I've seen people fear Big Bangs so much they insist that you change nothing but just add a Big Bang full of features anyway. I'm not sure if this isn't worse, trying to convince people that this isn't really different, and in many ways even more evil, than a replacement Big Bang. Maybe you could call these the Big Upgrade.

My former employer's parent company has an enormous mainframe based system built in the 60's. Over the years they have built shells around it, seemingly every decade a new one. Now the system is so complex, surrounded by shells that it can never be updated at all. On paper this would seem a proper plan, surround the old mess with something new to encapsulate it, and then you can upgrade the inside at your leisure. But for whatever reason it never happens and soon you have so many layers its impossible. Now you have a Big Universe you can never escape. Need new functionality? Either build another shell or do without.

Management types often can't understand why software can be so hard to write, so they often either fail to say no to a Big Bang, fear one but want Everything anyway, or demand something be done Quick and Dirty that can be fixed up later. Then when the whole thing becomes a disaster they can't understand why no one explained it to them in the first place. But it's difficult to tell people that software is hard if you don't do it right from the beginning, and that takes time. Time of course costs money, yet in trying to do something the wrong way ultimately results in wasting even more piles of money. It's a nasty cycle.

Moving a body of software forward has to be done just like moving a few lines forward; you can't do everything at once, you have to take it in reasonably small bites with an eye to keeping it stable. Now if you are writing an MVP in a startup it's not a big deal, you need speed, and maybe writing it quick will keep you moving (though I am often a big fan of slow programming). If you are working with functional systems that people depend on, Big Bangs are simply a way to get you blown up. Trying to force too many changes too fast with too many new features just gets you a longer and longer development cycle which tends towards infinity or bankruptcy.

Programming good software is hard but not that hard unless you make it hard on purpose. People in this industry have a tendency to make things difficult by arbitrary deadlines, massive rewrites, too many features all at once and the proverbial lets make a baby in a month with nine women. Good software takes time, and that time can't be shortened by some magic trick, or wishful thinking, or management directives.

Sometimes I wish my job involved going around and helping people with their development problems, I've seen so many, and also seen and participated in so many successes, that it would be very rewarding. Sadly I also know that people don't like to be hit with reality and often react very negatively. So it might be depressing; I've tried in previous jobs and sometimes it helps but often you just wind up just making yourself frustrated.

My dad when I was a kid took a job in another country so we had to move, they had hired him to implement some big changes. After a few months he told them if they continued they would be out of business in a year. He quit and a year later they were gone. He hated talking about this job for the rest of his life. I've seen it too.

So please for the love of good software keep the Big Bang in the world of physics! Try to teach your management folks that good software is possible for those who do it right and in the right time.

If they don't listen there are plenty of fish in the sea. Maybe some of them like good software.