The Codist

I have a lot of empty time at work, so I am learning node.js and redis. Node has such an enormous library of modules I felt like a kid at Christmas. Wow, I need that, and that and of course that and who can live without that. I wound up with 20 different modules. But how to make best use of all that stuff? What if something goes cuckoo?

A great example as well is the venerable Spring framework for Java. I remember when it first appeared and thought it an interesting idea in just a few lines of Java. Now Spring is the most massive monstrosity ever, trying to do everything for everyone. I highly doubt anyone actually understands even a small portion of it any more. They even advertise it with the meme-word "Enterprise".

I think sometimes we in the software world just enjoy making things more complex in an ever increasing desire with making things easier. But does this goal justify the path?

We build on top of lots of open source or proprietary software in order to minimize how much we need to do ourselves. But the flip side is we have to trust a whole lot more: things we didn't write or understand how they work now form the base of what we do write. When something goes wrong it's often because we don't exactly know what that combination of things will do.

People these days are big on automated testing to try and keep bad things from happening, but it often winds up a false prophet since we don't really test everything, just the stuff we wrote. The more we build on the more it's a black box of unknown quality and fuzzy interactions. We build software on tight schedules and time-boxed iterations with little time for understanding how it all should fit together.

I'm not saying toss out all frameworks and write everything from scratch. There is too much good stuff out there that is well designed and highly useful. But just because you combine a bunch of good stuff it might not be as good together. Combine it with whatever you write and when things go wonky it might be a massive pain to figure out where the issue actually lies.

We inherited an iPad app a couple of years ago; it was written by a third party and came from an office that operated independently from the main company (and got dismissed for that). Once we got the source we were expected to ship it in 10 days even without any assistance from the coders. That company usually wrote movie apps and apparently had never written a complex e-commerce app before. They used Restkit, a complex and not terribly well written framework for mapping JSON (and XML) to objects. We managed to get it done but not without wasting two days trying to figure out a race condition buried deep in Restkit which led to duplicate bookings, obviously a bad thing. Plus Restkit was so deeply embedded in the code we were never able to replace it.

It was a bad choice and hamstrung that app ever since. When I started to build a new mobile framework for our future work, I was determined to build something I could completely understand.

Yes, I wrote it from scratch without any frameworks other than iOS. That's not always the answer but in this case I had time to consider how to build what we would need in the simplest, most understandable way. It supplied the networking stack, JSON handling, our complete mobile api and everything an app would need to manage our data. By crafting something in my spare time over 6 months (at work though, of course no overtime!) I could focus on doing it clean and making it simple to use without any black boxes. Often there isn't time to do this thoroughly due to pressures to get it done quickly, which is what usually leads people to just sling together frameworks and hope they function.

It worked perfectly, we were able to ship on iOS 7 launch day and appear in the famous 1 second slide at the keynote. The other three programmers had no trouble building on top of my framework, and since then it's been totally stable. It wasn't some genius thing, just careful crafting of a framework with nothing not needed, well designed and tested ahead of time. I knew what every line did.

One of the problems with open source frameworks is that sometimes they offer so many features but you only need a few. Then as you add more frameworks your code subtly becomes more and more brittle without you realizing it. As problems appear it becomes hard to fix them. There is a trade-off between building on the genius of others, and piling too much stuff on the camel.

Yes, I haven't forgotten the camel.

Everything has a maximum load, whether it is a bridge, a brain, an atom or even a camel. Put a little too much on them and they become dust, fried, neutronium or a dead smelly beast. There is always a balance between taking advantage of other people's work and making things way too complex. You should always try to either be careful how much you build on but still understand what you have. Finding that point takes thought and judging when to build on and when to build it yourself isn't some quicky decision.

Today everyone wants to build everything at maximum speed. People still dream of snap together software, so easy even a salesperson can do it. I remember the blather than Sun put out when EJB's were first discussed, how in the future everyone would buy EJB's and build apps without programming by simply combining them. I thought it was pretty funny at the time. Over the decades this kind of idea seems to appear on a regular basis.

Programming is hard but you shouldn't make it even harder either by trying to glue too many things together or building software at such a pace you never have time to understand what you have.

One of the things I don't like about Scrum is there often isn't time to consider what you are doing and why. In the rush to finish user stories in time for the iteration contemplating what it being done is generally not part of the plan. Often user stories and application design have little in common and the order of construction may not be what a product manager demands. It's a different case of creating complexity, this time by forcing designs to not grow naturally, and not be periodically (if not continuously) edited to eliminate crap. In the rush to get things done we pile more on top of the poor camel, never thinking it might be about to croak. I don't really like the term technical debt, I'd prefer to think of it as avoiding editing and criticizing your work until it's too far gone to fix.

In the end what you want is to create quality software in a reasonable time, and with an eye to future changes. Balancing the need to do it fast and with fewer resources with the time and thought it takes to do it right isn't always an easy path. Trade offs of understand the whole of your code's parts plus taking time to decide if what you are doing makes sense is balanced against the "do it quick, do it cheap" mentality that pervades many companies and teams.

So often we wind up with camel crap for software instead of a nicely balanced beast of burden (and terrible alliteration).

So see if you can take a look at what you are doing, and decide if adding complexity or rushing the edit process is really making your end product better, or simply making your life in the future more miserable.

Be nice to your camel.