What the Hell Is Really Agile?

The Agile Manifesto is one of my favorite things in software development, because it just makes sense. What Agile has become since then is another story.

Over the years (and long before the Manifesto) people have tried to come up with many ways to try to make software development more functional, more productive and more in tune with what their customers need. Developing software has always been a challenge and remains so today and everyone wants to make it easier or faster or more predictable.

Despite the Manifesto I don't find most of the so called modern Agile techniques really match up all that well with the Manifesto and its principles. After all, if any of them were superior to the others, why would we still be coming up with new ones? To establish why I think this way, a bit of history.

I've been a working programmer since 1981. My first project in my first job was run using what today you would call a Waterfall, strictly step by step. Waterfall of course was coined later in the decade as a poor description of the original article by Royce in 1970, but the idea goes back to the mid-1950's.

That's the first and last project I ever did using Waterfall. I am constantly amazed at how many people assume there was nothing but Waterfall before whatever Agile process they follow. Most of the projects I did at that first job where quite iterative, though in a haphazard way. In fact I had no idea during the 80's that studying how software was made was even a research topic!

Starting with my first startup in 1986 or so until 1994 I led three mainstream desktop app product teams; Trapeze, Persuasion (only a subteam leader for this one release) and for 5 years with Deltagraph. These totaled 12 major releases, with Deltagraph having 100,000s of customers. I was the team leader and of course also a full time programmer on the teams which had 3-6 total programmers depending on the product. We didn't do anything remotely Waterfall. We did what made sense to get the job done. We did what today would fit quite nicely with Agile principles. We had no clue such a thing existed of course because it didn't yet.

Today I would describe what we did as mostly Lean, though of course missing all the fancy terms and buzzwords. Development worked from a simple list of desired features (of course called stories today), we did constant integration (manual in those days but daily builds), we had constant testing of the entire application (daily), we talked constantly and refined the feature list by doing a lot of prototyping. Nothing fancy but it worked well. 11 of the 12 releases were duplicated on floppy disks (yeah those old plastics things); Persuasion was delivered to the primary author who had Aldus as the publisher. Especially in the Deltagraph years it could cost nearly a million dollars to ship a major release so the publisher had to charge for it. Not once did we have to reship, each release had a high quality to it. Deltagraph's publisher's product manager and us worked closely together to ensure that the market got what it wanted, even if we had to add or remove features near the end of a release cycle.

Even after that period I haven't experienced any Waterfall type projects. I think Agile is a natural way to develop software efficiently, and if I had to give a name to what we did during those 9 years I would call it "Natural". It made sense to us then and to me still today, which is why I like the Manifesto so much.

But when I look at many of the agile techniques people like to tout these days, I don't really feel any love. It seems people have to make things more complicated, more fixed process, more buzzwordy (if that's a word) and less natural. It feels as if people don't trust programmers to do the right things and decide that being more formal and organized is the only way to ensure people are more agile. But that makes no sense, how can you force people to follow specific processes and do things in a formal way and expect something nimble to come out of it?

Scrum is my least favorite idea and as far as I am concerned it's barely agile at all. Iterations are not natural, lots of meetings disrupt the flow, too much planning makes for too rigid a development process. I do think Scrum would make sense for a team or company that actually followed Waterfall (or its lipsticked pig version SDLC). Compared to that rigidness Scrum would seem far more agile. But does it really work for everyone? No, but of course you are free to protest that it works for you. That is not any proof it's the only way to develop software.

Kanban, which we tried last year for a full new project, makes a little more sense since you do have a less restrictive workflow than Scrum. But it can bog down larger teams or teams with few testing resources (which we lacked) as the flow gets gummed up with too many things in progress that keep changing. We made it work by relaxing most of the restrictions since the project was CEO-driven so it could get done, and in the end it probably barely represented Kanban. Kanban came from automobile manufacturing as a way to optimized ordered parts, and I suspect makes more sense for mature products where most stories are fairly self contained.

Extreme programming actually predates the Manifesto (as does Scrum) and has probably changed the most over the years. I've liked some of the concepts behind this far more than the practical implementation which has many things that make no sense to me at all. Things like pair programming (which feels like a mighty waste of resources), requirements as tests, and constant redesigns while coding sound like a mighty messy process that would only work in an extremely devout and dedicated team. XP features are popular to try to merge with other agile ideas. I guess they don't work as whole all that well in practice.

Lean is still the one I like the best, as it feels more like my "Natural" process. To me it seems more Agile than the rest. Anything which has too many pieces that all have to work perfectly (if you can even describe to anyone what that means) is going to be hit or miss as to whether it actually improves the making of software.

I've always assumed, and still do today, that the most important factor in quality software development are the people doing it. And if the people are good they should be able to find a way to work with the product owner or customer, each other, and anyone else involved in the making of software in a way that gets the work done best.

Management never agrees with that last paragraph, and imagine that putting in more complex processes will somehow make things work better. No one ever got promoted assuming that less control was better than more, fewer buzzwords were better than lots, or that smart people will be able to figure it out correctly.

One thing that has always bothered me is that nothing in software development is really ever a proven idea; everything is usually proven by someone's anecdotal story or maybe something you saw at a conference or read in a blog post. Given that people are still trying to find out the best way to develop software even after the Agile Manifesto means that nothing we come up with is likely to work for every project, every team or every company. You have to find a way to make what you do work for you. What works for me is just a suggestion in the end.

As usual people like to find that silver bullet that magically makes everything perfect. After all the Agile Manifesto is an idea, and how to embrace and implement that idea is not as simple as those few lines of text. Just saying Do Scrum And It Just Works, or XP Makes It Better or even Lean In For Better Success are not shiny bullets but attempts that may or not work for you or me.

If I were a betting man, I would say we are still going to come up with more ways in the future, and what is a big deal today might be yesterday's Waterfall. After so many years that's one bet I would take.