The Codist

For a project I've started at home which might become a startup (but only if I can prove it makes sense to do so) I've been investigating a broad swath of server side technologies.

All I can say so far is "my head hurts."

I spent nearly a decade doing Java of various flavors including architectural stuff but it has been 6 years since I gave much thought to anything bigger than my blog. My day job has been iOS for quite a while, and before that a C++ game client.

Of course I read everyday on what people are doing outside of my area, a habit I've had since I started programming in the early 80's. But reading and evaluating are entirely different and the changes occur at a ridiculous rate so you can't ever think you understand everything.

My needs would require scalability and durability, and being me, simplicity is always something to seek since the fewer moving parts there are the easier the system will be to understand and keep correct. However the sheer mass of options is pretty intimidating. In my Java architect days there weren't that many options, a database, an app server and web server, and some apps. In today's cloud based world with needs for high availability, scalability, multi-datacenter and micro-services you rapidly outpace your brains ability to reason.

Reading what the big and not so big companies do is interesting but becomes tedious after a while as everyone uses some different combination of things and often changes happen at a crazy rate so whatever you read might be completely dead by now.

I have a number of programming languages under my belt, namely Java, Javascript, C, C++ and a little PHP, but that doesn't really help a lot; it's the other things in the architecture that become the hard stuff to learn. Where to store information and in what? How to scale over time? How to avoid losing information and providing high availability? I've looked at, played with, or read so far about Cassandra, Couchbase, Mongo, Postgresql, Redis, OrientDB, RethinkDB, Docker, HAProxy, Nginx and a host of other bits and pieces. I can't even remember them all any more.

At some point you have to pick something and move ahead, but the fear is always you pick something that causes pain later on, or that turns out not to be a good fit for your project, or has some fatal flaw you never noticed, or is hard to configure and maintain. The problem with too much choice is fear of choosing the wrong one.

Since this project is not general purpose I've even considered building a set of micro-services around an append-only database which would be less dependent on tuning a complex set of bigger options. It's a trade-off of knowing what your code is doing versus trusting the smart people who built something other folks are using that may or not work for what you want to do. It's not a great idea either to do it (partly) myself since it's not my everyday job these days. I've worked on high performance Java systems in the past but that's not a big help today.

If I were a high funded startup with tons of smart people I'd still have the same problem of deciding what to choose, but I'd have more opinions to listen to! I am in no hurry so I can take a long look at everything. Still, someday you have to say enough and start and hope for the best.

When I started programming I had no idea it would get to be so rich in choices, and so poor in knowing which ones to choose. In my first job I had a choice of Fortran and Fortran.

At least that's not a option any more!