After reading Paul Grahams latest post Why to Not Not Start a Startup, I thought about my own software startup and figured it might be interesting to write it up. Doing a software startup today is a completely different animal than it was for me, but the desire hasn't changed much.

At my first job at General Dynamics I was the "boy wonder", the young guy who understood "PCs" when it was a mystery to everyone else. I had one of the first production IBM PC/XTs (whopping 10MB hard drive), a Lisa, and got to test some oddball new OS called Unix. One time I had two GD VPs buying me pizza at 4AM while I coded something for the IBM President. But in 1984 when the Mac was first released I got the itch to do something on my own.

Late in the year I quit and started working on spreadsheet templates for the oil and gas industry, working in the shared office of a friend of mine and some other folks. This proved to be not a great business (most of the folks in that industry didn't have PCs) but one day I had a great idea on how to build a spreadsheet program that was easier to work with and could keep you from making formula errors.

Now in 1985 starting a software business was much more of a pain that it is today. No web, no email, you could only buy software in stores or by phone, the only marketing messages you had to work with were ads in computer magazines, trade shows and user groups. Everything cost money, so raising capital meant working friends and associates for money. Eventually I raised enough money by selling stock and I got a couple friends (Bob Murphy and Ken Clark) to start coding the new application. Most of the money people were friends and business associates of my office mates and one of them acted as the CEO.

One of the coolest things I got to do during development was to go to Apple's first full developer's conference in 1986. Every one in the entire Mac world was there, Apple rented a boat for us to tool around San Francisco bay in, and we all joked that IBM (the evil empire at the time) could kill the whole Mac market with one torpedo. It was fun to be one of the group.

After working on it for 10 months we all went to the Boston Macworld show in August 1986 to check out the competition. In those days if you wanted to see other applications (and didn't want to buy copies) the only way to really see what other apps looked like was to go see them at shows. Demos cost money since you had to send them out on floppies (seems like the stone age now). I discovered my UI for the program (now christened "Trapeze") just plain sucked. So I started working 100 hours weeks for the next 4 months to totally rewrite the UI in time to ship it at the 1987 Macworld show in San Francisco. I lived on Jolt cola the whole time.

Trapeze was basically an object-oriented spreadsheet, where formula relationships were only by name and not by position. Instead of the fixed row and column grid it had free-floating blocks of data with real names. Thus a formula for a block named "profit" would be "sales-expenses" and the program dealt with the dimensions. Everything could be styled, formulas could generate charts, and blocks of text and imported graphics along with the data blocks could all be moved around freely.

The interface I came up included a feature that as far as I know, was first seen in Trapeze out side of some early Unix uses. The popup (now called dropdown usually) and the hierarchical menu seem so normal today, but there was no support in the MacOS yet so I had to roll my own.

In those days in order to sell software you had to deal with distributors and mail order firms in order to sell anything. Ads in magazines, show booths and press favors all cost tons of money. Our first orders were big and things seemed to be going well. Then our first review showed up in MacUser.

Kaboom, our bubble burst and the slide began.

The review was written by a guy how had had a bad day when he wrote that review. Later on we actually had lunch with him (long after the end of this story) and he admitted being angry at something that day and took it out on us. Unlike today reviews were the major way anyone heard an opinion, there wasn't any easy way for actual users to comment on products. So this one review (all the others were positive) put a dagger in our sales.

Our biggest selling issue was that you couldn't import Excel or Lotus 123 data, since it made no sense to convert. Many of our users were passionate about the advantages of our approach (one guy even built up a solid practice analyzing businesses, inputing the data into a worksheet which then constructed an entire business model and glossy report, which he printed on a laser) but we couldn't get enough of them.

In May 1987 I was asked to appear on a nationally syndicated computer show (Computer Chronicles) and got to demo Trapeze alongside Excel with its product manager from Microsoft. I was able to scoop them a bit by getting an unreleased (at the time) color monitor and Mac from Apple, and coding up Trapeze to support color (8!). You can watch the show (and see me in my finest white pimp suit).

By the end of the summer it was obvious we couldn't sustain the business and it was shut down (in rather ugly but private fashion) and Trapeze was sold to another company (Access Technologies, which later split up and part became Deltapoint). Trapeze did ultimately sell about 13,000 copies.

I gathered the engineers and support folks together and we started a Mac consulting business, which went on to work on portions of Persuasion (for a guy named Peter Polash who sold it to Aldus and made a fortune) and Deltagraph (still kicking today).

Eventually we all went our separate ways, Bob is currently at PalmSource and Ken works for the show lighting company Vari-lite. I am still full of ideas, and as you can see above am looking for more consulting gigs (hint hint) and writing this blog.

So why do I consider it a success, even though it failed? It was worth the risk, the effort, and a lot of fun (mostly). Today I still get emails from Trapeze users who miss the application (even I miss it). I learned a ton about programming. I still have lots of ideas I couldn't never have imagined had I stayed at General Dynamics (now Lockheed locally) and just been a drone.

So like Paul said in his essay, do a startup. No matter the result or the difficulty it's worth doing. I was 27 when I started but age, especially today, doesn't matter. With web you can start with almost nothing and still make a difference.

And maybe people will be sending you emails (or brain-to-brain messages or whatever is popular then) 20 years from now about how much they loved what you did.

My Tags: startup success trapeze

If you believe the various statistics, 70 or 80 percent of all software projects fail or are cancelled (for example, a little old but still relevant). Reading this kind of thing always makes me wonder what the definition of failure, and thus success, is?

One definition could be, any project which is not cancelled, and is in active use by someone, must therefore be a success. It's not very satisfying; it's like defining life as the absence of death. If I am technically alive, but my brain function is zero, it's not a very useful existence. I have seen many projects which were developed, put into production, used by a large number of people and actively updated, yet strategically speaking are destroying the company's future. If the software is buggy and unstable, is difficult to update based on the needs of the business, hurts customer retention and new business by its inability to function properly, and wastes large amounts of employee work time, then saying it's a success has a hollow feeling.

Yet in most companies today, there live many software systems which exhibit exactly this lame definition of success. Almost everyone I know who uses a computer system as part of their daily work (and are not developers themselves) tell me stories of software that just doesn't work. Workarounds, frustration, inability to get work done, poor interfaces and frequent downtime are common complaints, despite the often mission-critical nature of many of these systems. Every such application had managers, developers, architects, testers and above all decision makers involved in the process, yet the result was still crap. I bet too that many of these systems were at some point considered a "success" as they were rolled out to customers. But are they a success, simply because they are in use?

No, and they shouldn't be. In many places, however, it may be the only type of success you can get; the organization may simply be incapable of doing any better. What an ugly truth! You (of course not me we all say) write crap because you can't do any better.

Many projects are cancelled long before they reach real users. Yet are these to be considered failures? In most cases canceling a disaster before it happens is actually a good thing if it leads to a better system. Sometimes projects are cancelled or never even started due to poor decision making, fear, expense, or other reasons not related to the actual development. This of course muddies the definition of failure and success. Is canceling a bad project a success or finishing a pile of crap a failure? It makes your brain hurt.

A better definition might be, any project which is not cancelled, is actively in use, continually meets the needs and strategic goals of the organization with frequent updates, maintains a high degree of availability and stability, and no one complains about it, must be a success.

Sadly it's tough to find applications like this in most companies, so success might be an elusive quest. When you really look at what software is supposed to do, this definition makes sense. I need software to function to do my job, my company needs me to do my job, and the company's customers needs my company's services or products. If these goals are being met (and are able to change with the market) then there is a measure of success. Of course a lot of software is not strategically necessary for a company's business but ultimately if the business is affected even subtly then success still matters.

Writing successful software is hard work, but sometimes working on live failures may be even more difficult. One of the most difficult decisions to make is when to toss out the old and start over. Do you stick with the crap you know, or risk developing new crap you don't know as well? My philosophy has always been "when in doubt, throw it out" but that isn't so easy in larger systems, where the risk of a rewrite may be more expensive than dealing with the existing failure.

As an example, in 1993 we were finishing up our 5th major update to Deltagraph. We approached the publisher about starting on a rewrite in C++, as the original source was then 5 years old and written in C (with some object extensions that embarrass me now). With the company (at the time, Deltapoint) looking towards other products the decision was made to stick with the code, and we parted company at that point. Now 13 years later the current owner is still saddled with the same codebase, which makes updating (especially the Windows version) a living hell. By now the code has seen 100 engineers and 18 years of development, so I can only imagine how awful it is to work with. But in this case it is still a success, as it has provided customers with a superior product, and the companies that owned it (Deltapoint, SPSS and Red Rock) with a lot of revenue over the years. Would it be as successful if we had rewritten it (or for that matter anyone since then) in a more modern language?

So even my example is a muddy definition of success, the code sucks, its old, everyone hates to work on it (I suppose) but it still meets the needs of the company and the customers.

It looks like even defining success is a failure. The nature of our work as programmers is surely an odd way to make a living.

My Tags: it success disasters programming

I write a lot of posts on software project disasters and stupidity which can be both instructive and entertaining. It's time to begin a series on projects from the other side of the coin: they actually succeeded.

The first post in this series covers the development of the Consumer's Digest Online website in the late 1990's. Despite the success of the project itself, don't both looking for the site, as CD Online failed to pay their bills for the development and hosting of the site in 2001 and it was killed. The magazine still exists but has never built a replacement website since as far as I know, the owner of the debt may still have a valid claim. So in a way the project both succeeded and failed.

The company I worked for, Tensor Information Systems (a consulting firm which last quite a while but ultimately died in 2001) received a contract from CD Online to build an extremely challenging website and product search engine. The core idea was to provide the customers on the web the ability to search for products based on wide-ranging set of criteria, with the ability to find vehicles that could be configured to meet the criteria as the major feature. The key to the vehicle search would be to allow the user to specify features, and then have the search engine attempt to find cars that could be configured based on the auto maker's own rules for packages and features. This had never been done (even by the auto makers themselves) and as far as I know, hasn't been repeated since. This feature had scared every firm that CD had approached (one had actually tried to do it earlier and gave it up after a year of effort) but our firm was aggressive (and possibly foolhardy as well). The magazine had printed in the magazine that the site would be available in November around the time that we accepted the project, which gave us a six month development window. Unfortunately contract negotiations whittled the time to an almost impossible 3 months before coding could actually start.

Since our company had no actual experience with search engines we hired a contractor who supposedly had some experience while the rest of the team worked on the remaining site features. The technology we used at the time was Apple/NEXT's WebObjects written in Objective-C. We would be hosting the site ourselves on HP/UX and using Oracle as a database. CD Online would be responsible for entering all the data from their product databases in addition to all the data the vehicle manufacturers provided (model information, feature/package rules, pricing, etc) once we had built a data entry tool.

I wasn't a part of the team to start with.

The contractor we hired wasn't up to designing the core search database so one of our senior people was dumped into the project and had to design a database on the fly so that development could continue. The core problem was that the rules determining what feature and packages were combinable for vehicles were complex and not easily coded into a relational database. Basically features (like leather steering wheels, heavy-duty suspensions, engines, etc) are combined into packages (collections of features) that can be combined according to rules (package X requires packages Y and Z and optionally G or E, but disallows packages Q and T). The rules were inheritable, so if you had package X and required package Z, you would also require whatever package Z required. Some vehicles had as many as 10,000,000 different potential package combinations. The rules existed for both the obvious (you can't have two engines) and the sublime (you could only have leather seats with sport mirrors).

The database would ultimately have ten's of thousands of features, products, packages and their rules.

Building a search engine for such a fuzzy set of rules coded in a relational database seemed impossible (to all the other firms who had turned down or failed to build this). The contractor we had hired to build this core technology delivered some working code halfway throughout the short schedule. It did a preliminary search of the database and then sorted through the results in memory, which worked well for him in his development database which had only a few items in it. Once it was run on the growing production database (data was still being entered) it became unusable. Running on production hardware it took an average of 70 seconds to return a single set of results, and often the results were completely wrong. Now the project was six weeks away from being public and the core (and star) feature was unusable. Bad karma. He was fired.

Our resident DBA (not really a designer) and the architect (who had designed the database in a hurry) now tried to optimize the SQL query at the core of the problem, which when printed out was 5 pages long and appeared to to be the main culprit. No matter what they tried nothing really changed. It was clear that a redesign of the database might help but there was no time since so much of the code (and the data entry) was already complete. The hardware was already purchased and setup, and adding 10x hardware was not affordable to CD Online (and in case would not drop the response time to an acceptable level anyway).

I shared an office with the architect and overheard the wailing and gnashing every day (I was in charge of a project for the post office) and offered to help around 4 weeks before the deadline. My idea was a way to solve a problem where you couldn't change the code, the database design, or the hardware, and do it in only 4 weeks. It took some mighty convincing arguments before I was even allowed to investigate the idea (they continued to try the optimization of sql). My guess was that even though the number of combinations appeared almost infinite, the might be a way to "predigest" the data in such a way that it could be search entirely in memory instead of in the database and bypass Oracle entirely.

My first plan was to read the requisite data from the database and then do some statistics on it. Basically the rules data resulted in a tree of possibilities (and/or/not) for each vehicle model (non vehicles were also in the database but rarely had rules for configuration). With hundreds of models and about a dozen manufacturers the trees appeared infinite but I found I if could winnow them down to sets of recurring subtrees, the tree of trees could be represented fully in memory in around 20MB. The data was sorted by price (so as you went down a tree path you knew the price of the vehicle based on the current configuration) and only sufficient data was kept to allow one to run the search; at display time Oracle was consulted to fetch the actual displayable detail.

The key here was to extract the data from Oracle weekly into a set of static files which were digested into a form which could be read into memory when the application launched (actually into each instance). Then I wrote the actual search engine in C that traversed the trees based on the existing code that collected the user's requests. Another complicating factor was the the HP/UX memory allocated was horrible on deleting objects (I had written a commercial memory allocator library and knew all the tricks) so I created a suballocator for my own use and pre-allocated enough space at launch for both the data and the searches to avoid the HP/UX overhead. After the search ran I would grab all the data from Oracle and pass the result back to the existing code which displayed it to the user.

All of this happened in four long weeks of work. It worked, returning searches in less that 1 second on any set of criteria (which were extensive) and no code changes were necessary in the rest of the application so we were able to start serving CD Online around the original magazine promise date. The website was an immediate hit and would serve around one to two thousand users simultaneously on most days.

The most amazing thing to me (other than this worked at all!) was that no bugs were ever found in the runtime search engine during its lifetime. Later on to increase the number of instances available to serve the site, someone else built a shared memory version of the data so only one copy would load (it was static); this allowed us to double the instances without adding more RAM.

Sadly CD Online ran up a bill of around $800,000 before Tensor pulled the plug. This killed CD Online and ultimately contributed to Tensor's downfall as well. Since then a lot of vehicle search engines have appeared (think vehix.com for example) but no one has attempted anything like ours again (I could be wrong).

The lesson to be learned from this project is that sometimes an impossible problem can be solved, if you only look differently at the problem. Another lesson I learned is that fuzzy data and relational databases are a nasty combination, and that often RAM is your friend. Today, I would probably look at a distributed solution (think Google) but in 1999, and facing the limit of unchangeable code, database and hardware), this turned out to be the only successful solution.

Next time: Sabre save a bundle.

My Tags: programming development success database