That Time I Trashed The Company Mainframe, And The Lesson I Learned

I started my career in the fall of 1981, and in the first six months, I wrote a Jovial language source code formatter in Fortran. I had no prior experience in programming jobs or college programming classes; all I knew was what I had taught myself at home.

That formatter was the only complete Fortran codebase I had ever seen, and I wrote the entire application. It ran on the Harris Super-Minicomputer. It would eventually be used to deliver source code to the US Air Force, but not yet. We had no Jovial compiler; we just had a syntax checker.

I worked for the world's largest defense contractor, supporting the F-16. They were contracting with a vendor to produce a Jovial compiler, assembler, and linker. My second (as far as I recall) project would be testing an early version of the assembler, which was written in Fortran and ran on the IBM mainframe. This would be my first use of the mainframe, and as it turned out, it was also the last.

Since I had no idea how to test an assembler, and no one gave me any hints, I decided the best thing to do was to write a little program, run it through the assembler, and see what happened. The assembler supported the MIL-STD-1750A spec processor (to be used in a future F-16) and the Zilog Z8000 as an interim processor. I was familiar with basic assembly language, primarily Z80 and 6502, so I put together a simple 20-line program, used the necessary JCL, and submitted the job, assuming I would see the result the next day.

Later that day, my phone rang. In 1982, we had no email (executives did, but no one else); therefore, we all had a phone as our primary communication device. When I picked up the phone, all I heard was a lot of swear words and yelling. The IBM mainframe operator was screaming at me for submitting a job that caused his operator console to overflow with errors. He was acting as if I had trashed the entire mainframe and made his life a living hell.

To this day, I have no idea how a user application could do such a thing, but I don't know much about the system they were running or what could have triggered such a mess. He ended the one-sided conversation by telling me to never rerun this app before slamming down the phone.

Well, that was not fun.

Now, I was expected to write up a bug report. The problem was that I had no idea what a bug report should contain or even what it should look like. No one gave me any hints, so I naively assumed I had to figure out what the bug was!

I couldn't rerun the assembler; there was no debugger (and likely I had no idea what that even was), but I did have the source code. So, I printed it out and decided to go through the entire application, trying to assemble my small program in my head, line by line.

At some point, I found it used a hash map to store labels, but I had no idea what that was (due to my limited education), so I went to the library and read up on that data structure.

After a couple of days, I puzzled over the hash function. It appeared to be either an endless loop or infinitely recursive (I don't recall today what it was); in any case, once you got in, you never left. I presumed that would cause an issue (again, I had no idea how that would affect a mainframe) and wrote my findings on paper (no email again; apparently, someone would collate bug reports and mail them to the vendor). I imagine the vendor was surprised to see the bug report had a bug documented in it!

After a while, we received the next release, and I called the operator and told him that the bug was probably fixed and how I should run another test. He told me to submit the job, and he would clear the mainframe before running it after hours. He was much calmer this time around. He ran it and everything was fine.

What was the critical thing I learned from this? It had nothing to do with the mainframe. With my limited experience, I could read and understand a relatively complex codebase that I had never seen before and identify a problem with it, simply by reading it. I discovered the importance of reading source code. To be a good programmer, merely being good at writing is not enough.

Today, of course, reading source code is an important skill to have. Code reviews are common, yet in my experience, many programmers gloss over them and try to complete them as quickly as possible. Often this results in LGTM. Generally, you don't want to code review an entire application. Still, sometimes you need to validate a potential codebase, such as in an open-source project or a vendor-supplied library. This skill is essential to be a good programmer.

I would like to know if Computer Science programs spend as much time teaching how to read code as they do on how to write it. After more than four decades as a programmer, I know how valuable this skill is. Not only is it useful on its own, but it can help you write better as well.

That is the lesson I learned here: reading source code is essential, and I could actually understand a codebase I had never seen before. Confidence-building things like this really helped me move forward in becoming a more professional programmer. Confidence like this helped me a great deal when I faced things I thought were impossible.