Interviews Can Be a Terrible Way to Identify Good Programmers

After reading this article and comments, I really really wanted to laugh. Or maybe cry. Or even throw up.

In a nutshell the author of the article seemed to think you should hire a programmer on the spot if they went way overboard on the coding tests and other challenges, like the guy who at the end of a two hour coding test was seen feverishly adding HTML to his Javadoc comments.

I wouldn't hire this person under any circumstances. No, really. I've hired and interviewed enough people in my life that I want nothing to do with someone who fails to comprehend reality. Heck I've never made anyone write code in an interview and probably wouldn't hire someone who demanded to do so.

I know some of you might think I'm nuts and offer the old argument "but you wouldn't hire a carpenter without asking him to hit a nail". My point is what are you hiring them for? If you want someone to build a house, skill at hitting a nail is almost immaterial. Sure it's nice to know they can smash one into a two by four with one blow but that's a far cry from building a house. The skill set needed to build a quality dwelling far exceeds what you can tell from a tiny sample of a tiny portion of someone's ability. Unless you are hiring people to write short pieces of code, what's the point of testing them on that?

You can ask me, but how do I tell a good programmer from a terrible one without a coding test? Shouldn't I see how they work on a problem or if they are sharp enough to see a piece of code that has a bug? Sure, go ahead if it makes you feel more comfortable, but it won't tell you squat about how good they are in a year long project with 10 other people, writing code in multiple languages or environments, dealing with adversity, taking charge of difficult problems and working together with others. Your careful interview testing will tell you almost nothing about that. So why spend an inordinate amount of time on relatively minor parts of a programmer's skill set?

I've said it before in previous posts but give me an hour one on one and I can tell more about how a (senior level) programmer will be in the long run than all the coding tests in the world. Why? Good programmers talk like they work when you engage them in details of past projects they spent months on. You can't fake something you invested a good chunk of your life on. A good programmer learns from errors they did or others did; they remember how they dealt with some nasty design problem or fixed a complicated bug. Good programmers who learn and grow over time have good programming in their DNA, and love to talk about them with a peer. Crappy programmers evade and toss out bullshit or forgot what they did. They don't remember what other people on the team did or the environment around the project. Good programmers see everything; bad programmers forget everything.

Even after 30 years in this business I can still remember details of projects going back decades, especially those that I spent a lot of time on. Everything I've done is engrained in my mental DNA. Lessons I learned from the very first program I ever wrote (in a high school computer class on a teletype machine with a paper tape) are still with me. With all the good programmers I've known over the years you can hear the same thing if you listen to what they say and how they talk. The mediocre and terrible ones all sounded the same to me.

When I hear about 2 days of interviews at some big companies I have to laugh. Have they done any testing on effectiveness? I wish someone at Google, for example, would take a few positions and interview two ways (or more, heck they have plenty of time and money); one the normal massive overkill method, and another, meeting candidates with a couple peers at a coffee shop for an hour. Then make decisions about both positions and track the quality of the actually job performance and contributions for the next year. I bet there is little difference. But the normal way costs a lot of money and time, and the second costs spare change and almost everyone has a good time.

Now I am not saying you should not have some pre-qualifications like you do today, I'm talking about the actual interviews. Today for sure you get a lot of random people for every job posted.

My favorite idea is still contract to hire everyone after your (hopefully reasonable) interview; that way you can see how they do real work on real projects before you commit. Since you can't spend a few weeks at their former job position seeing what they did (like you could do with the home builder) this is still the best way to get a real picture. Programming is not about writing lines of code, it's about completing projects or shipping applications or whatever you do and generally working in a team. None of that is evident in any kind of interview test or whiteboard session.

Not every programmer has to write perfect code or invent Einsteinian algorithms. You're not trying to recreate the Avengers where everyone is a superhero. You want a team that makes your business better, your customers happier and makes more money or whatever is important.

I remember a long time ago going to the company gym to play a little basketball and there was a team trying to find some people to play a practice game against. They were all clearly amazing athletes. Me and 4 other random guys agreed and we played a full court game. We killed them despite being rather average in ability, because somehow we functioned as a team on both ends of the court. The better team lost because despite their individual prowess they really didn't play together at all and actually got in each others way. I've always remembered the feeling of what a team really is. Yet if you'd interviewed the 10 of us and made us perform basketball skills neither me nor my random teammates would have been considered. Sometimes what you want in the end isn't evident in a single moment of time and you should never think you can somehow milk that one moment to get the perfect programmer.