I worked for a games company that gave the candidate a two-day interview where they actually had to write a 3D game (albeit a small one). It proved to be a very successful way of filtering out those that could actually do what they said they could on their CV.

This remind me about my last interview last week, they asked me if I were ready to develop a poll system in 40min. They notice that I could not work full time so they did not hire me, even that they had my resume before the interview, they did not read it :S

If you're a talented programmer, then having an interviewer ask you to write code to reverse a string is irritating. Unfortunately, the interviewer gives you this question in the knowledge that about 70% of candidates (regardless of experience level) will be unable to answer it.

So written programming tests are a bozo filter, sure. You know you're not a bozo so the test is annoying. The interviewer doesn't know you're not a bozo (yet) and a simple programming test is a good way of finding this out.

Some examples from our test:

Less that 20% of candidates can spot the problem with this OO design

class Car : public Engine {};

Less than 60% of candidates can spot a memory leak in a five line C function.

About half don't seem to know what a static member variable is.

I find tests to be quite useful. We use the results of the test to decide whether to even bring them in. Yeah, they're trivial, but they tell me a lot.

I've had people submit answers that provide correct results, but in such a terrible way that I didn't want to talk to them. Similarly, I've had people submit results with bugs, but at least showed that they understood both the problem and the technology they were working with. I can at least find out if the person is willing to write useful unit tests (I can then write a failing case for when I notice an issue).

At my previous company, we would give people about an hour to send us solutions to five really simple programming problems. I did all of them in about 20 minutes one day, with documentation and unit tests. I consider them roughly the equivalent of handing a writer an pencil and a piece of paper (and all of the internet for that matter since it's homework) and asking him/her to use a certain word in a sentence.

I want to see the handwriting. Is the code really ugly? What happens if invalid input is given to a function? Does the prospective interviewee even understand the question?

Most often, I can dismiss candidates without bringing them in and without spending more than a couple of minutes looking over what they write.

To be honest, that's sad.

Hire people that talk WELL you mean. Jeez.

Hmm. And I thought this was a local phenomenon in India.

My preferred course of action during an interview (besides pairing with them on writing code) is to ask for an opinion. If the person I'm interviewing displays an opinion on technology, any technology, thats a huge plus (he needn't share my opinion, if you're wondering :-)). You'd be surprised how many poor developers can be identified simply because their lack of interest is reflected in their lack of opinion.

Incidentally, the idea of someone having to write code during an interview 'on a computer' struck me as being odd. Why on earth would you ask a person to demonstrate his coding skills on a whiteboard? Code snippets, yes - stuff like 'show me how you override equals()' - but a full problem to be solved on a computer can and should be part of every interview process.

I read somewhere that the vast majority of interviewers subconsciously make their decision about a candidate in the first 4 minutes of the interview. All the rest is just confirmation. This suggests that the most important factor by far in getting a job is personal presentation. Whether it should be or not is another matter, but I guess you'll never change human nature.

I find some interviewers asking how can you override equals() at the same time they dont know when and why they need to override hashcode() also. I attended lot interivies in INDIA some got through and some not. The worst i performed i clicked, some the best I performed I lost. The best and clicked was ORACLE. But my observation in US is good. My best interview i faced in US is with Goldman Sachs, a Russian Manager, i did not perform well, but I found where I am behind, but got selected in another project with Goldman only, interview was not upto the stuff i have. But I support the narrator rather writing the code piece discussing on core concepts like Data Structures, DataBase core Knowledge like using Korth, CJDate evaluations is best. Writing a piece of code is something binding to a language, if i ask how can you override equals(), it is JAVA like language specific, but i expect candidate should have mobility, adaptability, that is i may use JAVA today and switch to .NET [Business Driveen changes], the person should be able to work.

I don't agree with the notion that sending people to the whiteboard is useless. I do this every time and countless time I instantly understood the level of experience they had. Things to look for:

Blatantly wrong syntax ("Experienced Java Developer" who doesn't know how to write a "main" method - never written any significant amount of code

mixing different languages / syntaxes - never written any significant amount of code. The type of "developer" who is proficient in 5 languages by googling a few "hello world"-type "applications" and playing around

Inability to apply simple programming concepts like loops or conditions to requirements

Things NOT to look for:

Canditate is nervous (who wouldn't be)

Candidate is hestitating to write code (again, doesn't mean anything)

The type of task you want to give him/her shouldn't require more than 6-8 lines of code, typically envolving loops and conditions.

Another good idea is to ask about an opinion. If you are interviewing a "java web application developer", ask if he/she prefers a rich domain model or an anemic one. If he/she goes "huh?", he/she is either not qualified or some 9-5 drone with no passion for programming.

I interviewed for a company with 4 back-to-back whiteboard interviews where I had to write 6 different puzzle problems using C, C++, or Java (depending on the interviewer). It was the type of interview that you described. I had a USB key with my own past projects that I thought would be good to show them, told them about it, and they didn't want to see them. I spent three and a half hours scribbling code on a whiteboard and didn't even get the job.

This is exactly why, at Innova Partners (Columbus, OH), we interview people in a more sane fashion: we do a quick resume scan to see if we like them and we invite them in or call them to determine basic eligibility.

Our big thing is that we then invite the prospect in for a day. We treat him like part of the team, we buy lunch for him (just like everyone else), and we give him a baseline assignment that we give all candidates. We give him the entire day to complete the spec however he likes; we're a PHP and Python house, but if someone came in and cranked out a damn fine .NET app, we'd count that too.

At the end of the day we grill them about their code. We're not harsh, I don't think, we just require the coder to be able to defend his decisions and also to be able to highlight his weaknesses.

We find that this gives us a great indication of whether someone will fit into the team and be a productive member thereof. It'll be quickly obvious if the guy isn't a cultural match for the company, or at the end of the day if he's not a quality programmer. So far our success rate is high.

Interesting responses. This has always been a sore point with me in the industry, so I'm going to toss my chips into the hat...

90% of the business development I do doesn't require me to write functions for reversing strings, sorting arrays, hashing transmitted data, giving gold to pirates, etc. So, of course, showing a prospective employer how a bubble sort works has been blown out of my buffer for quite a few years. Ask me to explain why I would use a quicksort vs. bubble sort, that's more realistic.

For business application development, I completely agree with the author, and strongly disagree with the whole 'arbitrary test' / 'my brain is bigger than yours' method of interviewing.

Simple questions about a technology go a long way. What's a reference type vs. a value type? What's a delegate? What is polymorphism? People who don't actually use or grok these concepts will never give you a clear answer, or the answers sound like they're coming from an automated phone system...regurgitated, unassimilated factoids.

Talking to someone, interacting with them and communicating with them have always been the most effective interview tool in the 10+ years I've worked in the industry. Tests, code samples, etc. are mostly worthless.

I agree that you wouldn't hire a homebuilder based on his ability to hit a nail. But ... would you hire a carpenter who couldn't hit a nail? See previous comment about bozo filter...

I personally have never really met any interviewers that where any good at coding themselves. I actually get insulted when they ask me to code on the white board now. Anyone can write code if you do it for long enough. I've worked with many programmers and rarely do you meet many that can code a hill of beans anyway. I think 85% of coders are drones and can you blame them? Who can remember all the details. I use the internet as I write code all the time, There's just too many details to remember and I don't try. It's all changing all the time, what a waste to commit it to memory. It comes down to clear logical thinking and experience. If you lack those you're going to suck at coding anyway regardless how good you can regurgitate an answer to the the standard question on an interview. I'm tried of re-writting crappy code and sometimes I just say f-it, it works. Most code will be replaced in 3 years anyway, so does it really matter how well it's written? Not really! Stupid thing is I wish I wrote more code for myself so I can retire rich like founds of Google or Ebay....get your head out of your arses you software monkeys and write code that makes YOU money...enjoy your code monkey wheels.

Not everyone has the required skills in the interview to pick someone from purely talking to them. A large number of companies are run by managers who are not technical yet they still need to recruit. So they fall back on using testing to try and find the right person. I agree this is certainly not ideal and they will fail to find certain kinds of coders that do not perform well in pressure situations but may be absolute geniuses. I recently started a series of articles on interview - http://blog.assembleron.com/2007/05/15/interviewing-programmers-101-part-1/ covers learning to interview.

Back in my seamier managerial days, I was told to interview a number of people "discovered" by the CEO trolling tech conferences. Most of what passed through my door was, to say the least, a sad reflection on the industry. (One girl was so horribly under-qualified that I ended the interview early with a brusque "You are not suitable for this position." She left in tears which earned me a permanent observer from human resources.) Finally, as a filter I told the interviewee that one day prior to the interview they would receive a programming assignment (web CRUD operations on a simple database table; 99% of what we did), and they had to bring in a printout of their implementation to the interview.

That worked pretty well, and it helped me identify the dangerous heroic programmers as well as the bluffers. One architect actually refused to do it, claiming that such a simple task was beneath him.

I was writing about this the other day (interviews and how you separate the wheat from the chaff without coming across like a monster) and came up with what I think is at least one pretty good interview question - http://48klocs.blogspot.com/2007/06/rethinking-fizzbuzz-function-z.html

It's a tough spot that we find ourselves in - short of having a person work alongside you for three months or so, how do you know that they're the right fit? A lot of people are smart enough to BS their way through anything you throw at them and no matter how many impressive bullet points are on their resume and how many projects they've been on, it's ultimately a market for lemons out there.

You can always threaten interviewees with a test. For instance, tell them to disqualify themselves if they can't complete a certain assignment in one hour, as the interview will contain a harder one-hour assignment.

Here's one recurring annoyance which I encountered yesterday: interviewers that have a very specific answer in mind, and are not open to alternate, equally efficient solutions. To wit: the interviewer wanted to know how to tally the count of 1 bits in a 64-bit number. What he wanted was an obscure algorithm involving repeatedly ANDing the number to itself minus one (which turns off a bit) until the number is zero. My solution? A precomputed 256-entry lookup table. I tried to argue that his solution was O(b) while mine was O(1), but he was obsessed with the constant factor.

Not sure I'd accept even if they made an offer; would I really want to work with people that stubborn?

How would you do a 256-entry table?

Pseudocode for that problem is

DO

IF number % 2==1 COUNT++

number>>1

REPEAT 64 times

IIRC there was actually an x86 instruction that did this at one point

In the '80s it was fairly common to show code samples - I still have the binder of green bar with several different languages in it from a tech school.

In '83 when I was a hotshot code warrior I interviewed at a nice place in Beverly Hills, gave them a sample of code. Interviewed again a few weeks later with different people. Decided I just didn't want the commute and stayed where I was. 6 months later I start a job at a security company, read in the paper about how the place I interviewed at was defrauded out of millions, an inside job, big FBI investigation. I'm thinking, gee, I wonder if I have to worry about what happened to my code... Still later, I go to work for another place, and they go to the gummint auction to buy the computer (PDP 11/70), but it gets too expensive so they buy the artwork off the walls instead. So I wound up staring at the artwork every day to rest my eyes from the C ITOH terminal. That wound up being one of the best jobs I ever had, lasted 5 years. And you know what? Never gave them a resume, at least that I remember. It was a startup, I was employee number 3, hired initially as a moonlighting designer, introduced through a contractor at that job I stayed at to avoid the commute. He had dragged me down there to see this cool new DEC color terminal...

Completely agree - writing code on the white-board and answering logic puzzles seems a waste of time.

I think you'll find a blog article I wrote some time ago says similar things and expand on it too

http://softarc.blogspot.com/2006/10/my-favorite-java-developer-interview.html

Best,

-Frank

Recently I was on an interview that annoyed me with stupid, and very shallow (not in-depth) questions. They also hadn't read the things I had sent them. First I told them what I had written in my CV, then we came to a 30-40 minute long test. The test started with the theory

-what is polymorphism

-what is method parameter overloading (the interwiewer called that "method polymorphism")

-what is a database

-what is ACID, had to explain A, C, I, D

-what is database normalization

-what is a process

-what is a thread

-what is thread synchronisation

-how does thread synchronisation work in Java

-what are design patterns

All those questions were seeking for a short answer, so anyone with a good memory could answer them without having to know these things and explain them in-depth. Then came something really annoying. It were stupid code-fragments with errors that I should find:

for (int i = 0; i = 10; i++)

println(i);

int a = ...

int b = ...

if (a > b);

{

foo()

}

some more similar questions that I can't remember now

Then the most stupid one was a recursive function looking something like this (I don't remember exactly):

void foo(int a) {

a--;

if (a == 0)

return a;

if (a % 2 == 0)

a = foo(a/2)

else if (a % 3 == 0)

a = foo(a+2)

return a;

}

println(foo(10));

What the heck was this shit! I had to work down the whole call to figure out the answer. But that is really a question for somebody at the first year in college, not for someone that just graduated.

Then came some trivial refactoring. I had to "do it better" - from instantiating using concrete types and many if (obj instanceof ...), to use the base class and replace the if's with polymorphism.

This interwiew annoyed me, because the questions were not in-depth. For example there wasn't any deeper question like "explain the advantages and disadvantages of static vs dynamic typing".

Greets, Ivan

I would hope that companies that do all day interviews at least have the courtesy to only subject interviewees to this if there is already an excellent chance they will get the job. Or maybe pay them for their time. Anything more than an hour is kind of an insult to the person being interviewed.

I have had interviews that last 3 or 4 hours where I meet with a bunch of people and then I don't even get a call or an email to tell me that they are not interested. I miss a half day of work to get interviewed by people that haven't even looked at my resume. I don't know what the answer for hiring a person is, but it could at least start with showing some respect for the person looking for a job.

I've been for a lot of sw interviews in the UK - average duration about 3-5 hours.

One place had bought a list of questions off the internet, asking you to explain/decipher very cryptic lines, similar to those listed above by Ivan Dolvich. It was too long ago (2004?) for my feable mind to provide good examples today, but included stuff like pointers to function pointers which returned pointers to char array pointers, double (and multiple) negatives, non-portable use of incrementers, and all sorts of wierd code which should clearly be broken down into multiple lines (by any sane person) as it probably wouldn't compile any more or less efficiently.

When asked at the end if I had any questions, I asked the interviewer (a non-technical manager) if he or anyone else in the sw team had actually read the questions through, to which he said "no". (Although I expect I did very well as I'd spent much of the previous week on notice-of-redundancy with nothing to do at "work" but revise K&R ) I told him that they should be very careful about hiring someone who did well on the quiz because all that it would show is how good they are at reading convoluted code - and would likely mean that they've either worked with other cryptic coders, or made a habit of writing in such a fashion themselves. I also told them that if the quiz did infact represent the standard of code developed within their company that I had no interest working there.

Interviews are a COMPLETE WASTE of time. I have been on both sides of the interview process and I have NEVER EVER seen an interview that accurately represents what the candidate is capable of doing or even his personality. The fact is most people are very nervous when interviewing because it's such an ordeal and they are not going to be themselves. How can they be? You're in a monkey suit standing in front of people who are judging you answering stupid questions and wondering what in the world these people think of you. It's very nerve racking. I try to give people benefit of the doubt when interviewing because I have been on the other side and know how it feels to be interrogated. But I think some people get a real kick out of making people sweat. It makes them feel important and even superior. Those types of people should never be interviewing anyone.

I think the best way to see if someone is a good fit for job is by what his former manager and his peers say about him. I remember hiring one guy who was very quite and reserved and never really gave us a good reason for hiring him, but we did anyway just because he had a good reference. He turned out to the be best employee! Then I've seen guys who really know what they're talking about in theory, but after they are hired you never get to see any real product out of all the stuff they said during the interview.

I say have a short interview, about 10 minutes, that goes over some key pieces of their resume and ask for 4 references, 2 managerial and 2 peers. Interview those guys instead. They are the ones that really know your candidate.

Josh: Actually, I'd hard-code it. Generating the table wouldn't take long, but it is completely independent of the execution environment, so the time-space tradeoff leans towards shoving it into the initialized static area, e.g.,

static char entry[] = { 0, 1, 1, 2, 1, 2, 2, 3, ... };

But if it had to be generated at run time,

for (int i = 0; i < 256; i++) {

entry[i] = 0;

if (i & (1<<0)) entry[i]++;

if (i & (1<<1)) entry[i]++;

...

}

Or, if I wanted to save a few milliseconds,

for (int i = 0; i < 256; i++) {

int c = i;

entry[i] = 0;

while (c) {

entry[i]++;

c &= (c - 1);

}

}

(Of course, the latter would have to be peppered with comments so the person inheriting the code wouldn't spend a week scratching his head over it.)

titan: Hiring managers would LOVE to speak with your references, except there's all sorts of legal crap that's been raised in the last decade about doing that: lawsuits for slander, confidentiality breaches, and so on. The official response I had to give at my last company was "Company policy does not allow me to comment on former workers." For the few competent ones that managed to achieve escape velocity I'd call back from home and give unofficial endorsements.

I have been in the business for over 15 years and I have seen one fad after another come in go. In Seattle, unfortunately, most start-ups and established companies take their lead from the mothership in Redmond (that is, Microsoft). If Microsoft does it, it must been cool, leading edge and correct. I have done several stints over at Microsoft in various groups (saleable products and internal business apps) and worked alongside many FTE. Are they the smartest people in the world. Hell no (despite what their PR tells the world)! Are there some amazingly smart geeks there? Without a doubt there are some amazing developers there. But they are the exception, and not the rule. The problem is one of balance. I have noticed that the majority of devs there don't have an understanding of quality and, in the case of saleable product development, don't really care what the user-experience is. This is mainly due to the segregation between the devs, product ownders and QA. The devs code to the spec without much thought for how the customer (the people that ultimately pay their wages) like the product. I'd call it a general lack of business sense and it's fostered by the Microsoft "culture". I don't blame the devs, qa or product managers because it is mainly driven from slavish requests from on-high. Get it done and get it done yesterday!

The problem is, because Microsoft is the 3000 # gorilla, other companies lazily adopt everything from this compeny. This includes the interview methodology, the SDLC, nomenclature, even adopting verbiage used internally in MS. Things such as OOF (out of facility), OOO (out of office), FTE, IC, ZBB, ZRB etc, etc. Please understand that I am not saying this is true for ALL tech companies here in the Puget Sound area (aka Seattle/Redmond), but it is rampant. The effect is stymied ingenuity and building products that can't stand the test of time. What does it matter anyway? If a company is successful, they will probably be bought out by Microsoft/Google/Yahoo. etc. It's a perverse scenario where money NOW is more important that quality and bringing truly good, worthwhile products to market.

How does all this relate to interview woes? It is this. It seems that there is a general lack of interest in the candidate as a person. Resumes are a mere formality and are seldom read. I have been on plenty of interviews where it is obvious that the interviewer has not even read my resume! How insulting is that? We'll, it's also a good clue (or a red-flag) that you've got someone who doesn't know how to interview and it's usually another developer. I know how it works. A position needs to be filled, you have got a butt-load of work to do and your manager slaps a pile of resumes down on your desk. You're not given ample time to thoroughly read the resumes and then you have to participate in a gang-bang interview. The easiest way to slide through an interview is give some dumb-ass whiteboard questions. Sorry,but I think asking someone to write a implement a linked-list, remove nodes, add nodes and balance the tree ON A WHITEBOARD is ridiculous. And, oh yes (in the case of .Net) you can't use any built in methods; you do it all from scratch because obviously, that's the way we code in this organization!

How about reading their resume? I mean really reading it. Then talk to the candidate and see what they are like. It's easy to ferret out BS artists. It really is. But know that BS artists also are masters at knowing how to "ace" tech interviews, just like they can "ace" MCSE exams! Microsoft's certification sounds impressive, but they do not make good developers. Good puzzle "solvers" and a laundry list of certifications do not a good developer/architect make (necessarily).

Next, employers (that is the actual people doing the interviews) should do some hard work like picking up the phone and talking to the references provided. If they are satisfied with what they hear, then move to the next step and engage the candidate for further talks; otherwise don't even consider bringing them in.

I do however like the idea of giving candidate a real-world coding test. I would love to be able to do this; but I've never been asked to. Have them write an app at home and bring it into the interview and have them 1) demo the app 2) explain how they did it. The interviewers would have looked at their code beforehand, of course. Now you have REAL dialogue potential and you will also see how the candidate takes criticism, check their code for extensibility, scalability and its ability to be refactored (if need be).

Unfortunately, it is only until the majority of companies become a little more original and think for themselves as to how to hire good people and stop following MS blindly that this will occur. It's that bad and I don't see it getting better.

By the way, you employers who insist on taking up a candidates entire day. Show some class buy them a nice lunch, or at least allow them to take a lunch. This is something called respect. Geez.

Anybody can hit a nail. It is the ability to hit (or miss) the nail without leaving any marks on the finished wood. I was a carpenter for 30 years before I went back to school and got a degree in MIS. They ask such asinine questions about work experience. Companies are so afraid to teach a person that it will be 20 years before they can actually answer a question that isn't drummed into them as policy. I can tell by talking to a person for 10 minutes whether they are a carpenter because I have experience. The main problem is that inexperienced HR people, business people, the bosses son, and others are involved in the hiring process. They are trying to hire somebody smarter than them but don't have a clue as to what to say to do it. Could it be that all of these companies are hiring the wrong people, thus costing them gazillions over time. What will happen when the truly smart and technology oriented take over - wait, maybe thats the problem.

In the 80s they decided to pay carpenters real low wages so many of them quit. The experienced quit and then they re-trained a bunch of new ones that are just now getting good again. But they were getting too high of a wage themselves because they for the most part have been replaced by illegal aliens. I myself, when I started was hired because I would work less than the Union carpenters that they had employed. If they people on the bottom don't know what is going on then how can the top. No wonder the Indians are able to do so much better in programming. They don't lay off the experienced hands every 10 or 15 years to try to get cheaper help. They have started doing this in the IT and Programming field now, so what do we expect. They aren't trying to hire the best person. They are trying to take your dignity and destroy your intelligence, at least in the area that is not related to programming. What they can't lower in wages here they will import to other countries. Is it any wonder that they have people interviewing you without any clue as to what is needed. What kind of an indignity is it to not even read somebody's resume first. Why do you think they insist on insulting each candidate they interview including the ones they hire. Maybe they are just looking for the person who will just take this kind of behavior until we are all working for peanuts.

This is a perennial topic of interest,and I agree that interviews today are mostly dysfunctional.

As a hiring manager who spent years coding, I do ask probing questions, but not of the 'what is the 14th parameter to GetFooInfo()', but rather open ended questions with a technical focus.

Like the author, I want to see how this person thinks and expresses complex ideas.

The rest can be picked up on the job. The tools and languages change all the time anyway.

The main question is are technical interviews just as good or better at predicting the interviewee's success at the job than regular penetrating behavioral interviews? I beg to differ if everyone say the former is a better predictor.

In the company I worked in, my VP boss never gave anyone a technical interview because he trusts your resume, and expects you to be able to learn other technicalities on the job and not be dumb. So far in his 30 years in the same company, there was only 1 or 2 out of 50 or so that he had hired and didn't work out after 1 year. That's at least a 95% success rate. He did it by looking at their degree, experience, summing up the person behaviorally and also see if they are a fit personality. That's it!

Over 95% success rate without technical interviews and these are programmers who either majored in Computer Science, Mechanical, Civil, or Aerospace Engineering who coded non-trivial software and solvers which other professional Engineers use to design the cars which you and I drive, airplanes and military jets and also the F1 formula race car which took 1st place after using the software to calculated air flow efficiency on different frame design (not to mention pace maker blood flow and other stuff). Again, all WITHOUT technical interviews. Surprised?

So why are other industries which doesn't have this mission critical software requires stupid technical interviews. Take Microsoft for example. Their software is not mission critical and their software CONTINTUES TO CRASH AND HOG MEMORY! Yet they still have intense technical interviews. What's the point then! Since they had intensely interviewed these candidates, shouldn't the software be flawless and not crash nor hog memory?! This just shows that technical interviews are not a good predictor of success nor guarantee good software engineering practices.

Now MAYBE it's because majority of the people in my company has at LEAST a MS degree in their field?... But go figure again how places like Microsoft which give technical interviews have their software crash and hog memory like cray while places that didn't have them are able to make provide critical software?