Reply to comment

Why admissions interviews should be banned

May 9, 2008

An important part of the admission process to a competitive college is the admissions interview. I'm against it. And that isn't just because interviews were originally instituted to keep Jews out of Harvard. It's because they are poor predictors of future performance and, even worse, they are poor predictors that people weight very heavily.

I was first clued into this by none other than Google. Google recently revamped the way it chooses new hires, and an important part of the overhaul was minimizing the importance of the interview. As Laszlo Bock, Google's vice president for people operations said, "Interviews are a terrible predictor of performance."

This stands to reason. We all know people who make great first impressions but then turn out to be lousy employees/students/friends/etc. Similarly, we know people who originally struck us as dull but turned out to be our best employee/student/friend/etc. However, it would be nice to have something quantitative to back up this observation, and so I've been on the lookout ever sense.

It is in this context that I read this following quote from a classic Science paper by Tversky and Kahneman:

It is a common observation that psychologists who conduct selection interviews often experience considerable confidence in their predictions, even when they know of the vast literature that shows selection interviews to be highly fallible. The continued reliance on the clinical interview for selection, despite repeated demonstrations of its inadequacy, amply attests to the strength of this effect.

Tversky and Kahneman probably did not think this was a problem with the clinical interview per se. They give several other examples, including a study in which participants read a short description of a particular lesson a student teacher gave. Some participants were asked to evaluate the quality of the lesson, giving it a percentile score. Others were asked to guess the percentile score of that student teacher's overall abilities 5 years in the future. The judgments in both conditions were identical. That is, the participants believed that the quality of a single lesson fully predicted how good a future teacher would be. They don't take into consideration that the student teacher might be having a bad or good day.

Tversky and Kahneman have an explanation for why people care so much about interviews. Across the board, people believe that small samples are much more reliable than they are. I recommend the original paper if you want the full argument, but they bring up many examples. For instance, participants believe a random sample of 10 men is just as likely to have an average height of 6 feet as a random sample of 1000. This is not mathematically possible, but even experts in statistics can, under the right circumstances, fall for this.

This is why I think the admissions interview, as well as the job interview, should be scrapped. It takes place over a short period of time, which means it is an inherently unreliable predictor of future performance. It's unreliable, but, even knowing that, the information gleaned from it irresistible.

Tversky, A., Kahneman, D. (1974). Judgment under uncertainty: heuristics and biases. Science, 185(4157), 1124-1131.