Over the past six months, I’ve been drowning in evaluation hell. I had to assess 30 students for a prestigious federal award, ranking them against each other on multiple criteria. I also had 200 job applications that needed to be sorted, compared, and whittled down to a shortlist. I did all of this the old-fashioned way—reading files, taking notes, making gut calls. And I did not ask AI for assisance.
But I’m no longer convinced that was the responsible choice. Like Paul Bloom, I am starting to wonder if not using AI in such cases is the right thing to do.
Here’s the uncomfortable truth: I have little confidence in the quality of my comparative judgments, even in my grading of a pile of student papers. I know my evaluations are skewed by whether a candidate came first or last in the pile, whether I am in a good mood or nursing a headache. I usually get a feeling about a paper or candidate and then just go on vibes. I’ve been doing this for decades, and every time I finish, I’m left with…
