Discussion about this post

User's avatar
Paul B.'s avatar

Oh wow. I sometimes laugh to myself about the 2000-2010 psychology community believing in cute unintuitive phenomena that were later shown to not replicate. I figure that I'm good at applying common sense... but that cute WEIRD result on illusions totally fooled me. It was so cool (and Henrich is so smooth) that I honestly didn't even consider it wouldn't replicate

Expand full comment
Sean Trott's avatar

I’m quite sympathetic to Heinrich’s argument but I think this is a reasonable and well-argued perspective!

I suppose it might come down to whether our starting assumption should be that a given behavior or phenomenon generalizes or doesn’t. Ideally there would be a clear set of theoretical principles to determine this, as well as a good set of heuristics to navigate the trade offs. Lower level behaviors like visual perception seem like good candidates for “assume more likely to generalize” than something like ethical values, as you point out.

FWIW, there’s a very similar problem in research on LLMs right now, and my view is that generalization is too often assumed when what’s studied is a small sample of models trained on English data.

Expand full comment
16 more comments...

No posts