Thanks for sharing your memories! That debate was a big deal for the discipline of psychology. I hope you now see the positive impact you had on how psychology research methods and statistics are taught and used. I know it caused a lot of discomfort to some in the department I was in, while others tackled the substantive concerns with gusto.
I remember PsychMAP and it's coconspirator Psych Methods. I didn't stay long in either for the exact reasons you mention (I didn't know about the sock puppets though).
My only quibble though is you say "the reformers have won"....I guess that's *sorta* true, but did they really? I'm less sure.
My casual observation is stuff like preregistration is still not the norm (and, of course, many papers don't follow their preregs). Even the "reformers" ignore stuff like crud/noise and the self-defensive arguments around small effect sizes that are likely false positives.
I think there's been progress, but I see it more like the rebels at the time of Bunker Hill rather than at Yorktown.
I saw someone put it recently (paraphrased and I forget who): "It's the same bad science, just with bigger samples". I sadly think that's mostly true.
In the end, do I have much more confidence in psychological science today than I did 20 years ago? I'd say not much. the main "change" psychology seems to have made due to the replication crisis, is simply to lower standards of evidence. Where (in my field) people used to brag about r = .3 (likely due to p-hacking), people simply started bragging about r = .03 with little change in message. Until we fix that, I don't think we've accomplished much.
I was also in both PsychMAP and Psych Methods. The two blur together in my mind though, and I also didnt know anything about the sock puppets. Mostly I just remember that Uri guy. He had...a strong personality (coincidently, I believe he was a personality psychologist).
I'm still unsure whether the issue is that our methods are not rigorous enough, or whether the problem is the premise that we are discovering hidden truths about human nature that are replicable. An experimental average, albeit statistically different from a control group, seems like a pretty low bar for saying you have discovered something about human nature. Perhaps the human brain does not consist of a series of effects, tendencies, and biases? Perhaps the idea of stable dispositions modulated by context is just entirely the wrong way to think about the brain/mind? If so, then simply increasing the rigor will not solve the problem
I would upgrade my subscription if the paywalled version named names
Great post, especially the Lebowski reference at the end.
If you're a fellow Little Urban Achiever, this is the right place for you!
I’m not sure that I am, but I am good…and thorough.
Thanks for sharing your memories! That debate was a big deal for the discipline of psychology. I hope you now see the positive impact you had on how psychology research methods and statistics are taught and used. I know it caused a lot of discomfort to some in the department I was in, while others tackled the substantive concerns with gusto.
The dude abides!
I remember PsychMAP! I think that by the time I checked it out, the period of high drama had mostly ended.
Great essay and thanks for sharing.
I remember PsychMAP and it's coconspirator Psych Methods. I didn't stay long in either for the exact reasons you mention (I didn't know about the sock puppets though).
My only quibble though is you say "the reformers have won"....I guess that's *sorta* true, but did they really? I'm less sure.
My casual observation is stuff like preregistration is still not the norm (and, of course, many papers don't follow their preregs). Even the "reformers" ignore stuff like crud/noise and the self-defensive arguments around small effect sizes that are likely false positives.
I think there's been progress, but I see it more like the rebels at the time of Bunker Hill rather than at Yorktown.
I saw someone put it recently (paraphrased and I forget who): "It's the same bad science, just with bigger samples". I sadly think that's mostly true.
In the end, do I have much more confidence in psychological science today than I did 20 years ago? I'd say not much. the main "change" psychology seems to have made due to the replication crisis, is simply to lower standards of evidence. Where (in my field) people used to brag about r = .3 (likely due to p-hacking), people simply started bragging about r = .03 with little change in message. Until we fix that, I don't think we've accomplished much.
I was also in both PsychMAP and Psych Methods. The two blur together in my mind though, and I also didnt know anything about the sock puppets. Mostly I just remember that Uri guy. He had...a strong personality (coincidently, I believe he was a personality psychologist).
I'm still unsure whether the issue is that our methods are not rigorous enough, or whether the problem is the premise that we are discovering hidden truths about human nature that are replicable. An experimental average, albeit statistically different from a control group, seems like a pretty low bar for saying you have discovered something about human nature. Perhaps the human brain does not consist of a series of effects, tendencies, and biases? Perhaps the idea of stable dispositions modulated by context is just entirely the wrong way to think about the brain/mind? If so, then simply increasing the rigor will not solve the problem
I agree, there’s been progress but I’m surprised that NIH/NSF didn’t just require preregistration of all funded projects years ago.