Another excellent post. You were 100% right on 4 of the 5, and possibly the 5th, too. Chen ... et al ... Heine's paper was enough to move me from "completely dead" to "hmmm, maybe just on life support, in a coma."
Added value. I have a paper we never pubbed, on how replicable social psych stuff is, based on all sorts of sources. Problem is, "did it replicate" is a q that can be empirically assessed in lots of different ways, that do not necessarily lead to the same results/conclusions. What follows probably should only be done with: 1. studies that have been pre-registered and the prereg actually followed (goldish standard=registered replic reports) or 2. pubbed by people with impeccable credibility (e.g., Kahneman). Anything else is subject to pub biases, and if there are pub biases, this is not useful.
STILL, here is a very simple way to do it that, I think, has some value. The core problem is this: If there is no there there, successful replics at p<.05 will only occur 1/20 times, IF there is no pub bias. So what should we make of an otherwise credible lit that has 5 successful and 15 unsuccessful replics? That's A LOT of failed replics. But success is > 1/20.
Compute a simple binomial! What's the probability that something that has a 1/20 chance will occur 5/20 times? I use this online binom calculator:
The answer for my hypothetical is .0025. So this is below .01, which we've mostly taken to using as the threshold for "should we take our own, or anyone else's, results seriously" given the difficulty of replicating stuff at .05>p>.01. So such a lit means there is probably a real effect in there. I never did this for TMT, but, heuristically, it might capture what is going on there.
Disclosure: I have a TMT paper that found the effect (on antisemitism!) in three experiments. Should you believe it? Who the hell knows? No one has tried to replicate it, so I do not want to make too much of it. Just disclosing my semi-vested (I am not really *that* invested in this) interest here.
Still, even if there is a weak/fragile effect there, you are still 90%+ right on it, because the effects are not what they were cracked up to be.
On the other 4, you were completely right. "Bamboozle" is the right term to use for implicit bias claims, because, even if the advocates were "earnest" in the unjustified claims they promoted, it is because they leapt to conclusions w/o adequate evidence. So, at best, its self-deception. Bill von Hipple has a great paper in BBS on the evolution of self-deception. TL;DR: the theory argues that self-deception evolved because if we earnestly believe the nonsense we promote, we will be more persuasive to others. No one would shrink from calling beliefs in bizarre conspiracy theories, such as Qanon (the world is run by a cabal of pedophiles) or the Protocols of the Elders of Zion "bamboozling" just because someone earrnestly believes them.
I don't know if you're familiar with the aphantasia literature, but that stuff needs to be taken down a peg or two. I'd like to see it get the Mickey treatment!
I'm strongly of the view that *if* aphantasia is real (i.e., people who are entirely without visual imagery) it represents at most 1% of the population—everyone else likely falls along a spectrum of visual imagery intensity. Currently, I've seen the number 5-10% thrown around which seems insane for a field that almost entirely relies on self-report and is highly susceptible to people mistakenly believing that visual imagery is identical to normal vision.
Another excellent post. You were 100% right on 4 of the 5, and possibly the 5th, too. Chen ... et al ... Heine's paper was enough to move me from "completely dead" to "hmmm, maybe just on life support, in a coma."
Added value. I have a paper we never pubbed, on how replicable social psych stuff is, based on all sorts of sources. Problem is, "did it replicate" is a q that can be empirically assessed in lots of different ways, that do not necessarily lead to the same results/conclusions. What follows probably should only be done with: 1. studies that have been pre-registered and the prereg actually followed (goldish standard=registered replic reports) or 2. pubbed by people with impeccable credibility (e.g., Kahneman). Anything else is subject to pub biases, and if there are pub biases, this is not useful.
STILL, here is a very simple way to do it that, I think, has some value. The core problem is this: If there is no there there, successful replics at p<.05 will only occur 1/20 times, IF there is no pub bias. So what should we make of an otherwise credible lit that has 5 successful and 15 unsuccessful replics? That's A LOT of failed replics. But success is > 1/20.
Compute a simple binomial! What's the probability that something that has a 1/20 chance will occur 5/20 times? I use this online binom calculator:
https://stattrek.com/online-calculator/binomial
The answer for my hypothetical is .0025. So this is below .01, which we've mostly taken to using as the threshold for "should we take our own, or anyone else's, results seriously" given the difficulty of replicating stuff at .05>p>.01. So such a lit means there is probably a real effect in there. I never did this for TMT, but, heuristically, it might capture what is going on there.
Disclosure: I have a TMT paper that found the effect (on antisemitism!) in three experiments. Should you believe it? Who the hell knows? No one has tried to replicate it, so I do not want to make too much of it. Just disclosing my semi-vested (I am not really *that* invested in this) interest here.
Still, even if there is a weak/fragile effect there, you are still 90%+ right on it, because the effects are not what they were cracked up to be.
On the other 4, you were completely right. "Bamboozle" is the right term to use for implicit bias claims, because, even if the advocates were "earnest" in the unjustified claims they promoted, it is because they leapt to conclusions w/o adequate evidence. So, at best, its self-deception. Bill von Hipple has a great paper in BBS on the evolution of self-deception. TL;DR: the theory argues that self-deception evolved because if we earnestly believe the nonsense we promote, we will be more persuasive to others. No one would shrink from calling beliefs in bizarre conspiracy theories, such as Qanon (the world is run by a cabal of pedophiles) or the Protocols of the Elders of Zion "bamboozling" just because someone earrnestly believes them.
Very nice post.
You are a rockstar for having accomplished a year of weekly essays! I’m so proud of you, as always. 💗
I don't know if you're familiar with the aphantasia literature, but that stuff needs to be taken down a peg or two. I'd like to see it get the Mickey treatment!
I'm strongly of the view that *if* aphantasia is real (i.e., people who are entirely without visual imagery) it represents at most 1% of the population—everyone else likely falls along a spectrum of visual imagery intensity. Currently, I've seen the number 5-10% thrown around which seems insane for a field that almost entirely relies on self-report and is highly susceptible to people mistakenly believing that visual imagery is identical to normal vision.