I am not a fan of IAT (in fact I distinctly remember writing one of my seminar paper on the problems of IAT, especially on whether it measures actual bias or mere association), but I actually think that Corneille and Gawronski and yourself are being a bit too harsh here. The main question here, and it really is more theoretical than empirical I think, is what is bias to begin with. If bias is indeed a single construct that self-reports and IAT both measure differently, than it makes sense to compare them and which one is better, more robust etc.
However if there really is a difference between implicit and explicit bias (as Gawronsky himself agrees), than comparing IAT, a measure of implicit bias, to self-reports, i.e. a measure of explicit bias, is rather odd to begin with (That's like measuring fluency by comparing reading comprehension to reaction times on a word-nonword task: sure, they both tap into a similar cognitive process, but it is very likely that they are better suited to evaluate different aspects of it, and comparing them would be useless).
So the question becomes - what evidence do we have that IAT badly measures implicit bias. I think some of the points raised by Corneille and Gawronski don't directly answer that, instead just pointing that IAT is not a great measure of bias in general. Even worse, other points just show that IAT is a bad measure of ^insert false assumption regarding implicit bias^, which is definintely not the problem of IAT, that accurately reflects a reality we just didn't hypothesize.
Therefore the problem, I think, is more related to your second point, about the rush to theorize and measure. Fortunatly, that does not mean the field cannot recover - it actually seems to me like a healthy process of regrouping, matching our expectations from IAT to its actual capabilities and properly theorize the differences between implicit and explicit biases.
That being said, I 100% share both the mad respect for Corneille and Gawronski, and the aversion to the rush of the theory-evidence-intervention-policy process. BTW, I think it is a very American (perhaps Anglo-Saxon?) thing - at least from what I see in Europe and here (Israel), people are actually pretty cautious (probably too cautions) about turning social psychology findings into policy.
I am not in this field, but I am frustrated by the constant claims that this is a "healthy process of regrouping". Social psychology has had a major impact in workplaces and politics. In my opinion, it has done damage to interpersonal relationships (e.g., those from an oppressed group should look at everyone as being against them). Research such as this is being pushed because it fits a narrative, and before it is accurately vetted (in fact, questioning it brought reputational harm). It is yet another indication I should not trust this field in any way.
>bias is absolutely real. Anyone paying attention can see how prejudice can shape things from job interviews to traffic stops.
Prejudice and bias aren't interchangeable. Researchers were biased in favor of the IAT (irrationally seeking only confirmatory evidence), whereas it is unclear to what extent preference for white sounding names is based in bias (someone's race legitimately correlates with desired characteristics in an employee not on the resume, like their tendency to show up on time or not).
Frankly, the field failed here because the discussion fails to include the biases and assumptions of people who have the very left-wing perspective typical of the researchers themselves, that racism is a strong explaining factor in behavior, that biases are specific instead of general, .etc.
I enjoyed this essay, the title not so much. Calling it a "bamboozle" implies nefarious intent to con people. This is just an example of the messiness of social science. Calling it a bamboozle stokes an angry reaction unnecessarily.
I think your quality writing deserves a quality title.
I trained as a personality psychologist with a good helping of psychometrics. I can 100% support the point that social psychologists especially are prone to creating individual difference measures to save some finding without any consideration of test-retest or any other version of reliability much less a multi-trait, multi-method matrix of validity. Moreover, the assumption that even consistent and reliable association effects revealed in various reaction time paradigms have never been shown to reliably relate to any macro scale human behavior. They may show us something about cognitive processing that is true and interesting, but its translation into macro social behavior is largely speculative.
Great article. Thank you. "study after study shows people can predict their implicit scores" Demming Kruger is not exactly this, but adjacent. oh wait, another better source that utterly contradicts this.
THE ENTIRE INTERNET.
People are not aware:
Plain racism. Seen Trump? Seen HIS ENTIRE ADMINISTRATION? Do you think any of them think they are unfair/racist/biased? Yeah, that's what the article is about. Almost doesn't matter.
Eggs and pancakes. I'm an MD PhD pharmacologist so I know the tongue has a lot to do with taste especially salt and sweet. You might not know the tongue does this. Which might be the reason why you put the salt and the sweet as far from the tongue on top of the pancakes and eggs. I actually got trolled when suggesting this in NYT cooking. (see above, people are unaware) and you need under a fifth of the seasoning if you season the plate. The plating guy on instagram banned me for this. What we don't perceive isn't there at all bias.
Word bias. I am an active anti-Worfian as I get older. The words put fences up and mess up how we know about meaning. "DEI put unfit workers into positiions they had no experience for". What is unfit? what is no experience? Isn't the rate of development of fit and experience the real issue? wouldn't someone moving up in social class demonstrating those traits? Nobody questioned the words.
Before I switched directions (lacking one four hour course) to earn my BA. in psychology nearly 30 years prior, the one thing I felt where psychology really messed up (where polls mess up as well) is question and answer surveys will not give what I think are reliable answers. I became one of my chief subjects in the way I answered any type personality or preference tests. While implicit bias was not a term I was familiar with then, it was to a great extent looked for in measuring these tests and so what I noticed was that there was an implied bias in how responses were correlated and so to weed out bias, questions are rephrased in various, or reverse formats.
I was always called out, and the surveys thrown out because no matter how they tried to rephrase the question my answers were too consistent to be considered reliable. So when I answered the "best option" and referred my answers to my former answers to avoid inconsistencies in my responses and tried really hard in other questioners in attempting all inconsistencies.
I tend to favor behavioral studies that often get very different results than questionnaires of any kind.
I found questionnaires kind of like Platonic versions of socratic dialogue (not present in Xenophobic socratic dialogue, where Socrates tries to lead people to his perspective, and so I thought these personality tests and questionnaires were most frequently trying to get the results that they wanted to get. Then when I encountered Lisa Feldman Barrett saying she was never getting the results she thought she wanted, it confirmed me in my perspective that much of the bias is in the test, rather than the testee, to get the results the tester expects to find.
And I realized that I was not answering questions myself spontaneously but pouring over the questions to make sure my answers never contradicted themself in the slightest because I thought that was what the tester until they told me no one could only be 100% behaviorally A and % percent category b or c.
And of course I wasn't anymore than anyone else is, and I had to admit I was answering both to how I wished to be perceived and thought the surveys or personality tests were seeking to find an absolute perspective from every respondent.
Funny since those days I find myself unable to answer most surveys because my inconsistencies do exist and I find myself unable to answer any question as an absolute answer, because of course I do not generally always think any particular perspective is always the way I feel. So I'm told to answer as I feel right now. But how can I separate the inconsistencies between how I might feel now if it's different than I felt yesterday, and yesterday's feelings shape those differences perhaps due to factors that might make me slightly alter my feelings today.
I'm also very reactive to every real situation and since "my feelings right now" are always reflective of other factors and subjective to change with my feelings right now.
So the testers are still looking for an absolute but don't to not be detected as now how to actually categorize absolute absoluteness .
I came to the conclusion that the tests are designed to prove the expected tendencies that have already been designed to measure the overall categories they already believe and that those who take the tests want the testers to believe this or that about themself.
Example right here: This site is telling me it does not expect to say themself. I thought it was a valid word and looked it up. Merriam-Webster states, "When people want a reflexive pronoun to go with singular they and them, themself sometimes gets called upon to do the job."
So even computer programs are biased to the expectations they've been taught. People teach the words they expect people to use. And any test designed to detect bias is biased in the beginning with a double whammy of the tester's expectations of what bias the test is designed to detect, and the testee's expectations of what they think is bias and in many cases an attempt to conform to not being detected as bias.
I never took the IAT test, but it seems to me that Mr. Shulman's answer below, that bias itself may be implicit but attempts at measuring the differences between the two will tend to be unmeasurable in comparisons to reports of self-reported explicit bias. And yet I might tend to think that is not as much measuring implicit biases at all, but measuring the expectations of what is bias designed into the test of what biases the test has been designed to detect instances of explicit examples of what bias is in the first place because I am uncertain how you can define any implied bias without a determination of what a bias is, and I do not how any bias can itself without first having a concrete explicit example of that bias to look for in the testing for an implied bias.
And if something is truly implicit without any examples of occurrence then how do you actually detect its implicitness? Or can implicitness only be implied by the person trying to apply the act to the person who might not have done the action. How can a person be implicitity doing an act or is it true you who lusted have already committed adultery. I don't think the point was to imply thinking about the act was to have done the action, although that is how it is generally interpreted, but the entire passage is not about what you have done, it is about not thinking you're better than someone because you haven't done the act you are condemning someone of having done.'
And what these days are really trying to do is apply condemnation to something people haven't ever acted upon.
And so while everything we do probably carries with us our experiences and reflections, like I was trying to say about how I developed an ability to distinguish what I am thinking now from what I thought before, somehow implying guilt to thought that have not being done would sort of open the door to telling everyone we don't care if you've still guilty and suggests that if you maybe thought to do so, then what would be the point of anyone developing any self-control if you are going to be equally condemned for what you did or did not act upon.
If you mention the word adultery, I will think about adultery, and then the first step is not to think of myself control, that is self-control, but if I go beyond and think I might like to commit control, and yet I control that impulse and do not commit adultery, should I be condemned if I thought it and didn't act upon it.
So when we come to racial biases, I think many species have a territorial -protectiveness. Your pet cat doesn't want a strange cat intruding upon her territory. Our species, at least, can have a territorial allegiance to the larger idea of nation. But since humans evolved into the world, as we can see in most every language, an etymological distinction between the Us, the People We are in this Place and the They who are not part of the People Who Should Be in this Place.
So if the "bias" is to measure if we have a greater identity with those we most closely identify, the very test is simply worthless because of course we are. Not just these IAT tests, but with most surveys or tests of this kind to detect deeds not done, the attempt to imply guilt by-thinking is an attempt to say the person who restraints themself from explicit condemnable actions of race-antagonism is somehow implicit in the explicit action is undoing those who are controlling themselves, thoughts not exempted, experiences of association that might have been entered into those thoughts not exempted even if you've restrained yourself from ever acting upon or even concluded that those thoughts introduced are correct, you are nevertheless guilty for that awareness of those ideas that have been introduced because you learned overt racist and they will remain in your consciousness, or should so that you don't follow the leader to the nearest lynching.
I would perhaps welcome a way of determining how successful "implicit" racism has been resisted and how many have not explicitly committed racists actions but what we seem to actually be doing is condemning people for not having committed racists actions whether or not the resistance has been a great struggle or a minor one you hardly think of.
If the point is condemnation of implicitly being racist, it would seem to be designed to weaken or make meaningless any self-control to commit overt racist actions. So the aim should be not to be a means of measuring complicity in the overt or explicit but to find ways to aid and help people to gain self-control over actions of complicity.
And that doesn't seem at all what these tests are meant to measure.
If of interest to other readers, a couple of additional references. 1) Jesse Singal wrote about the IAT in Chapter 6, "The Bias Test" of his 2021 book, "The Quick Fix: Why Fad Psychology Can't Cure Our Social Ills". One of the questioning researchers Singal names is social psychologist Hart Blanton. A sample paper, written with five co-authors, is 2009 paper titled "Strong Claims and Weak Evidence: Reassessing the Predictive Validity of the IAT " https://scholarship.law.upenn.edu/cgi/viewcontent.cgi?article=2533&context=faculty_scholarship
"Think about all the intellectual energy, research funding, and graduate student careers that went into developing and refining these measures."
Consider that the gaps between population groups in IQ tests are real (not 100% environment) and that the behavior patterns people were recognizing were real (not irrational bias). Consider that social scientists wouldn't accept this fact and so created a vast apparatus of research and funding to find solutions for the persistent gaps without ever once considering the true root cause.
Imagine that for decades, in the most advanced and scientific society that has ever existed, that we completely ignored reality and made hugely consequential decisions to solve a problem that never really existed, and that because of those decisions the very survival of our democracy is now at stake, due to the rise of populist anger.
Imagine that social scientists still won't admin the truth.
I am not a fan of IAT (in fact I distinctly remember writing one of my seminar paper on the problems of IAT, especially on whether it measures actual bias or mere association), but I actually think that Corneille and Gawronski and yourself are being a bit too harsh here. The main question here, and it really is more theoretical than empirical I think, is what is bias to begin with. If bias is indeed a single construct that self-reports and IAT both measure differently, than it makes sense to compare them and which one is better, more robust etc.
However if there really is a difference between implicit and explicit bias (as Gawronsky himself agrees), than comparing IAT, a measure of implicit bias, to self-reports, i.e. a measure of explicit bias, is rather odd to begin with (That's like measuring fluency by comparing reading comprehension to reaction times on a word-nonword task: sure, they both tap into a similar cognitive process, but it is very likely that they are better suited to evaluate different aspects of it, and comparing them would be useless).
So the question becomes - what evidence do we have that IAT badly measures implicit bias. I think some of the points raised by Corneille and Gawronski don't directly answer that, instead just pointing that IAT is not a great measure of bias in general. Even worse, other points just show that IAT is a bad measure of ^insert false assumption regarding implicit bias^, which is definintely not the problem of IAT, that accurately reflects a reality we just didn't hypothesize.
Therefore the problem, I think, is more related to your second point, about the rush to theorize and measure. Fortunatly, that does not mean the field cannot recover - it actually seems to me like a healthy process of regrouping, matching our expectations from IAT to its actual capabilities and properly theorize the differences between implicit and explicit biases.
That being said, I 100% share both the mad respect for Corneille and Gawronski, and the aversion to the rush of the theory-evidence-intervention-policy process. BTW, I think it is a very American (perhaps Anglo-Saxon?) thing - at least from what I see in Europe and here (Israel), people are actually pretty cautious (probably too cautions) about turning social psychology findings into policy.
I am not in this field, but I am frustrated by the constant claims that this is a "healthy process of regrouping". Social psychology has had a major impact in workplaces and politics. In my opinion, it has done damage to interpersonal relationships (e.g., those from an oppressed group should look at everyone as being against them). Research such as this is being pushed because it fits a narrative, and before it is accurately vetted (in fact, questioning it brought reputational harm). It is yet another indication I should not trust this field in any way.
>bias is absolutely real. Anyone paying attention can see how prejudice can shape things from job interviews to traffic stops.
Prejudice and bias aren't interchangeable. Researchers were biased in favor of the IAT (irrationally seeking only confirmatory evidence), whereas it is unclear to what extent preference for white sounding names is based in bias (someone's race legitimately correlates with desired characteristics in an employee not on the resume, like their tendency to show up on time or not).
Frankly, the field failed here because the discussion fails to include the biases and assumptions of people who have the very left-wing perspective typical of the researchers themselves, that racism is a strong explaining factor in behavior, that biases are specific instead of general, .etc.
I enjoyed this essay, the title not so much. Calling it a "bamboozle" implies nefarious intent to con people. This is just an example of the messiness of social science. Calling it a bamboozle stokes an angry reaction unnecessarily.
I think your quality writing deserves a quality title.
Fair critique! I'm terrible with titles and perhaps here is another case!
I trained as a personality psychologist with a good helping of psychometrics. I can 100% support the point that social psychologists especially are prone to creating individual difference measures to save some finding without any consideration of test-retest or any other version of reliability much less a multi-trait, multi-method matrix of validity. Moreover, the assumption that even consistent and reliable association effects revealed in various reaction time paradigms have never been shown to reliably relate to any macro scale human behavior. They may show us something about cognitive processing that is true and interesting, but its translation into macro social behavior is largely speculative.
Great article. Thank you. "study after study shows people can predict their implicit scores" Demming Kruger is not exactly this, but adjacent. oh wait, another better source that utterly contradicts this.
THE ENTIRE INTERNET.
People are not aware:
Plain racism. Seen Trump? Seen HIS ENTIRE ADMINISTRATION? Do you think any of them think they are unfair/racist/biased? Yeah, that's what the article is about. Almost doesn't matter.
Eggs and pancakes. I'm an MD PhD pharmacologist so I know the tongue has a lot to do with taste especially salt and sweet. You might not know the tongue does this. Which might be the reason why you put the salt and the sweet as far from the tongue on top of the pancakes and eggs. I actually got trolled when suggesting this in NYT cooking. (see above, people are unaware) and you need under a fifth of the seasoning if you season the plate. The plating guy on instagram banned me for this. What we don't perceive isn't there at all bias.
Word bias. I am an active anti-Worfian as I get older. The words put fences up and mess up how we know about meaning. "DEI put unfit workers into positiions they had no experience for". What is unfit? what is no experience? Isn't the rate of development of fit and experience the real issue? wouldn't someone moving up in social class demonstrating those traits? Nobody questioned the words.
Before I switched directions (lacking one four hour course) to earn my BA. in psychology nearly 30 years prior, the one thing I felt where psychology really messed up (where polls mess up as well) is question and answer surveys will not give what I think are reliable answers. I became one of my chief subjects in the way I answered any type personality or preference tests. While implicit bias was not a term I was familiar with then, it was to a great extent looked for in measuring these tests and so what I noticed was that there was an implied bias in how responses were correlated and so to weed out bias, questions are rephrased in various, or reverse formats.
I was always called out, and the surveys thrown out because no matter how they tried to rephrase the question my answers were too consistent to be considered reliable. So when I answered the "best option" and referred my answers to my former answers to avoid inconsistencies in my responses and tried really hard in other questioners in attempting all inconsistencies.
I tend to favor behavioral studies that often get very different results than questionnaires of any kind.
I found questionnaires kind of like Platonic versions of socratic dialogue (not present in Xenophobic socratic dialogue, where Socrates tries to lead people to his perspective, and so I thought these personality tests and questionnaires were most frequently trying to get the results that they wanted to get. Then when I encountered Lisa Feldman Barrett saying she was never getting the results she thought she wanted, it confirmed me in my perspective that much of the bias is in the test, rather than the testee, to get the results the tester expects to find.
And I realized that I was not answering questions myself spontaneously but pouring over the questions to make sure my answers never contradicted themself in the slightest because I thought that was what the tester until they told me no one could only be 100% behaviorally A and % percent category b or c.
And of course I wasn't anymore than anyone else is, and I had to admit I was answering both to how I wished to be perceived and thought the surveys or personality tests were seeking to find an absolute perspective from every respondent.
Funny since those days I find myself unable to answer most surveys because my inconsistencies do exist and I find myself unable to answer any question as an absolute answer, because of course I do not generally always think any particular perspective is always the way I feel. So I'm told to answer as I feel right now. But how can I separate the inconsistencies between how I might feel now if it's different than I felt yesterday, and yesterday's feelings shape those differences perhaps due to factors that might make me slightly alter my feelings today.
I'm also very reactive to every real situation and since "my feelings right now" are always reflective of other factors and subjective to change with my feelings right now.
So the testers are still looking for an absolute but don't to not be detected as now how to actually categorize absolute absoluteness .
I came to the conclusion that the tests are designed to prove the expected tendencies that have already been designed to measure the overall categories they already believe and that those who take the tests want the testers to believe this or that about themself.
Example right here: This site is telling me it does not expect to say themself. I thought it was a valid word and looked it up. Merriam-Webster states, "When people want a reflexive pronoun to go with singular they and them, themself sometimes gets called upon to do the job."
So even computer programs are biased to the expectations they've been taught. People teach the words they expect people to use. And any test designed to detect bias is biased in the beginning with a double whammy of the tester's expectations of what bias the test is designed to detect, and the testee's expectations of what they think is bias and in many cases an attempt to conform to not being detected as bias.
I never took the IAT test, but it seems to me that Mr. Shulman's answer below, that bias itself may be implicit but attempts at measuring the differences between the two will tend to be unmeasurable in comparisons to reports of self-reported explicit bias. And yet I might tend to think that is not as much measuring implicit biases at all, but measuring the expectations of what is bias designed into the test of what biases the test has been designed to detect instances of explicit examples of what bias is in the first place because I am uncertain how you can define any implied bias without a determination of what a bias is, and I do not how any bias can itself without first having a concrete explicit example of that bias to look for in the testing for an implied bias.
And if something is truly implicit without any examples of occurrence then how do you actually detect its implicitness? Or can implicitness only be implied by the person trying to apply the act to the person who might not have done the action. How can a person be implicitity doing an act or is it true you who lusted have already committed adultery. I don't think the point was to imply thinking about the act was to have done the action, although that is how it is generally interpreted, but the entire passage is not about what you have done, it is about not thinking you're better than someone because you haven't done the act you are condemning someone of having done.'
And what these days are really trying to do is apply condemnation to something people haven't ever acted upon.
And so while everything we do probably carries with us our experiences and reflections, like I was trying to say about how I developed an ability to distinguish what I am thinking now from what I thought before, somehow implying guilt to thought that have not being done would sort of open the door to telling everyone we don't care if you've still guilty and suggests that if you maybe thought to do so, then what would be the point of anyone developing any self-control if you are going to be equally condemned for what you did or did not act upon.
If you mention the word adultery, I will think about adultery, and then the first step is not to think of myself control, that is self-control, but if I go beyond and think I might like to commit control, and yet I control that impulse and do not commit adultery, should I be condemned if I thought it and didn't act upon it.
So when we come to racial biases, I think many species have a territorial -protectiveness. Your pet cat doesn't want a strange cat intruding upon her territory. Our species, at least, can have a territorial allegiance to the larger idea of nation. But since humans evolved into the world, as we can see in most every language, an etymological distinction between the Us, the People We are in this Place and the They who are not part of the People Who Should Be in this Place.
So if the "bias" is to measure if we have a greater identity with those we most closely identify, the very test is simply worthless because of course we are. Not just these IAT tests, but with most surveys or tests of this kind to detect deeds not done, the attempt to imply guilt by-thinking is an attempt to say the person who restraints themself from explicit condemnable actions of race-antagonism is somehow implicit in the explicit action is undoing those who are controlling themselves, thoughts not exempted, experiences of association that might have been entered into those thoughts not exempted even if you've restrained yourself from ever acting upon or even concluded that those thoughts introduced are correct, you are nevertheless guilty for that awareness of those ideas that have been introduced because you learned overt racist and they will remain in your consciousness, or should so that you don't follow the leader to the nearest lynching.
I would perhaps welcome a way of determining how successful "implicit" racism has been resisted and how many have not explicitly committed racists actions but what we seem to actually be doing is condemning people for not having committed racists actions whether or not the resistance has been a great struggle or a minor one you hardly think of.
If the point is condemnation of implicitly being racist, it would seem to be designed to weaken or make meaningless any self-control to commit overt racist actions. So the aim should be not to be a means of measuring complicity in the overt or explicit but to find ways to aid and help people to gain self-control over actions of complicity.
And that doesn't seem at all what these tests are meant to measure.
If of interest to other readers, a couple of additional references. 1) Jesse Singal wrote about the IAT in Chapter 6, "The Bias Test" of his 2021 book, "The Quick Fix: Why Fad Psychology Can't Cure Our Social Ills". One of the questioning researchers Singal names is social psychologist Hart Blanton. A sample paper, written with five co-authors, is 2009 paper titled "Strong Claims and Weak Evidence: Reassessing the Predictive Validity of the IAT " https://scholarship.law.upenn.edu/cgi/viewcontent.cgi?article=2533&context=faculty_scholarship
"Think about all the intellectual energy, research funding, and graduate student careers that went into developing and refining these measures."
Consider that the gaps between population groups in IQ tests are real (not 100% environment) and that the behavior patterns people were recognizing were real (not irrational bias). Consider that social scientists wouldn't accept this fact and so created a vast apparatus of research and funding to find solutions for the persistent gaps without ever once considering the true root cause.
Imagine that for decades, in the most advanced and scientific society that has ever existed, that we completely ignored reality and made hugely consequential decisions to solve a problem that never really existed, and that because of those decisions the very survival of our democracy is now at stake, due to the rise of populist anger.
Imagine that social scientists still won't admin the truth.