This weekend I found out that Srull & Wyer (1979) has not replicated in a large multi-lab replication effort. I find this shocking. As followers of my blog knows, I did a trace of who cited this article, and as part of that trace, I rather carefully went through the actual research for those I deemed did priming. This included the original. And, yes, even if the n in each cell was terribly small, the priming was rather intense (nothing subtle about it). I thought it would at least replicate, but with a more truncated effect.
There weren’t that many researchers that used the Donald story in that 5 year trace, but here is the table of the effects as I scraped them off the papers:
Mean hostility ratings of the Donald vignette, by type of prime. (hostility only)
|S & W 1979|| Proportion
|S & W 80|| Proportion
|B & P 82||Priming type||Carver et al|
|60 80%||9,7||50 70%||6,9||80%||7,47||Hostile||6,47|
|30 80%||8,5||50 70%||7,1||20%||6,75||Neutral||5,86|
|60 20%||6,7||50 30%||5,2||0%||6,99|
|30 20%||5,7||50 30%||5,5|
The first two are the original Srull & Wyer and their follow up (only show the rating immediately after the priming, and only for hostility). The task was sentence-unscrambling First number is number of sentences, and the second the proportion of sentences that are hostile.
The third is Bargh & Pietromonaco’s work, where the hostile words were presented parafoveally for 100 ms, which they suggest is outside awareness (participants are terrible at recognizing words – so that is tested for). Much smaller differences here, but that would be expected. In the fourth, the prime was short films where a boss either berates a secretary or behaves neutrally towards the secretary – so a very different kind of prime (emotion induction?). The Donald story had also been adjusted, because the Floridian students thought the original was much more non-ambiguously hostile than the Illinois students. Also a smallish difference.
After having looked so closely at older research (and realizing their weaknesses), and this non-replication, I’m losing confidence in so much of my field. I don’t really know where to go with this. We can’t replicate ourselves out of it (as lots of others have already pointed out).
We need to have rather rapid ways of aggregating bodies of research to see whether there is any point of even pursuing a closer meta-analysis or a replication effort. (I think Ulrich Schimmack’s tools could be very useful for this).
It all is like Oakland – no There There.*
*Gertrud Stein, of course.