Late spring, I participated as one of the reviewers in the Red Team challenge.* I was selected for my knowledge of research on emotional expressions, which now goes back to the time before emotion research was fashionable (as soon as it got, I kind of slid out. I don’t like crowds).
I was so excited about the possibility, that I plowed through the edited volume “ the science of facial expressions”, read the paper and began writing a review before I even knew I had been selected (actually lotted in), and been informed about the format of feedback. (it wasn’t supposed to be the standard review format. Oh well).
It’s a while since I was this excited about something in science!
I wanted to share a bit of my experience.
The challenge was formatted as finding errors, noting them down on a google docs spread sheet. These were to be judged by an independent judge (Ruben Arslan).
For some issues this works well: Computational errors. Confusing parts in text. Sections that seems to not quite represent earlier research.
But, I did find this format itself challenging, as my commentary was not always bite-sized, and not always about errors but about alternative interpretations, questions about methodological choices, references to other theories. I did the best I could with the ones I had and in the end also included the entire review that I wrote – which is not as easy to judge. (I did read the entire spread sheet of issues with Ruben’s and Nicholas’ comments).
Some of my “issues” weren’t so much about errors as it was about bringing up challenges from the field. For example the work assumed a categorical/discrete view of emotions both in posing and in rating of emotional experience. But there is an approaching century old controversy about how to conceptualize emotions – dimensionally or categorically. Another example was the opening statement that most theories of emotions incorporate facial feedback which, in the fashion of a long time essay advisor, I thought was much too broad, as I can think of lots of theoretical accounts surrounding emotions that are completely uninterested in whether there is something like facial feedback. Nothing was made of that. I just wanted something more precise. Is it an error? Not really.
I also came across this chapter in the above mentioned book: Form and Function of facial expressive origins, by Daniel H. Lee and Adam K. Anderson. Their main interest was in looking at the face as a moving surface, and how different expressions seems to have opposite action (an expansion in fear, a contraction in disgust). On page 1983 they mention how the posing of expressions did not seem to have any effect on pupil size or behavioral changes (measured by looking at eye movements), and concluding from this that there were no evidence for no autonomic feedback. I’m not sure how to incorporate that into the paper.
In the end, I did like the paper. I thought it was a cute, minor contribution, but doesn’t resolve the facial feedback question. Of course, no paper could
Unlike Frank Zappa, I wasn’t in it for the money. I’m a reasonably well paid academic in a secure position. I appreciated it (moar artisanal muesli for me!), but that is not what enticed me. Now, I’m not categorically against using money as an incentive, and I do think we need a better incentive structure, with more acknowledgement of all the work that goes into producing good research.
Scientists as a Social Species
So what excited me about the Red Team challenge?
It was a paper in an area where I have expertise about a topic that I care about. It was trying out a new technique for improving the scientific literature, and science in general, which is also an area I care deeply about. It was also conducted by people I like and care about – in the capacity as scientists and science reformers. I felt I could make a contribution that was deeply meaningful.
Let me expand a bit on this. My absolute favorite book on meta science is David Hull’s “Science as a process” which I came across in the late 90’s as a graduate student (and then re-read several times). The thesis he advances (with thorough observations) is that science proceeds in an evolutionary manner**. The knowledge that is amassed through the scientific endeavor does not spring from the individual scientists heads, nor does it require that scientists are dispassionate and ostensibly impartial and objective. No, it is a product of the churning between groups of scientists passionately amassing and rebutting evidence and arguments, as described by Sperber and Mercier in The enigma of reason. It is not fool proof. It is not necessarily efficient (Hull points out that “efficiency” is a value, not something necessary), but it does slowly, incrementally approaches something that could be a little t, provisional truth. I’ve recently seen Cordelia Fine mentioning this in an Aeon article***.
In his conceptualization, the individual scientist is inconsequential, unless they belong to a deme, a scientific community where subgroups either champions or rebuts a particular topic. The social is as important as is the desire to understand the world just a little bit better. A single scientists may have fantastic insights, but these won’t matter if shouted into a lonely void, without others engaging with the ideas.
Science need to have active communities engaging with and arguing about ideas for our understanding to deepen. Hull asks towards the end of the book, what would happen if the argumentation was attenuated or stopped? And my thought is that what has been seen in the science crisis is in part what happens when critique and argument is not rewarded, just production of papers. The inquiry is not corrected if critiques are stopped (but that is for another post to develop).
But, to come back to me, this felt deeply meaningful to me as I felt I contributed to two small areas in science (emotion research and meta-psychology), and that my small, prosocial contribution could possibly help making science better. It doesn’t matter if in the future there will be no facial feedback or the scientific process will take up other procedures than the red/blue team style. It is a small step on the way.
*A challenge to find errors in the paper by Nicholas Coles, Brooke Frohlich, Jeff T. Larsen, Lowell Gaertner and get paid for it. Proposed by Daniël Lakens and Nicholas Coles.
** He is regularly cited by researchers in cultural evolution
“scientific objectivity depends not simply on scientists being coolly detached with respect to their data, but ‘upon the depth and scope of the transformative interrogation that occurs in any given scientific community’.” (Fine quoting Longino)