Tim Harford has a nice post summarizing some work on on-line interactions and how (potentially) they affect us. On some results I’m a bit meh, but not having read the studies (because, really, I have tons of other crap to do, so there is a limit to how much I can procrastinate), I can’t say much that is intelligent. Let’s just say, I’m not really at all sure that the conclusion that the net is not really good for the soul is warranted.
But, he starts out talking about Joseph Weitzenbaum’s ELIZA. The little LISP program set up like a mirroring therapist that fooled some people into all kinds of disclosure. I first encountered ELIZA in one of Doug Hofstadters books, and, if I recall right, I had to build a proto-ELIZA in one of my undergrad LISP classes.
I also used to throw the original paper at my clinical students in our theory of science seminars. The chapters (Bem & deJong) that the seminar was attached to summarized the philosophical thoughts around the early cognitive revolution and Artificial Intelligence, including discussions on Dennet’s stances (very similar to Marr’s levels), Fodors Language of Thought, the rise and fall of AI, etc. I had found, earlier, that the students were not very motivated to engage, so I hoped that this would get their attention. I included a link to a latter-day, Java emulation of ELIZA, and told them to try it out, and to go easy on it as, after all, they were all almost finished clinical psychologists, and ELIZA was just a program.
And, I think it did. I always had fun in the seminars (probably more fun than the students – we have done away with them now, so they can spend more time doing stats and methodology). But, one thing that struck me was how they anthropomorphized this program. ELIZA was unempathetic, stupid, broke easily, just a really crappy therapist, and of course had none of the important qualities that are needed for a good therapist. And, well, it is just a program of course, but it was really amazing how they imbued this thing with agency!
I tried to take that as a beginning to then try to probe the science and philosophy behind a project like ELIZA, like an instantiation of the ideas that have permeated AI and cognitive science, and cognitive psychology: the mind as a computer, the language of thought. How you can find those ideas in the anxiety and attention research which I have read quite a bit about (well, when I started, doing emotion was something more fit to clinicians, so they had data and paradigms). Try to understand them. Try to look at the limits. Look at the merits.
One of the issues Tim brings up (because it was researched) was whether it is just the posting, or if a response is required for facebook posts. Seems like the posting matters, but not necessarily the interaction . (That may need to be probed more). But, it also made me think more of how humans can imbue their surroundings with life and agency. Getting angry at chairs (well beyond the supposed animistic age at 4), punishing rivers, yelling at computers, falling in love with My Little Pony. All of Paul Blooms work on attaching essential meanings to paintings, and shirts, and lovey-animals (I still have my christening teddy-bear).
This anthropomorphizing, ascribing minds to other entities is, of course, a known (but occasionally forgotten) trouble with psychological research. Louise Barrett’s book was filled with this. Not too long ago, Joseph LeDoux suggested that we call the brain-circuits subserving emotion not emotion circuits but survival circuits, because using the folk-psychological term really adds a lot of fuzzy characteristics that will obscure the research, and, strictly speaking, we have no idea what a freezing defecating rat feels, and whether it is anything like the human conscious subjective feeling of fear. Cordelia Fine brings up the problem with calling certain variants of cognitive style found among those on the autistic spectrum as male brains, because we are liable then to attach a whole lot of cultural garbage onto the notion, and miss important phenomena. And, in the “evolutions empress” book I just got, one of the authors also commented that calling forced copulation rape, rather than something less (kinda?) loaded like, I don’t know, forced copulation (made that up, but it is descriptive) likely also brings in a great deal of cultural garbage that may make us feel either that we understand it better than we do, or reject the ideas because even thinking about it is just simply vile.
With that detour, I wanted to link into an old post of mine, where I discuss Tyler Cowen’s “Create your own economy”. He has a much cheerier view of the internet life.
When I was a kid, I was only allowed to eat candy on Saturdays. My parents had the rule to spare my teeth. But, of course, other parents had different rules. Somehow this bothered me – people who ate candy on Fridays, for example. Or could eat it whenever. It was like they broke a rule. A law. And it was Bad.
In Sweden, when students present their final thesis, they go through a defense, where other students serve as opponents on their work. These students rarely have good expertise about the research area they are opponents for – and we could hardly ask them to, so they discuss and critique what they can, which rarely is the meat of the work. And, frequently, they use their APA manual, treating it as the Laid Down Law, saying things like “I don’t think that is allowed according to APA”.
In my extracurricular quest for understanding us more as a species, I’ve come across researchers like Tomasello (excellent Nautilus paper here summarizing his work), and Richerson & Boyd, which (if I understand it correct) suggests that humans are very good at picking up rules and regularities, especially from other more prestigious individuals, and imitating them, and turning them more into laws than into hacks or cluges.
This is, possibly (although research is continuing) our super-power.
Coming up with new stuff is hard and costly. Imitating is a great short-cut, and a wonderful way of accumulating knowledges, and distribute it over multiple people.
As I’ve been going through Richerson & Boyd (maybe I’ll start practicing the math this summer), it just dawned on me how much of the things I know really are derived, and how little new knowledge I have really provided – although I would say I have provided some tweaked, value added stuff.
Everything I teach, just about, is derived. I trust that the researchers are honest researchers, that those that aggregate in reviews, textbooks and meta-analyses are doing a fairly honest job of it, so that I don’t have to spend the enormous time to actually fact check this from scratch.
This is, of course, our super-power, but also a vulnerability, because there can be lots of misses in our imitation and improving.
Witness the Cargo Cults. There are likely plenty of cargo cults elsewhere right now. I suspect some people I know would nominate Clinical Psychology. Or Social Psychology (grrrr), or Neuro Science. Or why not that computer-inspired chunk of cognitive science. (Hardwired. Spit, growl, hack) It is just time that someone got them on the defensive. Ad agencies are all about cargo cults.
Neuroskeptic, my favorite brain, just kvetched about another ritualized part of science writing – the limitations section. It has become an absolution for having run shoddy research, although I can see the original intent as useful (just like candy on Saturdays, and the APA manual, although it pains me to admit the latter).
There is something similar with the magical blessing of the p<.05, and the ritual of the inferential statistic.
Of course, when you are coming up as a student, the ritual, the things you are supposed to do are the things that scaffolds you into the knowledge (I think anyway). You do the moves, and slowly the reason becomes revealed, as you are mastering the different aspect, and more and more pieces of the puzzle are falling into place.
I sometimes have students asking for something, some patterns, for helping them figuring out how to behave in order to get that magical VG on their exam, and I keep trying to counter act it (at least in my marketing psychology course), because I don’t want them that ritualized.
In other places, that may be an advantage, I’m not sure. Then again, I’m not really sure why people think one should get a university education either.
The ritualized, when it works, can be very productive, I think. You find a good trick, and now you keep building on it, and tweaking it, and expanding on it in interesting ways, and suddenly we have all sorts of flourishing music, and interesting architecture, and the internet, and new and improved living standards!
But, of course, this type of cultural transmission, cultural evolution, can become maladaptive and problematic – which R & B brings up. Their example is one from their own experience, of the academic who severely limits her number of offspring, in order to further her career, because that is what the senior academics do (well, those that do not have obliging spouses). And, that, from the biological perspective, is just maladaptive, if that is what you care about.
The rituals in science can become maladaptive too, and I think they are. Science is supposed to (at least in some part) be explorative. In other parts (the normal science part to go back to Kuhn), solving puzzle and filling in blanks and testing alternatives. Churning through the ideas and burnishing out the truth with little t. But, there will be bad detours.
I think well thought out descriptions of limitations can work out very well. I recall reading Rumelharts & Mclellans interactive activation model paper when recreating it in my undergrad neural network class. I was full of objections, as were all of us. But, in the end, I recall, I was impressed by how they limited the scope, and clearly blocked a number of objections, making it very clear that they were highly aware of these objections, and clearly stating that you have to take it for what it is, and the tiny, pointillistic, Seurat like addition that it makes to the body of research.
(The same cannot be said of a later neural net by McLelland that claimed to say something useful about schizophrenia. I may document my adventures with two computational/mathematical models of that at a later time, where the take home basically is that if an undergraduate can replicate the results with an ill thought out variant, I doubt you really have a model of anything useful).
But, when it becomes like ritualized little incantations, as NeuroSkeptic is saying, there really needs to be a change in the ritual, because now it is meaningless. (Now, who reads the damned discussions anyway? Can you just briefly state what you are after and how it fits, report the hell out of method and your analysis, and stop the blather. You’re no George Elliot. OK, as you were. Just a personal peeve of mine),
And, now, speaking of ritual. Sweden, in fact Malmö, just 20 km south of where I live, are pleased and stoked to hold the xxth European song contest. ESC for short, which I find kinda fitting…
I, perhaps unwisely, introduced my children to this annual ritual when we moved back. They are still young enough to be awed – I’m waiting for them to get old enough to snark.
I usually amuse myself by trying to figure out who they each have borrowed from.
This is one from a few years back – won in Sweden (alas not in Europe). Starts with a clear ABBA intro (sheer and pure Waterloo). Followed by familiar tones, that I cannot quite place, thought the Bowie-esque nature of the singer is unmistakeable. And, this one, I’m now old enough to completely unembarrasedley admit to, I actually like.
I went to the opera with my daughter the other day. The Magic Flute. Wolfie Mozart is always nice to listen to (I have his 40’th as my sound-track right now.) But, man, the view on women… I was talking to the mom of my youngest son’s friend who was also there, and it basically was the first thing out of our mouths. This was both our daughters first Opera Experience. Hmmmmm.
I remember watching the Bergman version when it first came out, because that is what Sweden did that evening. I don’t think I reflected as much on the view on women, but I did reflect on the kind of odd switcheroo of allegiances. You are set up being in the Queen of the Night’s corner. And, it is kind of cool. Three ladies who rescues the lad from the terrible worm. (My kind of fantasy. Ptooi on Damsels in distress dreams). The Queen of the Night tells her sad story of her daughter being abducted. You know right off who to root for.
Except, well, you find out, you’re wrong! (I hate that, tell me clearly who the home team is). It is Sarastro that is. Now, I generally don’t like when people use Mansplaining as a way of short-circuiting someone (well, some man’s) attempt to explain something that may very well be sexist, or at least not quite how one would like things to be. Usually I much rather listen to peoples versions of things, because it is interesting, and because we all have our biases. But, Sarastro’s explaining of why he is good and QON evil is Mansplaining. I mean, look at these excerpts! Here is the priest explaining to Tamino why he’s wrong about Sarastro, who, for all I can understand, has kidnapped Pamina, for god knows what reason.
PRIEST
A woman then has beguiled you,
A woman does little, chatters much,
young man, believe this tongue-game?
Oh, if only Sarastro could to you
Impart the purpose of his action.
Then, the priest goes on being coy, saying he can’t explain WHY Sarastro is good and QoN is Evil, no no, he has sworn! (I’m smelling initiation rite. Commitment and Consistency. Once Tamino has been through all that crap he HAS to believe Sarastro is good).
Or here, Sarastro to Pamina, who had run away because Monostatos was trying to rape her.
PAMINA
A child’s duty calls me
For my mother….
SARASTRO
Stands in my power;
You and your happiness would be killed
if I left you in her hands.
PAMINA
Mother’s name sounds sweet to me
It is she! It is she!
SARASTRO
And a proud woman.
A man must lead your hearts,
For without him every woman is misguided
To step out of her sphere.
Don’t you want to just strangle the supercilious Mansplainer? He kidnaps Pamina, places her to be watched by his slaves (Hello, he has slaves! Isn’t that the marker of the evil side?) one of whom plans a little rape, only to be thwarted by Papageno? What kind of guy is this?
And, then, the “testing” they have to go through basically means “show us that you have learned to ignore women, because, well, women, duh”
DUET – TWO PRIESTS
Beware women’s artfulness;
This is the first duty of the alliance!
Many a wise man let himself be beguiled,
He was wrong, and knew not the error;
Abandoned he sat, at the end,
His trust repaid with scorn!
In vain he wrung his Hands,
Death and dispair were his reward,
Death and dispair were his reward.
Silly, evil women. Couldn’t possibly have a valid different point of view, now could they?. Ever the psychologist, I started thinking about how nice and convenient it is to have clear good and evil to believe in, and then you don’t have to think so much about how those evil people may think and feel. So much more complicated these days. But, now, think about it from QoN’s perspective.
Tamino is buying right into it, the easily led lad. Could it be homosociality? And, Pamina Beware!
3 LADIES
Tamino, listen, you are lost!
Keep in mind the queen!
They’re whispering lots in every ear,
the false intentions of the priests.
TAMINO
wise man analyses and pays no attention
To what the local rabble says.
3 LADIES
They say, whoever swears to her alliance,
Will go to hell with skin and hair.
PAPAGENO
That would be the devil, would be the devil
Would be the devil, be the devil, infernal !
Tell me, Tamino! Is that true?
TAMINO
Gossip, from women repeated,
By hypocrites, however, thought out.
PAPAGENO
Still, the queen says it, too.
TAMINO
She is a woman, has a woman’s mentality
May my word be enough for you:
Think of your duty and act cleverly!
2 LADIES
Why are you so stubborn with us?
Also Papageno is quiet, so speak!
PAPAGENO
I would like to….
You see, that I am not supposed to!
That I can’t leave off the chattering.
TAMINO
Still! Still! Still! Still!
That you can’t stop that chattering
Can’t the chattering stop
Is truly a disgrace for you.
3 LADIES
We have to leave them with disgrace
No one will chatter certainly.
3 LADIES PAPAGENO AND TAMINO
Of a strong mind is a man
He thinks about what he can talk about.
Then we have the whole Monostatos thing. This is just… for 21st century sensibilities… Slave. Black. Aaaack! And, the papers did a bit about this in the reviews. They’ve created kind of a squat, short, pot-bellied thing, which makes me think of this kind of Buddha. Not so much black actually. But, well, not handsome and strapping, and kind of a joke. But, even if he doesn’t look black, the text leaves no doubt about what is intended.
MONOSTATOS
All feel the joy of love,
Bill and coo, flirt, snuggle, and kiss,
And I am supposed to avoid love,
Because a black is ugly,
I, then, been given no heart?
I am also fond of girls,
Always to live without a woman
Would truly be the blaze of hell,
Aaack. I can be un-PC, but this…! The Sarastro Posse – clearly not the Good Guys.
OK, so the Queen of the Night does have a distinct Bad Parent moment when she asks Pamina to kill Sarastro. But, that is one killer aria that accompanies it. Really. You want to be on the side of someone that sings like That!
Then, Sarastro has his one redeeming moment in this aria, about not exacting revenge on QoN
SARASTRO
Within these hallowed halls
One knows not revenge.
should a person have fallen,
Love will guide him to duty.
Then wanders he on the hand of a friend
Cheerful and happy into a better land.
Within these hallowed walls,
Where human loves the human,
No traitor can lurk,
Because one forgives the enemy.
Whomever these lessons do not please,
Deserves not to be a human being.
Perhaps he is redeemable.
I enjoy Opera, I really do. I used to go to the opera in LA when I lived there, and had a student discount, with my best buddy who had a BA in music from USC who also knew the house conductor, and consequently treated the opera as if it was her home. (She’d go, come, Åse – let’s move to the seats up here, near the front instead). I started scanning my memory about the plots, and the view of women, and they are all as reactionary. Cosi fan Tutte – women as fickle lovers. Ok so Susanna in Figaro has some sense. Carmen – my by far favorite – the woman with the large sexual appetite gets slain by the ex when she moves on to the next hot dude. Madama Butterfly. Gah. I love the music, and the intense emotions, but…
We need to redo them. Like some of the kids in my daughters theater group. They did a snow white version, where they all took on the persona of the mythical 14-year-old (it is a term in Sweden. Fjortis. That really awkward age in the beginning of puberty where you are trying on becoming grown, but it is so damned serious you can’t laugh about it, and you really are reverting to two again, and will definitely do exactly opposite of what anybody tells you to do, and really, I think about it with some tenderness as my daughter is only 2 years away, and it is not easy, but even she laughs about the stereotype).
In this version, Snow-white is a bit of a drama-queen demanding attention from the mice. She’s very exacting and dramatically sighing about her clothes, and the step-sisters really don’t seem that bad. All are chewing gum, and checking out their nail polish. She drags herself to the ball (where the prince and one of the stepsisters have started flirting). And she is Not Having Any Fun. All this is orchestrated by an increasingly exasperated director that is trying to keep her actors following the traditional script.
Back at Snow White and the step-sisters home, the prince comes with the shoe. Step-sister and prince are all happy to see each other, but director is insisting it should go the traditional way with Snow-white and the Prince marrying. But, they are all rebelling. Snow White doesn’t want the prince. No. Prince can marry the step-sister (they are both happy now). All she was is to design her own fashion label! And, when the fairy god-mother grants that, they all lived happily ever-after (except for the exasperated director).
Now, could we update The Magic Flute? I liked Peter Sellar’s re-make of Cosi fan Tutte (as I remember, it is a god awful long time since I saw it.)
I think what really is going on is a domestic dispute between QoD and Sarastro. Once upon a time they were lovers, it ended. Perhaps QoD just got sick of his reactionary attitudes towards women, and didn’t want Pamina growing up around that. Or Sarastro just felt castrated by a powerful woman, and rather than, you know, grow, he left. But, QoD hung on to Pamina, denying Sarastro visitation. Tsk tsk tsk. As my daughter would say Not Nice!
Sarastro went Out West and started a Cult. (Not sure which out West, but I figure, that is where you go to start a cult. Supposedly things are kinda Egyptian/masonic in the original, what with the Isis and Osiris, but, well, I imagine LA, and in LA you can do anything. If not Memphis, TN. Elvis is kinda cult). And, at some point, Sarastro and his Posse decided that it would be a neat idea to Kidnap Pamina, because it is not really kindapping if it’s your kid, right.
That’s the back-story. Now the Opera can start, kick-as women, Mansplaining men, and handsome easily led lads.
But…. We have to do something about the whole Monostatos deal also. That old version just won’t do! No slaves, no rapes, no unattractive black men. Clearly, the guys are part of Sarastro’s Posse, in on kidnapping Pamina who they think rightfully should be with Dad. This is what I think happened. Monostatos really is a handsome dude, catches Pamina’s eye, and they decide to run off, because, clearly a reactionary guy like Sarastro may not like the alliance. Monostatos may be in his posse, but, low status. The girl needs a prince! Would be good for the cult. The whole thing Pamina feeds Sarastro – oh, she’s scared, and daren’t own up to her desires. Learn from mom. Saying the guy you like, well, did, you know… so Dad won’t be mad at you? Not Nice.
See they are complex characters.
Opera continues. Easily Led Lads starts through the trials, having now bought into Sarastros Story. They are young. They want to know what is good or evil, and find the easy answers. Ah well. But, had they listened to the 3 ladies, maybe we wouldn’t have had to witness two near suicides. Sarastro, you need to figure out a better way.
In the end, Papageno gets his Papagena. QoN and Sarastro talk it out, not reconcile, talk it out. Maybe QoN can learn some anger management. Sarastro just has to continue working on his inner softie. And, as for Pamina and the boys. Well, I think I got it from Sarah Blaffer-Hrdy that in places that practice informal poly-andry (because you need the sperm from more than one guy to build a baby!), the optimal number of fathers is two. So, let Pamina have both.
CHORUS
Hail to you who are consecrated!
Hail to you who are consecrated!
You pushed through night. Thanks! Thanks!
Thanks be to you, Osiris! Thanks!
Thanks be brought to you, Isis!
May power be victorious
And crown as a reward
Beauty and wisdom
With an eternal crown.
Strength was victorious
Strength was victorious
And crowned as a reward
And crowned as a reward
Beauty and wisdom
With an eternal crown. etc.
I linked in Rober Trivers blog post on him blowing the whistle on research he was involved in, and his subsequent mistreatment. A follow-up here from Nature, basically confirming the issue with the research he was complaining about.
As I was telling my friend, if Robert f*ing Trivers gets in trouble for blowing the whistle, (Talk about silverback – although, of the eccentric type – some of us have a weakness for that eccentric type), imagine a poor graduate student who is nobody, with a whole future to be destroyed trying to bring up questionable practices.
We're thrilled to launch our next series of animations: This Thing Called Science. This series follows on from Critical Thinking, showing the way we think scientifically by considering skepticism, testing, blinding, uncertainty ethics and citizen science. Bridge8 edited, produced, animated and directed this series for TechNyou, an emerging technologies public information resource funded by the Australian Government Department of Industry, Innovation, Science, Research and Tertiary Education (DIISRTE).
I’m as grumpy as grumpy cat looks for the moment. Probably mostly to do with it being latish in the day, and some mild infection, and I’ll be all chirpy in the morning again. But, this piece annoyed me.
Greenwald & Banaji came up with the IAT – even I use the IAT. And are both good social psychologists. This just doesn’t seem that it should make any of them…blown away in anyway.
Liking the ingroup is not new. Hell, the effect isn’t even new within social psychology, because it is one of the techniques that Cialdini brings up under the “liking” way to yes. You want to get a leg-up, mention the common university, the similar name, the political party or pasttime in common. Hell, those letters of recommendations that universities in the US insist on. The personal. Someone you know, the networks, the friend of a friend of a friend.
It doesn’t seem like it places anything on its head. (Hey, we get that in our data anyway – even in my very own data. Yay, I have original data).
Also, reminds me of what students frequently ask when talking about prejudice. Can’t you also have positive prejudice? Like just deciding you like someone for silly reasons (like area of origin, or skin color, or belonging to your family). Which, in the literature is never considered that way, but, yes, of course. If you look at what the word means (but it is not how it is used).
And, from my wanderings around in Evolutionary Psychology and Anthropology, and various other areas looking at relationships, this is not in the least odd. Chris Boehm suggested we are selfish first, then nepotistic, then altruistic. And, there are plenty of variants of that (haldanes and giving his life for 2 brothers and 4 cousins or whatever that apocryphal story said). This is so human.
But, perhaps it is interesting to the audience, and I should put my grumpy self to sleep anyway.
I brought with me Robert Franks Passion within Reason (another tip from Jason Collins) on my trip to Brussels. I was reading it, as I was waiting for the symposium to start (and waiting for Daniel Lakens to show up), and it struck me that some of what he was talking about probably fits the situation.
Let me elaborate. The whole book is about how emotions have a rationality to them (yay, as that is what I believe too), and that they specifically are useful for sustained cooperation. They serve, in part, as honest signals.
As I was waiting, I was reading a part where he elaborates on how cooperation could evolve, when cheaters can take advantage of the honest agents, and drive them to extinction – this is well treaded areas of course. The population consists of honest agent, and cheaters. Everybody wants to work with a honest agent, as there is a higher payoff for both other honest and the dishonest agents. Honest agents that cooperate with dishonest agents are worse off than working alone. A dishonest agent who hooks up with another cheat does as well as if working alone. The cost is borne by the honest ones.
In the model, he first equips the honest ones with an honest signal – a blush to take his example. That way, you can find the other honest agents, and avoid the dishonest ones. Don’t we all wish the world was that easy.
But, as Robert Trivers shows (and Frank, and others), it pays to mimic a good signal, and cheaters who can will flourish. To a point.
But, maybe the cheat signal varies in how good it is. Think about animal mimicry – pretty good but not perfects. (For example batesian mimicry of Coral snakes by King snakes). And, the honest signal also varies in how good it is. The distributions differ. Only honests have the highest strengths, and only cheats completely lack the signal. But, there is an overlap, where the signal now is ambiguous.
Those with a clear signal have a valuable asset – everybody wants to cooperate with them. But, they also have a vulnerability – what if they pick a cheat. Frank adds an inspection cost. Someone with a high signal value (because it is really mostly in their interest) can pay some cost (in time, effort) to inspect the prospective partners signal.
Out of this, he creates a model of iterated cooperation. Some of the consequences of the model: the higher the proportion of honest agents in the population, the more likely an ambiguous signal is to be an honest signal. That is, it is less risky to cooperate.
The signal reservation point (the intensity of the blush as indicating honesty, for example) also goes down (less blushing needed), if there is high payoff for successful cooperation, and low penalty for being cheated. The gains outweighs the losses, and you can take more risk (simply be more trusting). The equilibrium points shift, depending on these parameters.
If you squint, or have a vivid imagination with low association thresholds as I (sometimes) do, you can kind of map this onto the research endeavor. Trust is necessary. You want to believe that other researchers are honest researchers, so you can build on their work. There will always be cheats. The goal may not be to catch them, but to mimimize their pay-off. Raise the payoff for honesty, lower the cost of being cheated. Make it less attractive for cheats (increase the population of the honest).
Of course a model like that is rather simplistic (hah – but really). Real life has to deal with not only those who fake data for fame and fortune, but those who polish their outliers a little too neatly, and prune their data-sets a bit too severe and topiary like.
The payoffs in the system has not been geared towards good research practices. (Yeah, that is about the billionth or so time someone has stated this). You believe Wichert’s and John’s research (and most colleagues) people in science want to do science because they want to find out something that is true. Because it is interesting. Because we are curious. Perhaps not for any – well – useful reason, but because following ones curiosity is itself rewarding, and hearing someone talking about what they have found out is rewarding. Now, what use is it to know about planets and stars and quarks? And, of course, some also are interested in doing research because they have some practical ideas.
But, research is discovery, and inherently uncertain. Demanding productivity in the shape of positive results and papers, preferably in high-impact journals to keep your job (to pay your mortgage, to feed your kids and yourself) sets things up for gaming. Also sets it up for not taking the right kind of risks – pursuing something that may not pay off in any way at all but could potentially lead to discovery. (Let me put in a plug again for Chris Chamber’s initiative at Cortex).
Another issue is how new researchers are trained, as in, which practices do they see and copy (and are told are standard). This was brought up in the panel (horror at what some young researchers thought were normal practices), it was clear in both Wichert’s and John’s research, and Nosek talked about incorporating procedures in their open science framework to improve these practices without unnecessarily burdening he researchers.
Research is a craft. You learn doing it under the supervision of a skilled senior researcher. Most of it is tacit, or informal exchanges in lab-discussions. There are lots of heuristics and rules of thumb. You learn what those that are successful do, and copy it as best you can. It is very Richerson & Boyd, cultural transfer. There is so much to learn, and so much to get your head around, I actually think it is impossible to not use the heuristics and short cuts and what works – like rest of life. Plus, you are investigating new things. You simply don’t know, even if you are very good at it. Otherwise, why do research?
Altering the practices has a cost – in slowing down productivity. (A benefit in making a more robust science). They must be reasonable (I still shudder at the utter byzantine rule systems around the ethics tests in the US – which seems more like a power issue, than one of making sure participants are protected. And, I am very much in favor for having checks.) But, they should include documented and shareable data. Easily shareable instruments. Good protocols covering both data-collection, and the data-processing. And, you should be expected to share it. If something is never ever checked, will take up precious time, and doesn’t seem to affect the near productivity, it will fall to the wayside.
Now, if we get the good practices in place, and are able to shift the reward structure, will it be stable? Likely not. It is all so evolutionary, and new ways of figuring out how to game the system will arise. I have been listening through Fukuyama’s the Origin of Political Order, and it is a story of stability altering with gaming and instability. I think this also is the point of Peter Turchin’s clio-dynamics. I think his analysis of Dune is apt, although I don’t think we are planning on overthrowing an empire and starting a new.
Oh, speaking of perverse incentives. I’ll just throw in this post from Steve Hsu, and the Nature Commentary he linked to.
My friends over on the humanities side and me are working on some similar ideas to those described in this blog. We have some intresting results, and some further ideas. Remarkably little has been done on something so influential.
Whatever my quibbles with David Shanks, the main point of his talk – a clear demonstration about the problems we face in reliability when more positive than null results are published, and replications end up in the file-drawers (not to speak of fraudsters). There needs to be an adjustment back (back?) towards ensuring reliability, rather than novelty. Perhaps also to try to puzzle together bigger pictures (by focusing on effects?).
As I was writing yesterday, I was becoming all insecure about the effects that I spoke about, which I have learned through textbooks mainly – not by looking at data, effect sizes, etc. Narrative claims about directions. Some graphs. Sure, there have also been a lot of papers I’ve skimmed. But, most of the stuff I actually work on and analyze are low-effects, not highly powered (well, except the 25 hours worth of reaction time collection over 12 participants resulting in over 30000 data-points each). Emotional effects on cognitive tasks (emotion induction – works well. Effect on cognitive task – wobbly). Individual differences in preference for ethnicity on judgment/perception – small effect. Even real? My hands on research basically has given me coals in the stockings. Not much to hang on my CV.
But, it worries me now that I don’t know how robust those narrative summaries of research are. They sound robust. Perhaps they are like those hooks Smilla describes in her sense of snow that look robust but can be picked apart with your nails.
I wrote elsewhere about my adventures in priming, which at least ended up in a publication. At the time we were struggling with getting this to work, we (well advisor and co-workers) were going, “I can’t believe Bargh is getting those effects! How does he do it?” There really seemed like a buzz of incredulity (never of suspected fraud I will hasten to say, just incredulity).
I saw him speak at APS right around that time, and listening to him, I felt very sympathetic to the ideas. We don’t really always know what we are about, and what influences us, and lots of processing happens without us really thinking about them or knowing about them, like awareness is kind of a final or need-to-know station. I still don’t think that is implausible, although perhaps it is not the priming that does it.
There were these nagging doubts. But, in the end, things like his elderly prime and the like were cited so often by so many different sources that I figured there was enough evidence to overcome skepticism. The 1000 Elvises who can’t be wrong (now, why that particular version of social proof sticks in my head…).
And, now I wonder if it just simply is social proof or mere exposure, and should I doubt my field?
I recently read Lee Jussim’s Social Perception and Social Reality, where he drives his thesis that we are frequently fairly good at social perception, and that stereotypes are not always hopelessly biased. He pulls together a lot of data, being very careful in advancing his thesis. He starts with the Pygmalion effect. I had heard before (anecdotally) that it was hard to replicate. He goes through the research and the few follow-ups. There is an effect, but it is rather small. Mostly, teachers are fairly accurate about perceiving the abilities of their students.
He goes through a piece of research that I had been taught in class, and which I was under the impression had been replicated in many versions – it is work were a male gets to talk to a female who he either thinks is plain, or attractive due to a picture. And, evidently the effect of the good-looking picture can be detected even in the answer of the woman, in a positive way. It is considered a halo effect of beauty. Dion, Bersheid, Walster. But, evidently, it has not been replicated. So, nobody knows if it can.
In some ways, the whole book is a meta-analysis, because he is pulling in and comparing a great deal of data and their effect sizes. He actually tries to put the effects into proportion with each other, rather than simple directions (that one then can exaggerate at will). There needs to be more of this type of books, pulling together a lot of research answering rather simple questions. It is also very engagingly written, and I’d recommend it to anybody, but it is very expensive. (I got it as an e-book loan). I ended up reading the book after reading his plea for reporting effect-sizes and using them in your reasoning in this Pigee blog.
There really is a need for putting in the proportions, the effects. I come across confused undergraduates all the time, because the research seems contradictory. Positive emotions make you use more heuristics, and become more stereotypical except, of course, when it makes you more engaged and go out of your way with the task. Stereotypes are biasing, but you can extract information about people in a thin slice. Similarly, when arguing on the internet, and people throw out directions, and not effects, and in fact, we don’t know. Often within the tiresome nature-nurture discussions. We need to know. We need to integrate these findings. Is there a bigger picture to be found? (Yes, I know, meta-analyses. I probably should do that. At least I don’t have to scare up students to collect data for me then).
The data I will present on Wednesday is not very strong. It has been laying around for years, in part because it is hard to interpret, in part because it doesn’t seem novel and exciting, and I’ll just be rejected again (that I have to get over). In some ways it was perhaps fortunate because I discovered a calculation error, which changed the data. Had I already published, I would have had to retract, and I would have been mortified. There are still results – in some instances clearer – in some not as fun.
It isn’t awfully sexy, but I’m beginning to think that the tack I need to take is to instead present it within the small body of research looking at this issue, trying to estimate an effect. I have by far more participants and data-points than earlier research, so I should have power on my side. But, in some ways, publishing this and the other less sexy things is really what science ought to be about. Plodding, puzzling, that normal science Kuhn talks about (yes, I have started listening to Structure). Because we want a science we can trust. Of course, nobody’s health rides on whether or not people’s attitudes really bias their perception of affect in other faces, but it could be played up, considering the infected discussions on racism right now.
We need the checking, and rechecking, if only to exclaim “perfect ice-cubes, again!”
I’m thinking of that scene at the end in Amadeus – I recalled it wrongly, it is both milder and more disturbing than I remember. Salieri, as the patron saint of mediocrity. We can’t all be Wolfie Mozart. I think we need the Salieris. Even of science. For the Mozarts to be there.
