The Consequences of Dishonest Honesty Research

A while back I wrote a long piece about an academic paper that purported to show that having people sign an insurance form at the beginning of the form induced them to provide more honest answers than having them sign it at the end. You can re-read that overly-long post here if you are so inclined.

The paper was bullshit, the journal that published it eventually retracted it, and one of the authors, Gino Francesca, was de-tenured and then fired by Harvard for this and other alleged misbehaviour. She is currently suing Harvard over this. Of course.

I am writing to say that I have found out, thanks to Gelman’s stats blog, that this particular paper had an impact out there in the real world, and, not surprisingly, not a good impact. The entire story is laid out in an article in The Times of London, which has a paywall. I am not a subscriber, but the Gelman blog lays out enough of the story that I can pass it on to you here. (You can read the whole account yourself here if you prefer.)

A guy named Michael Sanders was working in the Nudge Unit in the UK in 2015. ‘nudging’ refers to a whole set of academic findings that claims to be able to change people’s behaviour using simple-seeming changes in the way choices are presented – like where they sign an insurance form. Tony Blair’s Labour government went in big on this stuff, clearly.

[I add here that Gelman and company are deeply skeptical of all of this research, and not just because some of it uses falsified data, referring to those who do it as ‘Nudgelords’]

Back to the story. Having taken in and believed the findings in the above-mentioned insurance form paper by Gino et al, Sanders and company went off to Guatemala, a country with low tax compliance, to run a big trial to show that country how to increase said compliance. [Why this UK unit did this in Guatemala I do not know, probably the original Times story explains.] They would have some folks in this trial sign an honesty declaration at the start of their tax form, the others would sign at the end. Here now I quote Gelman quoting the Times story:

He was confident. “It was an enormous amount of data. If there was an effect, we would see it. With samples that size, you can detect the sound of a gnat’s wing.” And the effect? “Nothing.”

They had promised the Guatemalans big impacts, and with justification. The authors of the original research were, said Sanders, “a who’s who of behavioural science.” Yet they couldn’t replicate the work. It was inexplicable.

Of course, it was inexplicable back in 2015, when they ran the trial, but not later, once it had been shown that the data in the original study had been faked.

Most embarrassing if you are a card-carrying member of The Nudge Unit, UK Division. I wonder if they could sue the authors of that original study?