Science vs Saying Things
I have posted more than a few articles about bad research in the social and medical sciences here. These have generally focused on research that involved analyzing data to understand things about human behavior and/or illness, but where the research is badly done, including, in some cases, the relevant data being falsified.
If the research in the now-retracted paper (written about here) on getting people to more honestly report the mileage they drove their cars had been carefully done, it might be of some use to somebody. It wasn’t Nobel-prize stuff (I do hope) but insurance companies would have some reason to try having their clients sign an honesty pledge at the top of their forms rather than the bottom, and other types of organizations needing info from their ‘stakeholders’ might also have followed suit.
It might even be that putting the pledge at the top has the impact the researchers claimed it did; however, their research provided no evidence that it did, and that’s the problem. So, that is science, or, if you prefer, Science. Science doesn’t prove things, it looks for evidence that the world works a certain way, and no conscientious scientist thinks they have the world figured out after one experiment or data analysis. That analysis needs to be replicated more than once before anyone has any faith in it – or should. One of my early posts here was about a general belief among researchers – mostly economists – that careful research had established that there is no such thing as a ‘hot hand’ in shooting baskets, you may recall. It took over 30 years for another pair of researchers to figure out that all the evidence gathered about it had involved a statistical miscalculation, and that correcting it implied that all that evidence actually suggested that a hot hand did in fact exist.
Science, then. Data, experiment or observation, analysis, resulting in evidence for or against some clear hypothesis about the world. And, the understanding that mistakes get made all the time, and the world is full of randomness, so don’t be too certain about anything.
However, a lot of what gets published in academic journals looks nothing like that. It’s only connection to science is that it results in papers written by academics being published in academic journals. It does not, however, involve any of data, evidence or statistical analysis. It ‘says things’. In complex and hard to understand language, to be sure, but still just ‘saying things’.
This was brought to mind by an official email I got last week from NEST, the Network for Economic and Social Trends, about a talk they were sponsoring, which the email invited me to attend. The academic was one Dr. Farah Mihlar, who (I found her web page) is a Senior Lecturer at Oxford Brookes University, UK in Human Rights Policy and Practice. She is apparently in the School of Architecture, which is just….well, bizarre. What her work has to do with architecture I cannot possibly imagine.
Anyway, the title of her talk was ‘Compounded Suffering in Post-War’, and the email included an ‘abstract of her presentation’, which I include in total here:
“Suffering is strongly associated with war but not its aftermath. In spite of its ambiguity, the post-war represents a transformation from humanitarian crisis to development and peacebuilding. Consequentially, references to suffering are replaced by political, economic, social ‘challenges’ and ‘issues’ that fail to accurately and fully represent the experience of most war survivors, especially women, who often make up a larger percentage of the population. Responding to this gap, this research conceptualises compounded suffering as the complex amalgamation of multiple forms of suffering: mental, physical and emotional; grief, pain, agony, torture, hurt, trauma, anxiety, stress, difficulty, produced by the unique conditions of post-war, which in coming together, aggravates and intensifies. The post-war context produces a typology of suffering that is distinct in form and extent fundamentally because it is shaped through an acute, real, and continuing threat to human life and security, precluded by the trauma and experience of extensive death, loss, and destruction. Suffering is compounded by women’s ‘entanglement’ with power specific to the post-war context, where structural and other forms of violence are pervasive and ideologies and methods of war remain amidst the pretext of change and transformation.”
Yes, it is very very long, not what I think of when I see the word ‘abstract’. But, I don’t want my readers to think I make shit up.
I will say a few things about this; enough, I hope, so you see what I mean about ‘just saying things’.
The first sentence is “Suffering is strongly associated with war but not its aftermath.”
This is a statement about the world, it would seem. Humans strongly associate suffering with war, but not its aftermath. I am down with the first clause, but then – ‘but not its aftermath’? Really? Who are these people who do not associate the aftermath of war with suffering? Or is it just that people don’t associate it ‘strongly’ enough?
I am about to do a presentation in my Geezer’s Study Group, and as it happens, my talk will include discussions of two wars: the First Indochina War and The Algerian War of Independence. None of the sources I’ve read on these two sad events suggest that there was no suffering after those wars ended. People lose contact with loved ones and don’t find out until the war ends what happened to them. Others had to flee from combat areas, and come back after the truce is signed to find destruction, disease and loss. Combatants must deal with living with their injuries after the war ends.
Who are these people that Professor Fihlar asserts do not realize this?
It is an assertion, it is ‘saying things’ and it is deeply important that particular thing be said, else there is no research project, and no talk.
Here is the key sentence in the abstract, explaining what the research is all about.
“Responding to this gap, this research conceptualises compounded suffering as the complex amalgamation of multiple forms of suffering: mental, physical and emotional; grief, pain, agony, torture, hurt, trauma, anxiety, stress, difficulty, produced by the unique conditions of post-war, which in coming together, aggravates and intensifies.”
Yes, a very long sentence. These sort of folks tend to write like this, in my experience. In this case, I would suggest that it’s very important to write out all of those different types of suffering. After all, the researcher’s starting point is that most humans don’t realize they even happen after a war. Then, what this research does is it “conceptualises compounded suffering as….”
Now, how does one conceptualise something? The dictionary defines ‘conceptualise’ as an intransitive verb which means ‘to form an idea or principle in your mind’.
So, how do other researchers determine if the idea or principle formed in this researcher’s mind is, uh….useful? Well, not by gathering data, or doing an experiment, I am sure. I expect they talk about it. Or, possibly, argue about it, but ‘arguing’ is generally frowned on in the 21st century academic world. That could be regarded as creating an unsafe space, and this is not tolerated.
I could go on about that big sentence above. People in this sort of business are not particularly good on grammar or logic. Consider that last clause
“…produced by the unique conditions of post-war, which in coming together, aggravates and intensifies.”
This refers back to that long list of types of suffering, but how can it be said these are produced by the unique conditions of post-war? Everything on that list occurs during war, too, so what does ‘unique’ mean here?
And then ‘….which in coming together, aggravates and intensifies.”
What exactly is coming together here? Everything on that long list? And ‘aggravates and intensifies’ needs an object. Just what is being aggravated and intensified?
I found another paper by this same academic, titled ‘Coloniality and the inadequacy of localisation’, and here’s the first bit of the abstract of this paper:
“This article uses coloniality as an analytical framework to critique the concept of localisation. It argues that localisation is inadequate to respond to the asymmetrical power dynamic that it seeks to dislodge. Fundamentally, this is because localisation does not account for coloniality, which is the underlying logic of colonialism embedded within the humanitarian sector.”
So, more concepts. I don’t know what ‘coloniality’ or ‘localisation’ mean, or whether Mihlar or someone else ‘conceptualized’ them. A look through the actual paper shows lots of words, and the word ‘evidence’ appears twice in the paper’s 12 pages (yes, I checked). However, nothing one could call evidence, or even examples, is presented or cited. The paper ends with more than 20 references to other works, and I cannot swear there is no evidence in any of those. No, I didn’t check.
If people want to talk about things, and say ‘this thing here is bad’ that’s ok. If people want to discuss whether idea A is better than idea B, or what is the best way to conceptualize Trump’s approach to tariffs, go for it. I talk about things all the time over drinks/dinner with my friends, and there is little if any evidence ever brought to bear on what we say. We just don’t think of what we’re doing as being academic. Or research. Or science. Certainly, no one pays us to do it. Hell, I mostly ‘say things’ on this blog – but it ain’t science.
There is an even more basic way to see the distinction I am making between science and this ‘saying things’.
In the research on honesty, useless as it was, there was a scientific hypothesis being tested. That is, there was a claim made about how the world (more particularly, how the humans in that world) behaves. The claim of interest was:
“If you have humans sign a pledge of honesty at the start of filling out a form asking for unverified information, you will get more honest answers than if you have them sign the same pledge at the end of the form.’
The bad research actually done on this provided no evidence that this claim about the world is true, because it used fake data. The claim might still be true, and other researchers could devise new experiments designed to provide evidence. But, if you do the experiments right, there is the possibility that the evidence will not be found, or it might even be that you find evidence for the opposite- maybe signing at the bottom generates more honest answers.
Scientists (albeit not the data-fakers) put something on the line, and realize that their hypothesis, no matter how much they like it, may not in the end be supported by evidence.
The ‘saying things’ people face no such test. They say things, and other people might disagree, but those people are just saying different things.
I am not the first, by any means, to point out that the ‘thing-sayers’ don’t face this scientific test of the things they say. Thus, the thing-sayers have devised a very clever response. They say that the whole experiment-data-evidence approach is evil- white, male, hegemonic, colonial, etc. Saying this establishes – for them – the superiority of saying things. Ya gotta hand it to them.
When I was supervising undergrad theses in my last years of teaching, students were very fond of writing things like ‘as Flimblat has proven, X’, when all that meant was that that the student had read a paper by Flimblat in which he said ‘X’. Saying ain’t proving, I would correct them. They did not like that. In my experience, the people who write these ‘saying things’ papers are quite convinced of everything they write, and the students, sadly, take confidence for proof.
I leave it to you to decide which is worse for humanity. Academics following scientific procedures of experiment, observation and evidence, but doing it so badly or so fraudulently that it is meaningless or even deceitful,
or
academics just saying things.
Tough call from my perspective, but I do hope Professor Mihlar is not helping design buildings. Perhaps her job is just to keep them free from coloniality. Or to conceptualize how they might be.