Science’s integrity cops
The campaign to root out scientific misconduct fundamentally misunderstands the nature of science.
I have a recent publication in Minding the Sciences concerning the “irreproducibility crisis” (also variously named the “replication crisis” or “reproducibility crisis”). The crisis, however it’s named, was precipitated by a landmark paper by UCLA’s John Ioannidis, who examined several thousand papers in the biomedical sciences for how well they measured up to the supposed gold standard for good science: whether the results can be replicated. Very few papers lived up to the standards, and Ioannidis concluded that, in his words, “most published research findings are false.” As I wrote in Minding the Sciences, that “put the cat amongst the pigeons.” To quote from the article:
“Nature magazine devoted a special issue to the problem, including several mea culpas, that probably we could do better. Ecologists have chimed in to admit that “82-89% of ecological research … has limited or no use to the end user” and say study preregistration is likely to “reduce research waste”, so we should do it. Not so fast, say others. What’s the best use of our time and resources: replicating garbage results, or tending to our own work? Still others have asserted that the irreproducibility crisis is no big deal, and attention to it can be a diversion from performing “robust and efficient science.” Psychology and the behavioral sciences have come down solidly in favor of replication and pre-registration, and (perhaps defensively) they have data. Replication is better than “meta-analysis”, which takes many studies and applies statistical magic dust to tease out something sensible.”
The Minding the Sciences article was prompted by a high-profile retraction of a paper in Nature Human Behavior that supposedly demonstrated that replication works, and it is good. Unfortunately for that message, NHB retracted the paper because it failed to follow the protocols that are supposed to make replication work. Oh well.
There’s a deeper context to this story. I wasn’t able to flesh it out in the Minding the Sciences article, so I’ll do so here. This is a bureaucratic solution to a cultural problem. As such it will either be ineffective or do active harm to the thing it is intended to protect.
The cultural problem is that science is presently governed by a landscape of perverse incentives that both encourages and subsidizes disintegrity. If there is disintegrity in science (and I don’t doubt that there is), it will not be solved by looking for examples of it and punishing the perpetrators. Finding examples is virtually guaranteed because disintegrity is built into modern science. It will only be solved by addressing the perverse incentives that encourage it.
Stephen Turner and Darryl Chubin have outlined the cultural problem in their provocative essay The Changing Temptations of Science. Where science once was governed by an ethic of discovery, it is now governed by an ethic of production. With the transformation has come a shift in the ethical norms that govern scientists’ careers. In science governed by an ethic of discovery, the ethical norms are the so-called Mertonian norms, encapsulated in the acronym CUDOS (Figure 1).
In a science governed by an ethic of production, in contrast, the ethical norms are encapsulated in the acronym PLACE. PLACE is where science presently sits. Science is a competition, in which the currency of career success is governed by numbers: of grants brought in, of new scientists produced, and of papers published. It is no wonder that the voluminous scientific literature (460,000 published annually) is filled with unreadable, unread, superfluous, and yes, dishonest, publications. If we are to restore integrity to the sciences, returning science to an ethic of discovery is the obvious course of action to follow.
Figure 1. Screenshot from my presentation at the 2024 Heterodox Academy Conference, in my symposium Rescuing Science.
What has driven the transition to an ethic of production has been the experiment, begun in 1950, to federalize academic scientific research. Since 1950, spending on academic research has increased exponentially (Figure 2). Whereas prior to 1950, governments played only a minuscule role in funding academic science, governments are now the majority funder, accounting for about 60% of total expenditures for academic research.
Figure 2. Screenshot from my presentation at the 2024 Heterodox Academy Conference, in my symposium Rescuing Science.
With government’s increasing role has come demand for metrics of productivity. Universities have responded by evaluating scientists by the metrics of production outlined above. And because universities take roughly a third of the $100 billion channeled to academic research each year, scientists have found themselves increasingly bound to the ethics of production. Hew to those dubious metrics, or you don’t have a career.
Scientists are now reduced to being the turnkeys for opening up the spigots of government spending on science, which disproportionally empowers institutions, not scientists. Frankly, it’s a small wonder that there’s any integrity left in the sciences. What’s not a surprise is the rampant disintegrity that now prevails: we’re subsidizing it, and lucratively.
As science has become more and more a bureaucratic enterprise, the more it has stifled an essential creative dimension of science. As I put it in my 2017 book Purpose and Desire, describing how my research partners and I carried out an off-the-wall experiment with termites that we dubbed “complete moundectomy”:
“And because these were termites, we could act on our whim—glorious freedom!—safe from the bureaucratic killjoys that would have to be given a say if we were studying any creature that had a backbone.”[1]
If I had been working on say, voles, I would have had to jump through an existing gauntlet of regulations, bureaucratic procedures for multiple permits, inspections to ensure I was following their rules, and other annoyances that had no point other than to insert someone in the way of my doing a spontaneous experiment, conducted on a whim, that ultimately proved to be the foundation for a fruitful line of research to follow.
All this to say: the bureaucratic impulse to “manage science” is likely to snuff out the creative spark that is the real driver of science. This is not due to any malevolent motives on the part of the science bureaucracy. Of course, everyone wants science to be honest. But more importantly, we want science to do what it is intended to do, namely push back on the frontiers of our understanding of the natural world. The trouble is that bureaucrats (and lawyers and politicians) fundamentally misunderstand what science is. More troubling is that scientists generally don’t either.
Take, for example, the Reference Manual on Scientific Evidence, prepared by the National Academy of Sciences, as a guideline to judges, attorneys, and anyone else working at the interface of science and the law. It contains a chapter, How Science Works by David Goodstein, Professor of Physics at CalTech, which is an admirable summary of the prevalent understanding of a scientist of what science is. Scientists’ understanding of science falls generally into three categories:
Baconian science, otherwise known as the scientific method. Baconian science is governed by dispassionate observation of nature.
Popperian science, in which science operates through falsification of theory, and
Kuhnian science, in which science proceeds through scientific revolutions, a fundamental shift in prevailing theories, as in the overthrow of the Ptolemaic geocentric universe by the Copernican heliocentric universe.
Admirable though Goodstein’s summary might be, it may fail to encompass what science actually is, and so may fundamentally misdiagnose science’s integrity problem. Michael Streven’s book, The Knowledge Machine makes a compelling case that none of these models accurately describes how science progresses.[2] Most provocatively, he makes the case that major advances in science, from the Copernican universe, to Mendel’s genes, to Eddington’s experimental “proof” of Einstein’s general relativity, to Watson and Crick’s discovery of the structure of DNA, are permeated through and through with evasions, fudging of data, selective presentation, and sometimes outright dishonesty that would bring these discoveries under the jaundiced eyes of the science integrity cops.
Strevens’ most provocative idea is that science progresses not through adherence to one or more of the prevailing models for what science is. Rather, science progresses through a long adversarial conversation among scientists that involves imperfect competing claims, tools of persuasion, including using data as weapons and rhetorical devices to undercut rivals, and many other questionable behaviors. The scientific literature does not reflect this essential character of science or the process of discovery, so much so that Sir Peter Medawar has described the scientific paper as a kind of benign “fraud.”[3] The scientific literature is in fact a form of Kabuki theatre.[4] To look for dishonesty there is to fundamentally misunderstand science.
The solution? If we want to restore science to the ethical norm of discovery, we will have to take away the forces directing science to its present maladaptive ethic of production. Here is the uncomfortable part. Doing so will mean taking away the money that is distorting science’s ethical landscape. The experiment begun in 1950 must be acknowledged as a failure.
J Scott Turner is Director of Science Programs at the National Association of Scholars
The full presentation of Rescuing Science is available here:
[1] Chapter 1. Turner, J. S. (2017). Purpose and Desire. What Makes Something “Alive” and Why Modern Darwinism Has Failed to Explain It. New York, HarperOne.
[2] Strevens, M. (2020). The Knowledge Machine: How Irrationality Created Modern Science, Liveright.
[3] Medawar, P. B. (1963). Is the scientific paper a fraud. The listener 70(12): 377-378.
[4] E.g. Mullis, K. (2010). Dancing Naked in the Mind Field, Knopf Doubleday Publishing Group.
Watson, J. (2012). The Double Helix, Orion.