Do scientists feel pressure to produce positive results?

Science is supposed to be the impartial study of the natural world through observation and experiment. But a new study from the University of Edinburgh suggests pressure on scientists to report only “positive results” – that is, results that agree with their original hypotheses. These researchers suggest that this pressure is leading to a “decline” of science across the globe.

The formulation of hypotheses is part of the process of science. A hypothesis is a starting point in an investigation. There is supposed to be no assumption of its truth. Here’s an example of a hypothesis: if you jump out of an airplane without a parachute, you will plunge rapidly to the ground. It sounds logical, but, in science, if you want to prove the truth of that hypothesis, or any hypothesis, you need to conduct an experiment.

Many hypotheses, such as the one above, are bound to have positive results. After all, scientists are trained to be careful observers of the world around them, and they base their hypotheses on their observations. What’s striking in the Edinburgh study is not that the papers the researchers examined had positive results, but that the percentage of positive results increased over time.

The researchers examined more than 4,600 scientific research papers published between 1990 and 2007. They found a steady increase in studies whose results agreed with the original scientific hypotheses. Over the study period, positive results grew from around 70 percent in 1990 to 86 percent in 2007. The growth was strongest in economics, business, clinical medicine, psychology, psychiatry, pharmacology and molecular biology. These researchers’ findings – published in Scientometrics – also suggest that papers reporting positive results are more frequent in the U.S. than in Europe.

What’s happening here? Are scientists getting better at their jobs? Are they formulating and stating their hypotheses so well that their percentage of positive results has increased?

The Edinburgh scientists suggest otherwise. They point out that papers reporting null or negative findings are in principle as useful as positive ones. But, the Edinburgh researchers suggest, negative findings attract fewer readers and citations, so scientific journals tend to reject them.

The pressure on scientists to produce positive results is not a new subject in science. Scientists have been discussing this problem among themselves for the several decades I’ve been reporting on science, at least. Now, however, the problem might be worsening, according to the Edinburgh scientists, because – in science as in other fields – competition is growing. Jobs and grants in science are given to scientists who publish frequently in high-ranking journals. Will scientists increasingly pursue predictable outcomes? Will they produce positive results through reinterpretation, selection, or even manipulation of data? Clearly, this outcome would be harmful to science.

There are real-world outcomes as well. In recent years, when scientists’ studies indicate that global warming is both real and human-caused, global warming denialists often accuse them of falsifying or manipulating data for the purpose of “getting grants.” Now more than ever, the reliability and dependability – the trustworthiness – of scientific results need to be spotless if the public and policy-makers are to act on them. This issue is such a hot button in global warming politics, by the way, and the results of this study are so sure to provide fodder for the skeptics, that I wondered who funded the study of the Edinburgh scientists. When I find out, I’ll let you know.

Lead author Daniele Fanelli, University of Edinburgh Institute for the Study of Science, Technology and Innovation, said in a press release:

Either journals are rejecting more negative results, or scientists are producing more positives. It is most likely a combination of both.

Without negative evidence in the literature, scientists might misestimate the importance of phenomena and waste resources replicating failed studies. The higher frequency of U.S. papers reporting positive findings may suggest that problems linked to competition are greater in the U.S. than elsewhere.

Bottom line: A new study from the University of Edinburgh suggests that there is pressure on scientists to report only positive results – that is, results that agree with their original hypotheses. These scientists suggest – and this author agrees, as would, I’ll bet, virtually any scientist – that this practice is harmful to science.