Month: August 2014

The journals of negative results in science

Upto now, I found five journals dedicated to publish negative results. While only two of them provide new content on their websites (i.e., entries from at least 2014), one of these two has published no more than 25 articles in its twelve-years long history. The three “silent” ones are:

The one actually running exception I found so far, is the Journal of Negative Results in Biomedicine, hosted by a professional publisher. Worth to mention is also the campaign of the biomedical F1000 Research journal in 2013, when manuscripts about negative results were not been charged for publishing. Please send me a note if I missed something.

So far, there is clearly a need to publish negative, unexpected, or contradictive findings and there are indeed attempts to give them a shelter. The interesting aspect is that publishing negative results seems to require a strong lobby, like from a publisher. One might expect that an open-science solution should work just “out of the crowd”. However, in my opinion the critical point is the question why scientists might actually want to publish negative findings. Of course, there are many good reasons, but in the end of the day most scientists do not dare to make their “failures” public, as every publication counts in the CV. And no one really likes to tell the world what he did not manage to achieve.

Spuriously correlated: Negative results in social science

In their publication from 2007, the social scientists David Lehrer, Janine Leschke, Stefan Lhachmini, Ana Vasuliu and Brigitte Weiffen described the impact of negative results in their research area. Besides their excellent work in defining and classifying negative results, their publication aims to introduce the Journal of Spurious Correlations, which is dedicated to increase transparency in research.

The authors define negative results as “[…] findings that are validated outside the research context in which they are generated, but not by standards of the heuristic process that generated them”. So, in my reading, negative results are unexpected ones. They also address the problem to distinguish those unexpected findings from mistakes.

Being a physical scientist, I absolutely agree with their argumentation why negative results are of an important value for science. Their categorization into four classes (inconclusive results, non-results, confutative results and ersatz results) is something that in my opinion might not be transferred directly to other sciences. As the authors clearly state, social sciences have distinct methods for perfoming studies and evaluating data. They might be also similar for several aspects physical sciences (e.g. statistical data evaluation), but maybe not always.

Most interestingly is their attempt to overcome this obstacle with the Journal of Spurious Correlations. However, I was not able to find any article published, while the latest news entry is from december 2007. This correlates with a statement of Douglas McCormick from the same year.

Couple that with most researchers’ reluctance to publicly air what they consider mistakes, and with the difficulty of finding reviewers canny enough to separate the null-result wheat from the ill-executed chaff, and you wind up with some significant doubts about the workability of the project.

Ben Goldacre’s talk about publication bias and its consequences in medicine

In this very interesting talk on TED , Ben Goldacre explains the consequences of publication bias in medical studies.

In fact, there have been so many studies conducted on publication bias now, over a hundred, that they’ve been collected in a systematic review, published in 2010, that took every single study on publication bias that they could find. Publication bias affects every field of medicine. About half of all trials, on average, go missing in action, and we know that positive findings are around twice as likely to be published as negative findings.

If you (like me) prefer to read instead of listening, you can find the transcript here.