social sciences

Spuriously correlated: Negative results in social science

In their publication from 2007, the social scientists David Lehrer, Janine Leschke, Stefan Lhachmini, Ana Vasuliu and Brigitte Weiffen described the impact of negative results in their research area. Besides their excellent work in defining and classifying negative results, their publication aims to introduce the Journal of Spurious Correlations, which is dedicated to increase transparency in research.

The authors define negative results as “[…] findings that are validated outside the research context in which they are generated, but not by standards of the heuristic process that generated them”. So, in my reading, negative results are unexpected ones. They also address the problem to distinguish those unexpected findings from mistakes.

Being a physical scientist, I absolutely agree with their argumentation why negative results are of an important value for science. Their categorization into four classes (inconclusive results, non-results, confutative results and ersatz results) is something that in my opinion might not be transferred directly to other sciences. As the authors clearly state, social sciences have distinct methods for perfoming studies and evaluating data. They might be also similar for several aspects physical sciences (e.g. statistical data evaluation), but maybe not always.

Most interestingly is their attempt to overcome this obstacle with the Journal of Spurious Correlations. However, I was not able to find any article published, while the latest news entry is from december 2007. This correlates with a statement of Douglas McCormick from the same year.

Couple that with most researchers’ reluctance to publicly air what they consider mistakes, and with the difficulty of finding reviewers canny enough to separate the null-result wheat from the ill-executed chaff, and you wind up with some significant doubts about the workability of the project.