The Psychologist's Green Thumb

Abstract

The ‘psychologist’s green thumb’ stands for the assertion that an experimenter needs an indeterminate set of subtle skills or “intuitive flair” (Baumeister, 2016) in order to be able to successfully show or replicate an effect. This argument is sometimes brought forward by authors whose work has failed to be replicated on independent replication attempts, to explain a lack of replicability. On the one hand, this argument presenting replication failure as a failure on the part of the replicator seems ad-hoc. The ‘failed’ replications are typically more highly powered, more transparently carried out, and better described than the corresponding original studies. And yet, the original authors argue that the problem lies not at all with the study or the effect but with the replicator’s skill. References to flair and a lack of experimenter skill as explaining replication failures have consistently been quickly rejected by meta-researchers and others connected to the reform movement. On the other hand, there are conditions under which the psychologist’s green thumb argument will be potentially compelling, as the generation of some scientific evidence does require something like a ‘green thumb’ (e.g., Kuhn, 1962). Furthermore, it is not clear how we can distinguish between a replication failure that is due to the absence of the effect and one due to lack of skill without knowing whether the replicator is skilled or whether there is an effect (Collins, 1992). The original author, having previously ‘found’ the effect, may claim to have skills the replicator lacks and thus be able to make this distinction. Moreover, failed replications may result in the explication of hidden auxiliary hypotheses representing tacit, ‘green thumb’ knowledge or skill, leading to productive advances through “operational analysis” (Feest, 2016). Therefore, the idea that one needs a certain skill set to be a ‘successful’ experimenter may be convincing and less ad-hoc. In this talk, I will argue that initial biased reasoning towards a desired result is often a more likely cause of low replicability, even in contexts where appeals to ‘green thumb’ tacit knowledge arguments are conceptually persuasive. I will begin by investigating the conditions under which the psychologist’s green thumb is a persuasive concept. I will come to the preliminary conclusion that if experimenter skill takes the form of tacit knowledge that is not or seemingly cannot be shared, then a replicator may appear to lack the psychologist’s green thumb. However, it is unclear whether alleged ‘green thumb’ tacit knowledge amounts to A) experimenter skill to find evidence of a true effect, or B) biased reasoning towards a desired result. Given metascientific evidence regarding publication bias and the widespread use of questionable research practices, B) is likely a better explanation for many replication failures than the psychologist’s green thumb. In the context of field-wide replication failures, ‘green thumb’ tacit knowledge is a red herring at best – what is really at stake here is the articulation of background assumptions. We should strive towards experimental processes that can be and are sufficiently described for reproducibility and in principle replicability.