Noetic Now

About Noetic Now »

« Previous Post Next Post »

Experimenter Effects and Replication in Psi Research

by Marilyn Schlitz

An important unresolved question in scientific research in general, and in psi research in particular, is the effect that the expectancy of the experimenter has on experiment outcomes. I am pleased to report that two distinguished colleagues and I have received funding to conduct a new experiment to test the hypothesis that the expectancy of the experimenter will have an impact on the outcome of a psi task. We are now recruiting teachers to help us recruit student experimenters. Here is a brief history of this field of investigation to give a sense of what we're investigating. I'll conclude with details about the new experiment at the end of this post.

Psi research, the study of ESP and psychokinesis (mind over matter), is a controversial topic in science. Although proponents of psi can point to an extensive body of evidence for phenomena such as telepathy and precognition, most academic psychologists still do not believe that they are likely to exist (Bem, 2011). Central to the debate is the issue of replication. Can independent investigators replicate reportedly successful psi experiments?

An important factor in determining the success or failure of attempted replications appears to be the experimenter’s orientation toward the phenomenon under investigation—attitudes toward and expectations about the outcome of the experiment. In mainstream psychology, Rosenthal has demonstrated experimenter expectation effects in more than 300 studies, including studies in classroom and clinical settings (Rosenthal, 1978).

Experimenter effects have been observed in psi research for more than 70 years, with some researchers actually suggesting that evidence of psi is due less to gifted subjects than to the person who does the testing. Stanley Krippner, a humanistic psychologist, summarized findings showing differences among experimenters (e.g. Ramsey and Cabibbo, 1975), among data collectors (Johnson, et al. 1972), different reciprocal attitudes between experimenter and subject (Nash 1968), and differences across time by the same experimenter (Rivers 1950).

Although the experimenter effects are usually attributed to conventional sensory communication, some researchers have suggested that some of the variation may actually be caused by psi. For example, in one study, subjects did better at guessing psi targets prepared by a psi proponent than on those prepared by a psi skeptic.

There is also evidence that an experimenter can remotely influence a subject's responses through the mediation of psi. For example, my own work with William Braud (1997) reported that experimenters could influence subjects’ physiology from a distance. Using this same protocol, psi-skeptic Richard Wiseman and I collaborated in three attempted replications using the same subject pool and procedures. I obtained significant psi effects in two of the three experiments, but Wiseman failed to get an effect in any of them (Schlitz et al., 2005).

Retroactive Facilitation of Recall: A Replicable Protocol?

One area of psi research that has received recent attention is the study of precognition (knowledge of the future). Experimental tests of precognition have been reported for more than half a century, and a meta-analysis (evaluation across experiments) found that 309 precognition experiments yielded a small but significant success rate (Honorton and Ferrari, 1989).

In an article published in a leading psychology journal, psychologist Daryl Bem reported 9 experiments that tested for retroactive influence by “time-reversing” well-established psychological effects so that the individual’s responses were obtained before the putatively causal stimulus events had occurred (Bem, 2011). All but one of the experiments yielded statistically significant results, and results across all 9 experiments were statistically significant overall.

To encourage exact replications of his experiments, Bem created software that runs them and a detailed instruction manual, including the experimenter’s script. As a result, several replications have now been attempted, most of them adopting the protocol of Experiment 9, “The Retroactive Facilitation of Recall.” In this experiment, participants are first shown a set of words and then given a free-recall test of those words. They are then given a set of practice exercises on a randomly selected subset of those words. The psi hypothesis is that the practice exercises will retroactively facilitate the recall of those words and, hence, subjects will have recalled more of the to-be-practiced words than the unpracticed words. Bem later confirmed this observation, although other investigators have gotten mixed results. Controversy over the research has been featured in the media and scientific discussion across the world.

A particularly striking set of findings implicating experimenter effects emerged from an attempted replication of the experiment by Subbotsky, a developmental psychologist in the UK. In his own experiments, Bem ran a set of control sessions in which the computer selected the subsets of practiced and non-practiced words but practice exercises were not actually administered. Bem reported non-significant findings on these control sessions: Experimental participants recalled words from the two subsets equally well. Subbotsky’s (2012) study successfully replicated the retroactive facilitation effect for the experimental sessions, but he found a significant reversal on the control sessions: Experimental participants recalled significantly more of the non-practice words than the to-be-practiced words. The main difference between Bem’s and Subbotsky’s control sessions was that Bem’s experimenters were blind with respect to whether they were running an experimental or a control session, whereas Subbotsky’s experimenter knew when he was running a control session. These findings suggest that the retroactive-facilitation-of-recall experiment is a promising vehicle for a more systematic examination of experimenter effects in psi research.

In a new experiment, recently funded by the Bial Foundation in Portugal, I am working with Daryl Bem and Arnaud Delorme to test the hypothesis that the expectancy of the experimenter will have an impact on the outcome of a psi task. In particular, we will explore one of the most controversial aspects of psi research: precognition. The protocol involves three levels: (1) teachers who will recruit student experimenters, (2) student experimenters who will receive a standardized training in the experimental procedure, (3) participants in the study who participate in the psi task. We are now recruiting teachers who can help us recruit student experimenters. In this process, they will help us test this remarkable hypothesis that challenges the foundation of scientific assumptions about objectivity and replication. For more information, or if you are a teacher who can help us recruit student experimenters, please write to me at

Bem, D. Feeling the Future: Experimental Evidence for Anomalous Retroactive Influences on Cognition and Affect. Journal of Personality and Social Psychology, 100, 407-425, 2011.
Honorton, C. Replicability, experimenter influence, and parapsychology: An empirical context for the study of mind. Paper presented at AAAS, Washington, D.C., 1978.
Honorton, C. & Ferrari D.C. (1989) Future telling”: A meta-analysis of forced-choice precognition experiments, 1935–1987, Journal of Parapsychology, v.53, 281-308.
Kennedy, J. E. & Taddonio, J. L. Experimenter effects in parapsychological research. Journal of Parapsychology. 40:1-33,1976.
Krippner, S. Commentary. In Rosenthal, R. and Rubin, D. B. (1978) Interpersonal expectancy effects: The first 345 studies. The Behavioral and Brain Sciences, 3, 398- 399, 1978.
Krippner, S & Murphy, G. Psychical research and human personality. Proceedings of the Society for Psychical Research. 48:1-15, 1949.
Nash, C. B. Comparison of ESP run-score averages of groups liked and disliked by the experimenter. Journal of the American Society for Psychical Research. 62:411- 417, 1968.
Ramsey, M., & Cabibbo, C. Experimenter effects in extrasensory perception. Journal of the American Society for Psychical Research. 69:135-149,1975.
Rhine, J. B., Pratt, J. G., Stuart, C. E., Smith, B. M., & Greenwood, J. G. Extrasensory Perception After Sixty Years. Holt, New York, p 341, 1940. Ritchie SJ, Wiseman R, French CC. Failing the Future: Three Unsuccessful Attempts to Replicate Bem’s ‘Retroactive Facilitation of Recall’ Effect. PLoS ONE 7(3): e33423, 1992.
Rivers, O. B. An exploratory study of the mental health and intelligence of ESP subjects. Journal of Parapsychology. 14:267-277,1950.
Rosenthal, R. The silent language of classrooms and laboratories. In: W. G. Roll, R. L. Morris, & J. D. Morris (eds.), Proceedings of the Parapsychological Association, 1971. Durham, N.C.: Parapsychological Association, 1972. Replicability and experimenter influence: Experimenter effects in behavioral research. Paper presented at AAAS, Washington, D.C., 1978.
Schlitz, M., Wiseman, R., Radin, D., & Watt, C. Of two minds: Skeptic-proponent collaboration within parapsychology. Paper presented at the meeting of the Parapsychological Association, Petaluma, CA, 1995.
Schlitz, M. & Braud W. Distant intentionality and healing: Assessing the evidence. Altern Ther Health Med, 3(6):62-73, 1997.
Subbotsky, E. Sensing the future: The Non-standard observer effect on an ESP task. Lancaster University, UK, 2012.
West, D. J. & Fisk, G. W. A dual experiment with clock cards. The Journal of the Society for Psychical Research. 37:185-189, 1953.
White, R. A. The influence of the experimenter on psi test results. In: B. B. Wolman (ed.). Handbook of Parapsychology. Van Nostrand Reinhold, New York, 1977.

Noetic Research
  • Anonymous Icon

    Giorgio Piacenza Cabrera Jan 18, 2013

    The experimenter's SUBCONSCIOUS dispositions may be the culprit. These may be geared for experiences with or without a linear time framework and with or without an Aristotelian kind of classical logic. In these subcosncious dispositions RELATION between experiences -in whatever order- may be what is significant. If the physical world is a limited actualization of a mental realm that subsumes space and time and in which there are equal relations between its exterior objects and its subjective experiences those parts of our subconscious minds focused upon actualizing experiences without any particualr sequential order may activate a different form of causal connections with what Jung and Pauli understood as ther "IMAGINAL" world, a realm that remains linked with the space time structured physical world and which can cause it to organize itself.

  • Anonymous Icon

    monijan Feb 02, 2013

    Everybody knows that you can't describe or control true deep love. The moment you do so it will be destructed. Those tests are the try to control which results can only show the intentions of the controler and nothing of the substance. Like you can't push a button to produce love are those expiriences of prcognition, premonition, telepahy etc. to produce under controled circumstances. You will become something else. I think you would not dissect an animal to proof that it lives.
    I ask why you want to proof such. Is this not in principal to show something about the tester? Fear.
    I would say that You have less fear than your opponents because you are willing to accept . But nobody who recognise and accept this gifts for his/her own should allow to be controlled.
    I'm not against to show people who we really are ,our potentials ,but elsehow. Your opponents want still to burn sorcerers and sorceresses at the stakes and you are in fear they would do so. It is still a big problem in sience. Just me with warm regards from my heart

Stay in touch with IONS