Skip to article

    Door 06 · Strand 06

    What Would Count as Evidence of a Genuine Extraordinary Ability?

    A strong test defines the claim, rules out ordinary routes, reports uncertainty, and invites independent replication.

    3 min read489 wordsSource-aware essay · Preview only

    Claim status in this essay

    • Established evidence
    • Contested
    A sealed protocol envelope and measuring tool sit before a frosted divider, with another research team in the background.

    Article

    A compelling story can begin an investigation, but it cannot finish one. To test an extraordinary ability, researchers need to state what the person is claimed to do, what would count as success, and how the result differs from chance or ordinary information routes. The standards should be clear before the result is known.

    Define the claim before testing it

    An operational claim names the action, source or receiver if relevant, timing, outcome, and conditions. A telepathy test might specify which target a participant must identify and the chance level expected from guessing. A psychokinesis test might name the machine output and the direction or size of the predicted effect. If a claim shifts after a result, researchers cannot tell whether the original prediction succeeded.[1][5]

    The measured outcome should be objective enough for another team to score the same way. Researchers should state what would count as a meaningful effect, not only whether a threshold was crossed. A p-value alone does not tell us the size, importance, precision, or truth of a claim; effect sizes and uncertainty belong beside it.[5]

    Close ordinary routes for information

    Blinding and randomization can reduce cues and selection effects when the design allows them. Teams should check for sensory leakage, target-handling clues, experimenter knowledge, scoring choices, and flexible analyses. These checks do not imply that a participant is deceptive. They ask whether ordinary information could have reached the outcome through a route the procedure did not intend.[1][2]

    Preregistration records the hypothesis, exclusions, outcomes, and analysis before the data are known. Registered Reports move peer review before results, which can reduce pressure to favor striking findings. Neither practice makes a study immune to error, and exploratory results can still be valuable when identified as exploratory rather than presented as a prior prediction.[2][3]

    Make the result repeatable and inspectable

    Adequate power helps a study estimate a plausible effect with useful precision. Full reporting should include outcomes, effect sizes, uncertainty, deviations from plan, and null results. Sharing data and code can help others inspect an analysis when privacy, consent, and the study design permit it. None of these practices guarantees that an interpretation is correct, but each makes the evidence easier to examine.[1][3][5]

    Independent replication matters because a finding can depend on one laboratory’s sample, materials, or procedure. The Open Science Collaboration tested 100 psychology studies, a coordinated effort that illustrates the value and difficulty of replication. Its results should not be treated as a universal rate for all fields or as a verdict on every contested topic.[4]

    Let confidence grow with the evidence

    A clear preregistered positive result is a reason to investigate further, not automatic proof of an ability. A null result limits the tested design and effect, not every imaginable version of a claim. Confidence becomes more reasonable when independent teams reproduce a defined result, the estimate is precise enough to interpret, and ordinary explanations have been seriously examined.[1][2][4]

    If a surprising claim were tested again, which safeguard would most change your confidence: a blinded procedure, a predeclared analysis, open reporting, or replication by an independent team?

    Trace the claims

    Sources & further reading

    Citation numbers in the essay lead to the source notes below. Each source is linked to its publisher, journal, DOI, institutional record, or catalog entry.

    1. 1

      National Academies of Sciences, Engineering, and Medicine · 2019

      Reproducibility and Replicability in Science

      A consensus report on reproducibility, replication, statistical power, preregistration, and transparent reporting.

    2. 2

      Chris D. Chambers and Olmo R. Tzavella · Nature Human Behaviour 6 · 2022 · 29–42

      The Past, Present and Future of Registered Reports

      Explains pre-results peer review; registered reports do not guarantee freedom from every bias or error.

    3. 3

      Brian A. Nosek et al. · Proceedings of the National Academy of Sciences 115(11) · 2018 · 2600–2606

      The Preregistration Revolution

      Explains how preregistration distinguishes confirmatory plans from exploratory analyses.

    4. 4

      Open Science Collaboration · Science 349(6251) · 2015 · aac4716

      Estimating the Reproducibility of Psychological Science

      A coordinated replication project across 100 psychology studies; its results are not a universal replication rate for all science.

    5. 5

      Ronald L. Wasserstein and Nicole A. Lazar · The American Statistician 70(2) · 2016 · 129–133

      The ASA’s Statement on p-Values: Context, Process, and Purpose

      Explains why a p-value alone does not measure effect size, importance, or the probability that a hypothesis is true.