Readings We Recommend

Phenomenological and Methodological Determinants of Irreproducibility

Reference

Bak-Coleman, J. B.; Mann, R. P.; Bergstrom, C. T.; Gross, K. & West, J. Replication, varying effects, and the reliability of the scientific literature. SocArXiv, 2022 10.31235/osf.io/rkyf7

Brand, A.; Bradley, M. T.; Best, L. A. & Stoica, G. Multiple Trials May Yield Exaggerated Effect Size Estimates The Journal of General Psychology, Informa UK Limited, 2010, 138, 1-11 10.1080/00221309.2010. 520360

Bressan, P. Confounds in “Failed” Replications Frontiers in Psychology, Frontiers Media SA, 2019, 10 10.3389/fpsyg.2019.01884

Danchev, V.; Rzhetsky, A. & Evans, J. A. Centralized scientific communities are less likely to generate replicable results eLife, eLife Sciences Publications, Ltd, 2019, 8 10.7554/elife.43094

Farrar, B. G.; Voudouris, K. & Clayton, N. S. Replications, Comparisons, Sampling and the Problem of Representativeness in Animal Cognition Research Animal Behavior and Cognition, Animal Behavior and Cognition, 2021, 8, 273-295 10.26451/abc.08.02.14.2021

García-Pérez, M. A. Statistical Conclusion Validity: Some Common Threats and Simple Remedies Frontiers in Psychology, Frontiers Media SA, 2012, 3 10.3389/fpsyg.2012.00325

Gelman, A. & Carlin, J. Beyond Power Calculations: Assessing Type S (Sign) and Type M (Magnitude) Errors Perspectives on Psychological Science, SAGE Publications, 2014, 9, 641-651 10.1177/1745691614551642

Gelman, A. The Connection Between Varying Treatment Effects and the Crisis of Unreplicable Research: A Bayesian Perspective Journal of Management, 2015, 41, 632–643 10.1177/0149206314525208

Gollwitzer, M., & Schwabe, J. (2022). Context Dependency as a Predictor of Replicability. Review of General Psychology, 26(2), 241-249. 10.1177/10892680211015635

Higgins, J. J.; Higgins, M. J. & Lin, J. From One Environment to Many: The Problem of Replicability of Statistical Inferences The American Statistician, Informa UK Limited, 2020, 75, 334-342 10.1080/00031305.2020.1829047

Holmes, S. Statistical proof? The problem of irreproducibility Bulletin of the American Mathematical Society, American Mathematical Society (AMS), 2017, 55, 31-55 10.1090/bull/1597

Iso-Ahola, S. E. Replication and the Establishment of Scientific Truth Frontiers in Psychology, Frontiers Media SA, 2020, 11 10.3389/fpsyg.2020.02183

Kenny, D. A. & Judd, C. M. The unappreciated heterogeneity of effect sizes: Implications for power, precision, planning of research, and replication. Psychological Methods, 2019, 24, 578–589 10.1037/met0000209

Leek, J. T. & Peng, R. D. What is the question? Science, American Association for the Advancement of Science (AAAS), 2015, 347, 1314-1315 10.1126/science.aaa6146

Loken, E. & Gelman, A. Measurement error and the replication crisis Science, American Association for the Advancement of Science, 2017, 355, pp. 584-585 www.jstor.org/stable/24918342

Manly, K. F. Reliability of statistical associations between genes and disease Immunogenetics, Springer Science and Business Media LLC, 2005, 57, 549-558 10.1007/s00251-005-0025-x

Maxwell, S. E.; Lau, M. Y. & Howard, G. S. Is psychology suffering from a replication crisis? What does “failure to replicate” really mean? American Psychologist, American Psychological Association (APA), 2015, 70, 487-498 10.1037/a0039400

Moreau, D. & Corballis, M. C. When averaging goes wrong: The case for mixture model estimation in psychological science. Journal of Experimental Psychology: General, American Psychological Association (APA), 2019, 148, 1615-1627 10.1037/xge0000504

Mullane, K. & Williams, M. Unknown unknowns in biomedical research: does an inability to deal with ambiguity contribute to issues of irreproducibility? Biochemical Pharmacology, 2015, 97, 133-136 10.1002/icd.2295

Muthukrishna, M. & Henrich, J. A problem in theory. Nature human behaviour, 2019, 3, 221–229 10.1038/s41562-018-0522-1

Oberauer, K. & Lewandowsky, S. Addressing the theory crisis in psychology Psychonomic Bulletin & Review, 2019, 26, 1596–1618
10.3758/s13423-019-01645-2

Peters, U.; Krauss, A. & Braganza, O. Generalization Bias in Science Cognitive Science, Wiley, 2022, 46 10.1016/j.bcp.2015.07.002

Ramscar, M. Learning and the replicability of priming effects 80 Current Opinion in Psychology, 2016, 12, 5 10.1016/j.copsyc.2016.07.001

Sanbonmatsu, D. M.; Cooley, E. H. & Butner, J. E. The Impact of Complexity on Methods and Findings in Psychological Science. Frontiers in psychology, 2020, 11, 580111 10.3389/fpsyg.2020.580111

Schaller, M. The empirical benefits of conceptual rigor: Systematic articulation of conceptual hypotheses can reduce the risk of non-replicable results (and facilitate novel discoveries too) Journal of Experimental Social Psychology, Elsevier BV, 2016, 66, 107-115 10.1016/j.jesp.2015.09.006

Scheel, A. M. Why most psychological research findings are not even wrong Infant and Child Development, Wiley, 2022, 31 10.1002/icd.2295

Schwarz, N. & Clore, G. L. Evaluating Psychological Research Requires More Than Attention to the N: A Comment on Simonsohn’s (2015) “Small Telescopes” Psychological Science, SAGE Publications, 2016, 27, 1407-1409
10.1177/0956797616653102

Stanley, D. J. & Spence, J. R. Expectations for Replications: Are Yours Realistic? Perspectives on Psychological Science, 2014, 9, 305–318 10.1177/1745691614528518

Stevens, J. R. Replicability and Reproducibility in Comparative Psychology Frontiers in Psychology, Frontiers Media SA, 2017, 8 10.3389/fpsyg.2017.00862

Stroebe, W. & Strack, F. The Alleged Crisis and the Illusion of Exact Replication Perspectives on Psychological Science, SAGE Publications, 2014, 9, 59-71 10.1177/1745691613514450

Szucs, D. & Ioannidis, J. P. A. When Null Hypothesis Significance Testing Is Unsuitable for Research: A Reassessment Frontiers in Human Neuroscience, Frontiers Media SA, 2017, 11 10.3389/fnhum.2017.00390

Trafimow, D. & Earp, B. D. Badly specified theories are not responsible for the replication crisis in social psychology: Comment on Klein Theory & Psychology, SAGE Publications, 2016, 26, 540-548 10.1177/0959354316637136

Ulrich, R. & Miller, J. Questionable research practices may have little effect on replicability eLife, eLife Sciences Publications Ltd, 2020, 9 10.7554/eLife.58237

Brid, Alexander. Understanding the Replication Crisis as a Base Rate Fallacy

Voelkl, B.; Altman, N. S.; Forsman, A.; Forstmeier, W.; Gurevitch, J.; Jaric, I.; Karp, N. A.; Kas, M. J.; Schielzeth, H.; Van de Casteele, T. & Würbel, H. Reproducibility of animal research in light of biological variation Nature Reviews Neuroscience, Springer Science and Business Media LLC, 2020, 21, 384-393
10.1038/s41583-020-0313-3

Wilson, B. M.; Harris, C. R. & Wixted, J. T. Science is not a signal detection problem Proceedings of the National Academy of Sciences of the United States of America, 2020, 117, 5559–5567 10.1073/pnas.1914237117

Wilson, B. M.; Harris, C. R. & Wixted, J. T. Theoretical false positive psychology Psychonomic Bulletin & Review, Springer Science and Business Media LLC, 2022, 29, 1751-1775 10.3758/s13423-022-02098-w

Yarkoni, T. The generalizability crisis Behavioral and Brain Sciences, 2022, 45 10.1017/s0140525x20001685