e6cc70e7 a1fe 4086 b0dd 9cd818fe96da image

Archaeology: The Flaws of Peer Review

March 14, 2025 Trebor2510

Contents

1 Introduction
2 Prologue
3 Further Reading
4 Other Blogs

Introduction

The peer-review system in archaeology is often regarded as a safeguard for academic integrity and quality control. However, it has become a gatekeeping mechanism that suppresses alternative interpretations and enforces conformity to prevailing theories. Michael Gordin, a historian of science, has noted that peer review is often mistaken for a flawless vetting system when, in reality, it is prone to bias and conservatism. He argues that the process tends to favour established paradigms, making it difficult for groundbreaking ideas to gain traction.

Archaeologist E. Thomas Lawson has pointed out that the peer-review system is highly susceptible to institutional biases. In a study published in Science and Technology Studies, Lawson demonstrated how reviewers often reject submissions that challenge dominant models, not necessarily based on weak evidence but because they disrupt the consensus. This issue is particularly acute in archaeology, where entrenched narratives about cultural evolution, settlement patterns, and technological development dominate scholarly discourse. Those who propose alternative chronologies or challenge the attribution of artefacts often find their work dismissed outright, regardless of its empirical rigour.

Archaeology: The Flaws of Peer Review — Most Peer-reviewed papers are ‘Pulp-fiction’ constructed for publicity and funding

A stark example of this suppression can be seen in the case of the Cerutti Mastodon site in California. When Steven Holen and his team published their findings in nature in 2017, suggesting that humans may have been in North America over 130,000 years ago, the backlash was immediate. Critics did not merely question the dating methods; they dismissed the research as implausible despite using multiple independent dating techniques. This reaction illustrates how peer review often functions not as an objective filter but as a reinforcement of accepted timelines and theories, even when the evidence demands reconsideration.

Further, the peer-review system is primarily controlled by a small number of academic gatekeepers. A 2020 analysis by J. Scott Turner in The Journal of Scholarly Publishing revealed that many archaeological journals rely on a narrow pool of reviewers, many of whom share institutional affiliations and research backgrounds. This influence concentration creates an echo chamber where dissenting voices struggle to be heard. Turner’s study found that papers proposing unconventional interpretations had a rejection rate nearly three times higher than those aligned with mainstream archaeological views.

Stonehenge: The Lost Circle Revealed - debunked — Stonehenge: The Lost Circle Revealed – Debunked

This lack of openness is compounded by the anonymous nature of peer review, which allows reviewers to dismiss or delay controversial findings without accountability. In some cases, reviewers have been found to recommend rejection of papers that later, when published through alternative means, proved to be groundbreaking. For instance, the discovery of pre-Clovis human presence in the Americas faced decades of dismissal until mounting evidence from sites like Monte Verde forced a reassessment of long-held beliefs. The delay in accepting such evidence highlights how peer review can be a brake on scientific progress rather than a facilitator of truth.

Another fundamental flaw of the peer-review system is its failure to recognise the evolving nature of scientific inquiry. History has shown that many discoveries initially rejected by the academic establishment later became widely accepted. The heliocentric model, continental drift, and the role of bacteria in ulcers were all dismissed before overwhelming evidence overturned the consensus. In archaeology, similar patterns emerge, where findings that contradict prevailing models are routinely discarded until the weight of evidence makes resistance untenable.

Archaeology must embrace a more open and transparent review process to foster true scientific progress. Double-blind peer review, in which authors and reviewers remain anonymous, could help mitigate personal and institutional biases. Additionally, encouraging post-publication peer review—where studies are critiqued and refined after publication rather than being blocked before dissemination—would allow for a more dynamic and evidence-driven discourse. Greater interdisciplinary collaboration, incorporating fields such as genetics, geophysics, and AI-assisted analysis, could also help challenge entrenched assumptions and promote a more balanced evaluation of new findings.

If archaeology aims to uncover the truth about the past, then the discipline must acknowledge and address the limitations of its review system. Suppressing dissenting ideas only delays the inevitable reevaluation of flawed theories. By reforming the peer-review process to prioritise empirical validity over academic orthodoxy, archaeology can become a more dynamic and self-correcting field that genuinely pursues knowledge rather than merely defending existing paradigms.

Preconceived notions

The peer-review process stands as a cornerstone of modern scientific inquiry, intended to filter research, uphold quality, and ensure the validity of published findings. It is a system whereby experts in a particular field critically evaluate a submitted manuscript before it is accepted for publication, ideally identifying flaws, inconsistencies, and errors. While the intention behind peer review is laudable, the assumption that all peer-reviewed papers are inherently accurate and devoid of error is a dangerous oversimplification. Particularly within archaeology, where interpretation plays a significant role and definitive proof can be elusive, placing unwavering faith in the peer-review stamp can hinder progress and perpetuate flawed narratives. The provided sources offer compelling insights into the limitations of peer review in archaeology, highlighting instances where the process has failed to prevent the dissemination of questionable findings, often due to the inherent subjectivity of the field and a tendency to prioritise the reporter’s preconceived notions over objective facts.

The investigations surrounding the dating and origins of Stonehenge, particularly the peer-reviewed publications stemming from Mike Parker-Pearson’s (MPP) work on Bluehenge, the bluestone quarries and Waun Mawn, serve as stark examples of how the peer-review system can be flawed and how its weight of evidence can be unduly influenced by the author’s reputation and desired outcomes. The case of Bluestonehenge further highlights how prior assumptions can drive research conclusions, even when contradictory data exists. These cases illustrate a broader issue within archaeology: a tendency to align findings with prevailing theories rather than rigorously testing hypotheses against all available evidence.

Ashampoo Snap 2021.02.13 13h59m05s 002 768x452 1 — The Lost Stone Circle – many made up for good TV

The Subjectivity

Unlike the hard sciences that rely on quantifiable data and reproducible experiments, archaeology frequently deals with fragmented evidence requiring interpretation. Archaeological findings are often unique to specific sites and cannot be replicated. This reliance on interpretation opens the door to interpretative bias, where an archaeologist’s perspective, cultural background, or theoretical framework can influence the narrative constructed from the available evidence. Consequently, the peer reviewers, who are also subject to their own biases, may inadvertently endorse interpretations that align with prevailing theories or the author’s established viewpoint rather than rigorously scrutinising the factual basis of the claims.

This can lead to “possibilities are often presented as certainties, creating a misleading view of the past.” The “establishment’s over-reliance on ‘peer review’ as a conformity standard of credibility” has been criticised, with Richard Horton, editor of The Lancet, suggesting that peer review is merely a “crude means of discovering the acceptability—not the validity—of a new finding.” This crucial distinction underscores that peer review may often reinforce existing paradigms within the archaeological community, potentially hindering the acceptance of genuinely novel or contradictory evidence.

102344 richardhorton1 — Ricahrd Horton of the Lancet

The Case of Bluestonehenge: A Shift in Narrative Due to Contradictory Evidence

The first site to consider, Bluestonehenge, was under excavation in 2009. At that time, MPP was preparing to announce it as his bluestone site and believed it predated Stonehenge Phase I. However, the author of The Stonehenge Enigma (2013) raised concerns about this dating during their meeting. Based on his research of the surrounding 50 sites around Stonehenge and his post-glacial flooded environment hypothesis, they pointed out that Bluestonehenge would have been below water during the Neolithic period due to the River Avon remaining flooded.

MPP dismissed this research and published, but later carbon dating confirmed this point and supported the author’s hypothesis, as it is now accepted that Bluestonehenge was constructed between 2840–2230 BCE, which falls into the Early Bronze Age rather than the Neolithic period as initially proposed by MPP. This raises significant concerns about the reliability of peer-reviewed interpretations, as the initial dating—endorsed by peer review—was later found to be incorrect.

Bluehenge — MPP’s first attempt to justify his dates

The Craig Rhos-y-Felin Quarry: Cherry-Picking Evidence to Fit a Hypothesis

The investigations led by MPP into the origins of Stonehenge’s bluestones provide a compelling case study of how the peer-review process can be intertwined with pursuing a specific hypothesis, potentially overlooking contradictory evidence and leading to flawed conclusions. The initial breakthrough of finding the bluestone quarry at Craig Rhos-y-Felin was significant. However, the subsequent attempts to date the quarry and link it definitively to a specific construction date for Stonehenge, particularly around 3300 BCE, demonstrate a tendency to “attempt to find a date that would vindicate their Stonehenge dating methods.”

At Craig Rhos-y-Felin, over 42 carbon dates were obtained. However, the peer-reviewed reports emanating from this investigation were criticised for emphasising only a couple of Neolithic dates that appeared to support the 3300 BCE hypothesis while mainly overlooking the “extensive evidence found in the site’s man-made hearths, which showed that the quarry was first occupied in the Mesolithic Period.” These Mesolithic hearths, dating significantly earlier than the desired Neolithic timeframe, presented a challenge to the pre-existing narrative but were seemingly marginalised in pursuing a specific conclusion. This selective presentation of data, which the sources label as “clear ‘cherry-picking’ of the archaeological evidence,” suggests that the weight of the reported evidence was skewed by the author’s desire to validate a prior claim and that the peer-review process in this instance, did not sufficiently challenge this biased interpretation.

Page 61 Craig y felin excavation — Only accept one of the fourty-two carbon dates which was closest to his hypothesis

Waun Mawn: The Search for a Lost Circle and the Limits of Peer Review

The discrepancy between the desired 3300 BCE date and the slightly earlier dates obtained at the quarry site fueled the hypothesis of a “Welsh prototype stone circle which was moved 500 years later.” This second claim by MPP and subsequent peer-reviewed papers and TV programmes “claimed to find this ‘mysterious site,’ later to discover that it was nonsense raises serious questions about the rigour of the research and the extent to which the peer-review process scrutinised these evolving and ultimately unsubstantiated claims.

At Waun Mawn, only four bluestones were found, leading to a “sad attempt to fit a square peg into a round hole” by identifying ten further stone holes and guesstimating the existence of the remaining stones needed to match Stonehenge. Furthermore, the dating evidence from Waun Mawn is heavily criticised. While the program highlighted a single OSL date that appeared to align with the desired timeframe, the sources reveal that “there was not one OSL sample taken but 18, which gave results in the range of 6980+/- 2120 BCE to AD 1900 +/- 20.” The fact that “17 dating samples did not verify MPP’s Hypothesis, ONLY a single one,” strongly suggests a selective emphasis on a single data point to support a pre-existing claim.

MPP at carn goedog — The dates he found correponded to other carbon dates at Stonehenge but were ignored

Conclusion: Moving Beyond the Illusion of Infallibility

While the peer-review process remains a vital component of archaeological scholarship, it is crucial to recognise its inherent limitations and avoid believing that all peer-reviewed papers are accurate and error-free. The case studies surrounding Mike Parker-Pearson’s investigations into the origins of Stonehenge highlight how the subjectivity of archaeological interpretation, the pursuit of pre-existing hypotheses, and the potential influence of the reporter’s authority can lead to the publication of flawed findings. The selective use of data, the downplaying of contradictory evidence, and the development of speculative theories underscore the need for a more critical engagement with peer-reviewed publications, prioritising rigorous examination of the factual evidence and methodologies employed rather than relying on perceived authority.

Prologue

Archaeological research, while often considered a rigorous discipline, faces challenges concerning the reach and reliability of its published work. Studies have shown that a significant portion of academic papers receive minimal attention post-publication. Approximately 44% of all published manuscripts are never cited, indicating that nearly half of these works remain largely unread or unreferenced within the scholarly community.

lucbeaulieu.com

Furthermore, citation analyses reveal that the average number of citations per paper is below ten. Achieving ten or more citations places a manuscript in the top 24% of the most cited works worldwide, underscoring the limited engagement many papers receive.

lucbeaulieu.com

Regarding the accuracy of scientific publications, the rate of retractions has seen a notable increase. Although retracted papers still constitute a small fraction—under 0.1%—of the more than 25 million papers indexed in PubMed since the 1940s, this uptick highlights ongoing concerns about the reliability of published research.

en.wikipedia.org

In the realm of archaeology, the prevalence of flawed studies is challenging to quantify precisely due to the field’s interpretative nature and the scarcity of replication studies. However, documented cases, such as the Himalayan fossil hoax involving Vishwa Jit Gupta, illustrate that significant errors and even fraudulent activities can persist undetected for extended periods, thereby influencing subsequent research.

en.wikipedia.org

These insights emphasize the necessity for heightened scrutiny and critical engagement within archaeological scholarship to ensure the integrity and impact of its contributions.