In the past fifteen years, concerns have been growing in scientific communities regarding the challenge of reproducing significant and influential studies, leading to the coining of the term 'replication crisis' in 2009. This heightened awareness prompted the UK government to commission a report on Research Integrity in UK Science from the Science and Technology Committee (a House of Commons Select committee). The report, published in 2018, aimed to address these concerns and explore ways to improve the situation.

Three years later, in July 2022 (just as the UK COVID-19 pandemic restrictions began to be lifted) the Science and Technology Committee launched a follow-up inquiry, this time focusing on reproducibility in and across UK Science. The inquiry aimed to cover three key points:

1. whether there was a 'reproducibility crisis' in UK science and social science research, its scale and causes, and which were the most vulnerable research areas.

2. the roles of different stakeholders in addressing this problem, relevant policies and mechanisms that were needed, and the potential contribution of the UK Committee on Research Integrity (established as part of UK Research and Innovation (UKRI) following the 2018 report).

3. the measures required for an open, contestable, and rigorous research environment in the UK.

The inquiry gathered evidence from various sources, including oral testimony from sixteen witnesses and written submissions from a hundred individuals and institutions in the field of UK science research, which encompassed a wide range of perspectives and which included evidence from BioSS which you can read here. Our written submission focused on what we consider one of the significant contributors to the reproducibility crisis: a lack of critical statistical thinking across the scientific process and different applied sciences. This lack of critical statistical thinking manifests in various ways, such as a failure to differentiate between exploratory and confirmatory science, inappropriate experimental designs, inadequate levels of replication, biased data selection, limited understanding of statistical assumptions and terms, arbitrary thresholds, over-reliance on p-values, inadequate exploration of uncertainty, and over-interpretation of results.

Our perspective on these issues, as statisticians and mathematical modellers, cuts across science fields and application areas, just as most of our own work does! This broader viewpoint enabled us to identify common drivers and issues related to reproducibility, lending significant weight to our evidence. The committee recognized the value of this perspective, and we were delighted to see our evidence directly informing and being quoted in the inquiry report, which was published in April 2023 (and which you can read here).

The report highlights several key conclusions regarding the reproducibility crisis in UK science. Firstly, there is a lack of comprehensive assessment regarding the scale of the problem and the disciplines most affected. Additionally, concerns are raised about the academic publishing industry, which tends to prioritize original and attention-grabbing research rather than confirmatory reproducibility studies. The impact of short-term contracts and grants is also identified as a hindrance to reproducibility efforts. Lastly, there is a recognition of inconsistent and insufficient training in research integrity, reproducibility, and Open Science.

The report provides several key recommendations to address the reproducibility crisis in UK science. It suggests establishing a dedicated sub-committee within UK CORI to focus on reproducibility issues and calls for funders to prioritize reproducibility by providing the necessary resources for reproducibility studies. The report also emphasizes the importance of mandatory training on Open Science and Research Integrity at different stages of a researcher's career. Additionally, it recommends requiring the sharing of data and code as part of open access publication. The report further highlights the need for the wider scientific community and stakeholders to promote broader measures of academic success beyond publications and publication metrics. It also calls on publishers to publish confirmatory studies and commit to timely retractions and error corrections.

The reproducibility crisis continues to be a debated issue, spurring scientific practice reform. Open Science plays a crucial role in generating reliable results and building public trust in scientific findings and experts. However, changing entrenched practices and culture is a gradual process that demands sustained pressure, effort, and support. At BioSS, we have formed the Open Science Support Group, a team dedicated to assisting our staff in various aspects of Open Science, including reproducibility. You can read more about them and BioSS's Open Science activities here.