'Open Research' is an umbrella term for a range of practices. Some of which, such as Open Access to publication outputs, will be familiar to researchers across different disciplines; others, such as preregistration of research studies, will be familiar to some disciplines; while other practices, such as 'Open Peer Review', may be at a relatively early stage of being widely rolled out within the scholarly communications ecosystem.
The terms 'Open Research' and 'Open Science' are sometimes used interchangeably. While some of the challenges and practices outlined in this guide may apply more specifically to Physical or Biomedical Sciences, there will also be information relevant to Social Scientists and Arts and Humanities researchers. 'Open Research' is therefore used with the intention that it operates as a more holistic and inclusive term.
The separate tabs of this LibGuide aim to provide an introduction to Open Research practices, however these will look slightly different in each academic discipline.
In order to understand Open Research practices, it's important to understand the cultural shift which are aiming to be brought about by them. The challenges within academic research practice, publishing and dissemination are well documented, with some of the main challenges being detailed within this guide.
Practical suggestions for what steps researchers can take at different stages of the research and publication process to support Open Research practice are also provided. Some of these steps may be more relevant to some disciplines than others, but hopefully can provide researchers with ideas as to what steps they may be able to take to support open practice.
Additional training resources are also listed in the Further Resources.
If researchers have any suggestions for additions to this section of our LibGuide, for instance by highlighting helpful training resources in this area, we would welcome you getting in touch with us.
Open Research Practices can help to address several challenges which have arisen within academic publishing in recent years, as well as assisting with the production of rigorous and robust research.
Below is a list of some of the present 'problem areas' within the research publishing ecosystem which Open Research Practices seek to address.
The ability to reproduce research studies to ascertain whether the same results are produced, is an essential part of the scientific verification process. However, it has emerged over the past 15-20 years that many landmark studies in the Social and Biomedical Sciences cannot be reproduced to confirm the accuracy of their findings. A paper published in PLOS Medicine (Ioannidis, 2005) helps to highlight why this poses a significant problem.
There are a variety of reasons as to how the replication crisis came about, and the various factors listed below all have a role to play. The historically low rates of data sharing have also significantly contributed to a lack of research reproducibility and transparency.
There is still an ongoing concern that academic journals are being asked to fulfill two contrasting functions which they are not necessarily best suited for. Those functions being both to disseminate useful research findings to practitioners and researchers, and to act as the primary research record in terms of documenting the methodological steps in the process. Concerns that journals are not the best outlet for the second of these functions has prompted the funding of a new platform, Science Octopus, which has received financial support from UKRI and which is intended to act as 'the primary research record'. The Octopus platform is due to go 'live' for real submissions in Spring 2022.
HARKing involves retrospectively adapting a study to fit with the results generated. As defined in a 1998 paper on the topic this involves: 'presenting a post hoc hypothesis in one's research report as if it were, in fact, an a priori hypothesis' (Kerr, 1998).
There are several negative outcomes for the integrity and dissemination of science which can result from HARKing. Among these are:
The overarching cost of HARKing, as identified by Norbert Kerr in their 1998 article on the topic, is that at its heart, the practice represents an implicit 'violating [of] basic ethical principles'.
The temptation to engage in 'HARKing' can be effectively dealt with through the pre-registration of research studies. This provides researchers with clear parameters as to how research is being carried out and the purposes behind it.
P-hacking is the manipulation of a data analysis in order to make the outcome appear more statistically significant. Also known as 'selective reporting'.
Practices which can result in p-hacking include:
Data Dredging involves testing multiple hypotheses using a single data set, at the risk of producing spurious correlations.
A 2015 article published in PLOS Biology (Head, 2015) found that p-hacking 'is widespread throughout science', although the paper also concluded that its effect upon scientific consensus drawn from meta-analysis was limited.
The pre-registration of study protocols is an effective way of preventing the practices detailed above.
'Publication bias' is the term used to account for the fact that the great majority of scientific papers provide 'positive results'. Between 1990/1 and 2007 the number of scientific papers providing positive results increased from just over 70% to just under 86% (Mlinaric, 2017).
There are various harmful consequences for negative / inconclusive results not being disseminated. These include:
Both journal editors and researchers have potentially contributed to the growth of 'publication bias'; editors, due to the fact that some journals prioritise 'novel' research results, and authors, due to the fact they wish to publicise their research successes rather than disappointments. Researchers' reluctance to share negative and inconclusive results is sometimes known as the 'file drawer effect', due to the results languishing in drawers and not seeing the light of day.
There are various avenues available to disseminate negative or inconclusive research findings, among them being journals whose aim and scope relate to publishing negative / inconclusive results such as: 'Journal of Articles in Support of the Null Hypothesis', 'Journal of Negative Results in Biomedicine', and in the Spanish language 'Journal of Negative and No Positive Results'.
The pressure which researchers feel under to have articles accepted in the most prestigious journals in their fields, in order to secure career advancement, is often known as a culture of 'Publish or Perish'.
This culture can work against open research practices, incentivising 'closed' and even outright fraudulent academic practices as researchers may feel under pressure to manipulate underpinning research data so that they show novel and 'groundbreaking' results.
The 'Publish or Perish' culture is increasingly being challenged by research funders and research organisations, with 'responsible metrics' initiatives like the Declaration on Research Assessment (DORA) and the Leiden Manifesto being signed up to by an ever increasing number of research bodies.
For a perspective on research culture and the transformational role which Open Research Practices can play, please watch the TED Talk 'Research Culture is Broken; Open Science can Fix It' delivered by Dr Rachael Ainsworth in 2019.
Open Research can apply at different stages of the research and dissemination process. Below are detailed some of the steps which researchers can take to cultivate Open Research practices.
By publicly pre-registering a study, researchers outline what the intention of their study is and what methods they will be using. Pre-registration can provide benefits whether the study is a 'hypothesis-generating' (exploratory) study or a 'hypothesis-testing' (confirmatory) one, as laid down in this guidance provided by the Center for Open Science.
Pre-registration is already established practice in the case of clinical trials.
Pre-registering is a principle means for countering practices such as P-hacking and publication bias, practices explained in the neighbouring tab in this LibGuide.
Publicly sharing research protocols can support others to reproduce research and allow others to build upon your work.
A specialist platform like Protocols.io can be used to document research protocols. The protocol can be treated as a research output with a permanent DOI identifier to make it easy to cite and link to.
A Registered Report is a type of article which documents the background, study design, methods and planned analysis of a study before any data collection has been carried out. A registered report will be subject to peer review to assess the quality of its research design.
By publishing a Registered Report, the journal and author(s) will typically commit to publishing the research outcomes of the study, even if the outcome are negative or inconclusive results. This commitment to publishing the outcomes of a study can help to address the challenge of 'Publication Bias', whereby scientific research can languish unpublished due to a failure to evidence 'positive' results.
An example of this practice is the journal Cortex which will commit to publishing the results stemming from any accepted preregistered study.
Making software available as Open Source supports the Reproducibility of research, and potentially enables others to build on your research.
The platform GitHub is a popular platform for researchers and programmers to share open source code.
There has been growing expectation by funders, publishers, research institutions and educational organisations of the long-term preservation and open sharing of data which provide the validation of research outputs. The often repeated mantra is for researchers to strive to be: "As Open as Possible, as Closed as Necessary". This increased visibility of research, which potentially promotes researcher and institution profiles, leads to greater efficiencies in future investigations, as research does not need repetition. Through the adoption of policies and utilisation of open data repositories, the encouragement of sharing reproducible work leads to innovation in research, and collaborations.
Funders, including Wellcome Trust, UK Research and Innovation (UKRI), encourage researchers to share outputs’ related data in line with the FAIR Principles (Findable, Accessible, Interoperable, Reusable). Further information can be found on in the FAIR Principles tab in the RDM LibGuides.
Preprints are pre-peer reviewed manuscripts which typically are about to be submitted for journal publication. There are a wide range of discipline specific preprint servers where authors can make their work available to other scholars in their field.
The practice of sharing preprints is well established in disciplines such as Physics and Computer Science where the platform arXiv is widely used to share preprint material. Some research funders, such as the Wellcome Trust, are encouraging their funding recipients to make use of preprint sharing to make the research dissemination process more efficient. The Covid-19 pandemic has highlighted the importance of the rapid sharing of scientific studies with the research community. Some funders have established their own Preprint servers to quickly disseminate research prior to formal journal peer-review. Examples in this area include The Wellcome Trust and The Bill and Melinda Gates Foundation.
There is also increasingly integration between the worlds of preprint servers and journal publishing. The open access publisher eLife, for example, requires that papers which it considers for formal publication be first made available as preprints via platforms such as bioRxiv and medRxiv. This policy was inaugurated in July 2021 as part of eLife's 'publish, then review' model.
There are two primary routes for authors to provide Open Access to their research outputs: the Green Open Access route and the Gold Open Access route.
Green Open Access involves the final version being published with a subscription access journal or publisher, and the 'accepted manuscript' version (the version which reflects changes as a result of the academic peer review process but which comes before publisher formatting and typesetting) being archived on an institutional or subject repository. Open access can then be provided at the point of publication or following an embargo period.
Gold Open Access involves the final published version being published open access from the point of publication. An Article Processing Charge (APC) or Book Processing Charge (BPC) may be required to facilitate publication via this route. Coventry University authors may be able to take advantage of one of the 'Read & Publish' publisher agreements which we are party to in order to publish via the Gold Open Access route without additional cost.
Not all open access publishers charge authors to publish; open access which doesn't involve a charge to the author or the reader is often termed 'Diamond Open Access'.
The Directory of Open Access Journals (DOAJ) is the most comprehensive source for open access journal titles and provides details on which journals charge Article Processing Charges (APCs) and which operate on the cost-free Diamond Open Access model.
Many researchers will already be familiar with the various Open Access requirements which Coventry University staff and students are subject to. Information about the University Open Access Standard, the current REF Open Access policy and various funder Open Access policies, are available from the Open Access section of this LibGuide.
Open Peer Review is the practice whereby the identities of the authors are known to the reviewers, and the reviewers to the authors. Under Open Peer Review the different article versions along with reviewer comments are typically shared along with the final publication so that any changes made as a result of peer review can be documented.
Advocates of Open Peer Review feel that it can counteract some of the imbalance of power abuses which can result from more 'closed' forms of peer review where reviewers identities are not disclosed, and reviewer comments are not published.
Guidance on the practical implementation of Open Peer Review and details of those publishers who currently offer this option are contained in the article: Ross-Hellauer, T. and Görögh, E., Guidelines for open peer review implementation, Research integrity and peer review, 4.1 (2019)
Please note that many research funders already have policy stipulations around Open Access to research publications, research data and the responsible use of Research Metrics.
Many funders have introduced policies promoting Open Research practices. Below are a sample of such funders.
UKRI has specific policies in relation to several components of Open Research Practice:
The Medical Research Council (MRC) has a specific policy in place around the registration of studies with the ISRCTN registry prior to the study commencing. The MRC also expects results of studies to be reported within 24 months of a trial ending. Further information on the MRC's Policy on Open Research Data: clinical trials and public health interventions, is available from the UKRI website.
To help promote Open Research at the stage prior to publication, UKRI is also providing funding to the Octopus platform which is due to be launched in Spring 2022. As outlined in their August 2021 Press Release, UKRI's hope is that this platform can help provide an up to date 'primary research record' which can in turn shift Research Culture toward greater openness and transparency.
The Research England Development (RED) Fund is also helping to financially support 18 Universities who are members of the UK Reproducibility Network to drive the uptake of Open Research Practices.
The Wellcome Trust has specific policies in relation to several components of Open Research Practice:
Wellcome Trust have additionally launched the Wellcome Open Research platform as a new means for their funded researchers to rapidly disseminate research results.
Horizon Europe, the programme which since the start of 2021 has succeeded Horizon 2020, speaks of going beyond an 'Open Access' policy in asserting an 'Open Science' policy (section 16, Programme Guide).
Horizon Europe has established a series of 'mandatory Open Science practices':
Beyond these mandatory requirements are additional recommended practices. These are laid down on pages 38-53 of the Horizon Europe Programme Guide.
Horizon 2020 had already implemented specific policies in relation to several areas of Open Research practice:
Horizon 2020 and Horizon Europe funded authors can make use of Open Research Europe. An Open Access, rapid publication system which operates on a publish & review basis, and supports open peer review practices.
The NIHR has the following Open Research policies in place:
The NIHR also has their own Open Access, rapid publication platform NIHR Open Research. This operates on a publish & review basis, and supports open peer review practices.
The Gates Foundation has an Open Access policy which applies to research outputs (with the exceptions of monographs and book chapters) and research data. Please view the Gates Foundation website for more information.
In common with several other research funders, the Gates Foundation maintains its own Open Research platform, Gates Open Research, for the rapid dissemination of pre-published research on a publish & review basis, with support for open peer review practices.
The recommendation incorporates positions on: open access to publications, research data, open educational resources, open source software and open hardware.
The Open Science Framework (OSF) is overseen by the Center for Open Science, a non-profit organisation which has become a leading body in the Open Research movement.
The platform offers a number of functions:
The work of the Open Science Framework first came to widespread attention in 2015 following the publication of a project exploring reproducibility in Psychological Science research.
The OSF also provides a searchable indexing service for Preprint servers across a diverse range of academic disciplines.
Due to launch in Spring 2022, the development of the Octopus platform has been financially supported by Research England with the platform intending to help bring about a change in research culture.
Octopus breaks scientific research into eight elements which can be sequentially linked together within the platform. By breaking research up into component parts the intention is that it will be easier for researchers to publicly highlight research which is in progress and to elicit collaboration with other researchers.
The platform will be free for researchers to publish on, free for the public to access, and promises to embed principles of transparency and openness in the way it operates.
An initiative for scientific researchers to share their research in more or less 'real time' by providing online access to their online notebooks. Research groups based in Canada, the UK and USA are all currently active on the platform.
A new platform which provides a similar function to the Octopus platform. The main differences are that Research Equals allows researchers to link any steps in the research process together (Octopus follows a sequential order) and allows for a wider variety of 'steps' with a focus on research provenance.
Research Equals operates on a 'pay to close' model, meaning that if researchers wish to restrict access to any components a charge is payable.
The range of research funders who have set up their own Open Research platforms to disseminate research which they have funded has grown significantly in recent years. Below are some examples of such platforms; they will typically make available pre-peer reviewed material which then goes through an open peer review process on the Research Platform.
The FOSTER Open Science Project, funded by the EU to support their Horizon 2020 initiative, comprises a wealth of training resources aimed at a range of different stakeholders in the research ecosystem.
Free modular online courses are available on a range of different Open Science / Open Research topics.
The UK Reproducibility Network (UKRN) is a peer-led consortium which promotes research reproducibility in the UK and which aims to make sure that the UK remains a centre for world-leading research.
On the Network's YouTube channel are archived recordings of past seminars on a range of topics relevant to Open Research. The Network also advertise their upcoming events through their website.
An outcome of the FOSTER Open Science Project, the Open Science Research Handbook was devised in 2018 as an 'open, living handbook'. The work provides chapters introducing different aspects of Open Research practice. As it is designed to be a living and evolving document, readers can suggest changes and improvements to the Handbook.
The Turing Way is a community driven project to develop open-source guidance in relation to producing reproducible scientific data. The project relies on community contributions, similar to the collaborative principles behind Wikipedia. The Project was initiated by the Alan Turing Institute, which is supported by a range of leading UK Research Universities and the EPSRC.
A handbook produced through the Project is available through the Zenodo repository.
The Delft University of Technology (TU Delft) in the Netherlands has published this online MOOC which is currently available to access in an archived form. The course is primarily aimed at researchers wishing to know more about the core tenets of Open Research practice and features four modules which cover such topics as Research Data Management and compliance with the FAIR principles, Open Access to research outputs and Research Visibility.
Participation on the course is free, though there are also paid options.
Citizen Science, the practice of involving public participation in the scientific research process, has been growing in scope and popularity since the term was first added to the Oxford English Dictionary in 2014.
The EU Citizen Science platform, supported by the EU's Horizon 2020 programme, contains a range of resources to training materials related to Citizen Science and aims to promote best practice in this area. There are various online MOOC courses available, covering topics ranging from research 'storytelling', to research ethics. The platform also aims to raise awareness of Citizen Science and acts as a central hub for promoting events taking place throughout Europe.
LIBER has produced a visualisation of what they see as core Open Science Skills, incorporating the five components: Scholarly Publishing, FAIR Data, Research Integrity, Citizen Science and Metrics and Rewards.
ReproducibiliTea is a community building initiative designed in particular to bring together Post-Graduate Researchers or Early Career Researchers in local University networks. The initiative started at the University of Oxford, and is supported by the UK Reproducibility Network.
Originating from King's College, London, the RIOT Science Club runs events aimed at increasing awareness of Open Research practices. There is a YouTube channel dedicated to making available recordings of their past events.
The aim of the group is to 'create a levelling space where junior and senior academics alike can all mutually benefit from the exchange of ideas that move us away from the publish or perish culture, that stymie the replication crises, and that put researchers on the map in the Open Research revolution!'