Research & Scholarly Publications Team: Introduction to Open Research Practices

Open Research Practices can help to address several challenges which have arisen within academic publishing in recent years, as well as assisting with the production of rigorous and robust research.

Below is a list of some of the present 'problem areas' within the research publishing ecosystem which Open Research Practices seek to address.

Replication Crisis

The ability to reproduce research studies to ascertain whether the same results are produced, is an essential part of the scientific verification process. However, it has emerged over the past 15-20 years that many landmark studies in the Social and Biomedical Sciences cannot be reproduced to confirm the accuracy of their findings. A paper published in PLOS Medicine (Ioannidis, 2005) helps to highlight why this poses a significant problem.

There are a variety of reasons as to how the replication crisis came about, and the various factors listed below all have a role to play. The historically low rates of data sharing have also significantly contributed to a lack of research reproducibility and transparency. 

There is still an ongoing concern that academic journals are being asked to fulfill two contrasting functions which they are not necessarily best suited for. Those functions being both to disseminate useful research findings to practitioners and researchers, and to act as the primary research record in terms of documenting the methodological steps in the process. Concerns that journals are not the best outlet for the second of these functions has prompted the funding of a new platform, Science Octopus, which has received financial support from UKRI and which is intended to act as 'the primary research record'. The Octopus platform is due to go 'live' for real submissions in Spring 2022. 

HARKing - Hypothesising after the results are known

HARKing involves retrospectively adapting a study to fit with the results generated. As defined in a 1998 paper on the topic this involves: 'presenting a post hoc hypothesis in one's research report as if it were, in fact, an a priori hypothesis' (Kerr, 1998).   

There are several negative outcomes for the integrity and dissemination of science which can result from HARKing. Among these are:

• Failure to communicate aspects of a study which didn't work as expected (see also Publication Bias).
• The 'fudging' of unclear or ambiguous results to create a neater narrative.
• The misrepresentation of the initial purpose of the research, which in turn provides an inaccurate representation of how science is conducted.

The overarching cost of HARKing, as identified by Norbert Kerr in their 1998 article on the topic, is that at its heart, the practice represents an implicit 'violating [of] basic ethical principles'. 

The temptation to engage in 'HARKing' can be effectively dealt with through the pre-registration of research studies. This provides researchers with clear parameters as to how research is being carried out and the purposes behind it.  

P-hacking / Data Dredging / Significance Chasing

P-hacking is the manipulation of a data analysis in order to make the outcome appear more statistically significant. Also known as 'selective reporting'.

Practices which can result in p-hacking include:

• Conducting analysis midway through experiments to decide whether to continue collecting data
• Recording a number of response variables and then selectively choosing which to analyse
• Decisions around including or ignoring outlying data
• Stopping data collection at the point an analysis indicates a positive p-value

Data Dredging involves testing multiple hypotheses using a single data set, at the risk of producing spurious correlations.

A 2015 article published in PLOS Biology (Head, 2015) found that p-hacking 'is widespread throughout science', although the paper also concluded that its effect upon scientific consensus drawn from meta-analysis was limited.

The pre-registration of study protocols is an effective way of preventing the practices detailed above.

Publication Bias / Failure to Report negative or inconclusive results

'Publication bias' is the term used to account for the fact that the great majority of scientific papers provide 'positive results'. Between 1990/1 and 2007 the number of scientific papers providing positive results increased from just over 70% to just under 86% (Mlinaric, 2017).

There are various harmful consequences for negative / inconclusive results not being disseminated. These include:

• The possibility that the time of other researchers may be wasted by trying to replicate the same experiment / study which has already been performed
• The suppression of adverse outcomes can be particularly damaging in fields like pharmaceutical research. It has been estimated that 60% of clinical trials which found inadequate drug efficacy or safety concerns do not get reported.
• Human subjects who have participated in research have done so to advance scientific understanding. There is therefore a moral obligation to disseminate any findings.

Both journal editors and researchers have potentially contributed to the growth of 'publication bias'; editors, due to the fact that some journals prioritise 'novel' research results, and authors, due to the fact they wish to publicise their research successes rather than disappointments. Researchers' reluctance to share negative and inconclusive results is sometimes known as the 'file drawer effect', due to the results languishing in drawers and not seeing the light of day. 

There are various avenues available to disseminate negative or inconclusive research findings, among them being journals whose aim and scope relate to publishing negative / inconclusive results such as: 'Journal of Articles in Support of the Null Hypothesis', 'Journal of Negative Results in Biomedicine', and in the Spanish language 'Journal of Negative and No Positive Results'. 

'Publish or Perish' culture

The pressure which researchers feel under to have articles accepted in the most prestigious journals in their fields, in order to secure career advancement, is often known as a culture of 'Publish or Perish'. 

This culture can work against open research practices, incentivising 'closed' and even outright fraudulent academic practices as researchers may feel under pressure to manipulate underpinning research data so that they show novel and 'groundbreaking' results. 

The 'Publish or Perish' culture is increasingly being challenged by research funders and research organisations, with 'responsible metrics' initiatives like the Declaration on Research Assessment (DORA) and the Leiden Manifesto being signed up to by an ever increasing number of research bodies.

For a perspective on research culture and the transformational role which Open Research Practices can play, please watch the TED Talk 'Research Culture is Broken; Open Science can Fix It' delivered by Dr Rachael Ainsworth in 2019. 

Open Research can apply at different stages of the research and dissemination process. Below are detailed some of the steps which researchers can take to cultivate Open Research practices.

Research Planning phase

Pre-registration of study / Sharing Research Protocols

By publicly pre-registering a study, researchers outline what the intention of their study is and what methods they will be using. Pre-registration can provide benefits whether the study is a 'hypothesis-generating' (exploratory) study or a 'hypothesis-testing' (confirmatory) one, as laid down in this guidance provided by the Center for Open Science.

Pre-registration is already established practice in the case of clinical trials. 

Pre-registering is a principle means for countering practices such as P-hacking and publication bias, practices explained in the neighbouring tab in this LibGuide. 

Publicly sharing research protocols can support others to reproduce research and allow others to build upon your work.

A specialist platform like Protocols.io can be used to document research protocols. The protocol can be treated as a research output with a permanent DOI identifier to make it easy to cite and link to.

Publishing a Registered Report

A Registered Report is a type of article which documents the background, study design, methods and planned analysis of a study before any data collection has been carried out. A registered report will be subject to peer review to assess the quality of its research design.

By publishing a Registered Report, the journal and author(s) will typically commit to publishing the research outcomes of the study, even if the outcome are negative or inconclusive results. This commitment to publishing the outcomes of a study can help to address the challenge of 'Publication Bias', whereby scientific research can languish unpublished due to a failure to evidence 'positive' results.

An example of this practice is the journal Cortex which will commit to publishing the results stemming from any accepted preregistered study.

Making software available as Open Source

Making software available as Open Source supports the Reproducibility of research, and potentially enables others to build on your research.

The platform GitHub is a popular platform for researchers and programmers to share open source code. 

Data Collection and Analysis phase

Open Access to research data, analysis and materials

There has been growing expectation by funders, publishers, research institutions and educational organisations of the long-term preservation and open sharing of data which provide the validation of research outputs.  The often repeated mantra is for researchers to strive to be: "As Open as Possible, as Closed as Necessary".  This increased visibility of research, which potentially promotes researcher and institution profiles, leads to greater efficiencies in future investigations, as research does not need repetition. Through the adoption of policies and utilisation of open data repositories, the encouragement of sharing reproducible work leads to innovation in research, and collaborations. 

Funders, including Wellcome Trust, UK Research and Innovation (UKRI), encourage researchers to share outputs’ related data in line with the FAIR Principles (Findable, Accessible, Interoperable, Reusable).  Further information can be found on in the FAIR Principles tab in the RDM LibGuides.

Writing Up phase

Sharing of Preprints

Preprints are pre-peer reviewed manuscripts which typically are about to be submitted for journal publication. There are a wide range of discipline specific preprint servers where authors can make their work available to other scholars in their field.

The practice of sharing preprints is well established in disciplines such as Physics and Computer Science where the platform arXiv is widely used to share preprint material. Some research funders, such as the Wellcome Trust, are encouraging their funding recipients to make use of preprint sharing to make the research dissemination process more efficient. The Covid-19 pandemic has highlighted the importance of the rapid sharing of scientific studies with the research community. Some funders have established their own Preprint servers to quickly disseminate research prior to formal journal peer-review. Examples in this area include The Wellcome Trust and The Bill and Melinda Gates Foundation. 

There is also increasingly integration between the worlds of preprint servers and journal publishing. The open access publisher eLife, for example, requires that papers which it considers for formal publication be first made available as preprints via platforms such as bioRxiv and medRxiv. This policy was inaugurated in July 2021 as part of eLife's 'publish, then review' model.

Submission to Publication phase

Open Access to Research Outputs

There are two primary routes for authors to provide Open Access to their research outputs: the Green Open Access route and the Gold Open Access route.

Green Open Access involves the final version being published with a subscription access journal or publisher, and the 'accepted manuscript' version (the version which reflects changes as a result of the academic peer review process but which comes before publisher formatting and typesetting) being archived on an institutional or subject repository. Open access can then be provided at the point of publication or following an embargo period. 

Gold Open Access involves the final published version being published open access from the point of publication. An Article Processing Charge (APC) or Book Processing Charge (BPC) may be required to facilitate publication via this route. Coventry University authors may be able to take advantage of one of the 'Read & Publish' publisher agreements which we are party to in order to publish via the Gold Open Access route without additional cost. 

Not all open access publishers charge authors to publish; open access which doesn't involve a charge to the author or the reader is often termed 'Diamond Open Access'.

The Directory of Open Access Journals (DOAJ) is the most comprehensive source for open access journal titles and provides details on which journals charge Article Processing Charges (APCs) and which operate on the cost-free Diamond Open Access model.

Many researchers will already be familiar with the various Open Access requirements which Coventry University staff and students are subject to. Information about the University Open Access Standard, the current REF Open Access policy and various funder Open Access policies, are available from the Open Access section of this LibGuide.

Open Peer Review 

Open Peer Review is the practice whereby the identities of the authors are known to the reviewers, and the reviewers to the authors. Under Open Peer Review the different article versions along with reviewer comments are typically shared along with the final publication so that any changes made as a result of peer review can be documented.

Advocates of Open Peer Review feel that it can counteract some of the imbalance of power abuses which can result from more 'closed' forms of peer review where reviewers identities are not disclosed, and reviewer comments are not published.

Guidance on the practical implementation of Open Peer Review and details of those publishers who currently offer this option are contained in the article: Ross-Hellauer, T. and Görögh, E., Guidelines for open peer review implementation, Research integrity and peer review, 4.1 (2019)

Please note that many research funders already have policy stipulations around Open Access to research publications, research data and the responsible use of Research Metrics.

Many funders have introduced policies promoting Open Research practices. Below are a sample of such funders.

For more on specific funder open access and research data policies, please consult the relevant sections of this LibGuide.

UK Research & Innovation (UKRI)

UKRI has specific policies in relation to several components of Open Research Practice:

• Open Access to publications - the UKRI have had a policy in place in this area for several years. For more information on the UKRI Open Access requirements please see this part of our LibGuide.
• Open Data principles - for more information on funder requirements in this area and the support we can offer, please see this part of our LibGuide. 
• Responsible research assessment - UKRI is a signatory of the Declaration on Research Assessment (DORA), meaning the funder will not use quantitative metrics like journal citation scores and impact factors in isolation to evaluate research quality.

The Medical Research Council (MRC) has a specific policy in place around the registration of studies with the ISRCTN registry prior to the study commencing. The MRC also expects results of studies to be reported within 24 months of a trial ending. Further information on the MRC's Policy on Open Research Data: clinical trials and public health interventions, is available from the UKRI website.

To help promote Open Research at the stage prior to publication, UKRI is also providing funding to the Octopus platform which is due to be launched in Spring 2022. As outlined in their August 2021 Press Release, UKRI's hope is that this platform can help provide an up to date 'primary research record' which can in turn shift Research Culture toward greater openness and transparency. 

The Research England Development (RED) Fund is also helping to financially support 18 Universities who are members of the UK Reproducibility Network to drive the uptake of Open Research Practices.

Wellcome Trust

The Wellcome Trust has specific policies in relation to several components of Open Research Practice: 

• Open Access to publications - the Wellcome Trust has had open access policies in place for several years. Since 1st January 2021 Wellcome have had an open access policy which requires immediate open access to be provided to funded research outputs. For more information please see this page on our LibGuide.
• Policy toward Data, software and material sharing - grant holders are required to 'maximise the availability of research data, software and materials with as few restrictions as possible'. Please see the Research Data Management section of our LibGuide for more general guidance in this area.
• Promotion of Preprints - Wellcome permit the citing of preprints in grant applications and ask that grant recipients provide access to the latest research in preprint form in tackling health emergencies. 
• Responsible research assessment - Wellcome require that organisations receiving their funding publicly commit to a) assess research outputs based on their intrinsic merit, and b) discourage the inappropriate use of proxies or metrics.

Wellcome Trust have additionally launched the Wellcome Open Research platform as a new means for their funded researchers to rapidly disseminate research results.

Horizon 2020 / Horizon Europe

Horizon Europe, the programme which since the start of 2021 has succeeded Horizon 2020, speaks of going beyond an 'Open Access' policy in asserting an 'Open Science' policy (section 16, Programme Guide). 

Horizon Europe has established a series of 'mandatory Open Science practices':

• Management of research data in line with FAIR principles, adopting the principle 'as open as possible, as closed as necessary'.
• Information provided about any outputs, tools or instruments needed to validate any publication conclusions or to validate and re-use any data is provided.
• Digital or physical access to the results needed to validate the conclusions of scientific publications are provided, unless an exception applies.
• In cases of public emergency, provision for open access to all research outputs under open licences to legal entities that need access to research to address the public emergency.

Beyond these mandatory requirements are additional recommended practices. These are laid down on pages 38-53 of the Horizon Europe Programme Guide.

Horizon 2020 had already implemented specific policies in relation to several areas of Open Research practice:

• Open access to publications policy - Please see this page of our LibGuide for information about the Horizon 2020 Open Access policy.
• Research Data policy - Horizon 2020 requires that research data adhere to Findable Accessible Interoperable Reusable (FAIR) data principles. More information is available from the Research Data Management section of our LibGuide and through the Horizon 2020 website.
• Promotion of Preprints - As a signatory to the Statement on Data Sharing in Public Health Emergency, along with the Wellcome Trust, the European Commission promotes that research findings are made available via preprint servers prior to publication and peer review.

Horizon 2020 and Horizon Europe funded authors can make use of Open Research Europe. An Open Access, rapid publication system which operates on a publish & review basis, and supports open peer review practices.  

National Institute for Health Research (NIHR)

The NIHR has the following Open Research policies in place:

• Open Access policy in relation to research and review articles. Please see our LibGuide guidance for more on these requirements.
• Clinical trial registration and disclosure of results policy
• The NIHR position on the sharing of research data was published in 2019, with an updated version provided in 2021.

The NIHR also has their own Open Access, rapid publication platform NIHR Open Research. This operates on a publish & review basis, and supports open peer review practices.

Other Funders

Bill and Melinda Gates Foundation

The Gates Foundation has an Open Access policy which applies to research outputs (with the exceptions of monographs and book chapters) and research data. Please view the Gates Foundation website for more information.

In common with several other research funders, the Gates Foundation maintains its own Open Research platform, Gates Open Research, for the rapid dissemination of pre-published research on a publish & review basis, with support for open peer review practices.

UNESCO

A Recommendation on Open Science was adopted by the General Conference of UNESCO at its 41st session, in November 2021.

The recommendation incorporates positions on: open access to publications, research data, open educational resources, open source software and open hardware. 

Platforms supporting Open Research Practices

Open Science Framework

The Open Science Framework (OSF) is overseen by the Center for Open Science, a non-profit organisation which has become a leading body in the Open Research movement. 

The platform offers a number of functions:

• a dashboard-based project management functionality, with access controls, version control features, and project analytics;
• a central document store for collaborative sharing;
• integrations with a number of cloud storage services. Additionally the platform integrates with GitHub for code management, with figshare and Dataverse for data repositories, and with Mendeley for reference management;
• a pre-registration function for publishing time-stamped study designs;
• a preprint server for rapid communication of results.

The work of the Open Science Framework first came to widespread attention in 2015 following the publication of a project exploring reproducibility in Psychological Science research. 

The OSF also provides a searchable indexing service for Preprint servers across a diverse range of academic disciplines.

Science Octopus

Due to launch in Spring 2022, the development of the Octopus platform has been financially supported by Research England with the platform intending to help bring about a change in research culture. 

Octopus breaks scientific research into eight elements which can be sequentially linked together within the platform. By breaking research up into component parts the intention is that it will be easier for researchers to publicly highlight research which is in progress and to elicit collaboration with other researchers. 

The platform will be free for researchers to publish on, free for the public to access, and promises to embed principles of transparency and openness in the way it operates.

Open Lab Notebooks

An initiative for scientific researchers to share their research in more or less 'real time' by providing online access to their online notebooks. Research groups based in Canada, the UK and USA are all currently active on the platform.

Research Equals

A new platform which provides a similar function to the Octopus platform. The main differences are that Research Equals allows researchers to link any steps in the research process together (Octopus follows a sequential order) and allows for a wider variety of 'steps' with a focus on research provenance.

Research Equals operates on a 'pay to close' model, meaning that if researchers wish to restrict access to any components a charge is payable.

Funder Specific Open Research Platforms

The range of research funders who have set up their own Open Research platforms to disseminate research which they have funded has grown significantly in recent years. Below are some examples of such platforms; they will typically make available pre-peer reviewed material which then goes through an open peer review process on the Research Platform.

• NIHR Open Research
• Wellcome Open Research
• Gates Open Research

General Training resources

FOSTER Open Science Project

The FOSTER Open Science Project, funded by the EU to support their Horizon 2020 initiative, comprises a wealth of training resources aimed at a range of different stakeholders in the research ecosystem. 

Free modular online courses are available on a range of different Open Science / Open Research topics.  

UK Reproducibility Network

The UK Reproducibility Network (UKRN) is a peer-led consortium which promotes research reproducibility in the UK and which aims to make sure that the UK remains a centre for world-leading research.

On the Network's YouTube channel are archived recordings of past seminars on a range of topics relevant to Open Research. The Network also advertise their upcoming events through their website.

Open Science Research Handbook

An outcome of the FOSTER Open Science Project, the Open Science Research Handbook was devised in 2018 as an 'open, living handbook'. The work provides chapters introducing different aspects of Open Research practice. As it is designed to be a living and evolving document, readers can suggest changes and improvements to the Handbook.

The Turing Way

The Turing Way is a community driven project to develop open-source guidance in relation to producing reproducible scientific data. The project relies on community contributions, similar to the collaborative principles behind Wikipedia. The Project was initiated by the Alan Turing Institute, which is supported by a range of leading UK Research Universities and the EPSRC.

A handbook produced through the Project is available through the Zenodo repository. 

Open Science Course - TU Delft

The Delft University of Technology (TU Delft) in the Netherlands has published this online MOOC which is currently available to access in an archived form. The course is primarily aimed at researchers wishing to know more about the core tenets of Open Research practice and features four modules which cover such topics as Research Data Management and compliance with the FAIR principles, Open Access to research outputs and Research Visibility.

Further information about Open Access and Research Data Management practices within the Coventry University context, is available within this LibGuide.

Participation on the course is free, though there are also paid options.

EU-Citizen.Science Initiative

Citizen Science, the practice of involving public participation in the scientific research process, has been growing in scope and popularity since the term was first added to the Oxford English Dictionary in 2014.

The EU Citizen Science platform, supported by the EU's Horizon 2020 programme, contains a range of resources to training materials related to Citizen Science and aims to promote best practice in this area. There are various online MOOC courses available, covering topics ranging from research 'storytelling', to research ethics. The platform also aims to raise awareness of Citizen Science and acts as a central hub for promoting events taking place throughout Europe.

LIBER (Association of European Research Libraries)

LIBER has produced a visualisation of what they see as core Open Science Skills, incorporating the five components: Scholarly Publishing, FAIR Data, Research Integrity, Citizen Science and Metrics and Rewards.  

The diagram, licensed under a CC BY licence, is available from Zenodo.

ReproducibiliTea 

ReproducibiliTea is a community building initiative designed in particular to bring together Post-Graduate Researchers or Early Career Researchers in local University networks. The initiative started at the University of Oxford, and is supported by the UK Reproducibility Network.

RIOT Science Club

Originating from King's College, London, the RIOT Science Club runs events aimed at increasing awareness of Open Research practices. There is a YouTube channel dedicated to making available recordings of their past events.  

The aim of the group is to 'create a levelling space where junior and senior academics alike can all mutually benefit from the exchange of ideas that move us away from the publish or perish culture, that stymie the replication crises, and that put researchers on the map in the Open Research revolution!'