Responsible Metrics - RSP - Publishing Support - LibGuides at Coventry University

The term research metrics or research indicators describes the use of quantitative measurements to evaluate research and can be applied to individuals, groups, outputs and journals.

Coventry University has committed to the responsible use of these metrics for evaluation. In 2019, the university implemented a Responsible Metrics Standard aligned with the Leiden Manifesto. The implementation of this standard demonstrated the University’s intention to engage with the responsible metrics agenda and commitment to ensure the responsible use of metrics across the Group. Following this, in 2022 the university signed the Declaration on Research Assessment (DORA) to further this commitment. More information on DORA can be found below.

Assessment of the quality of research is a normal part of the research activity cycle. There are several benefits for the individual, a research group and the institution, in using metrics to support qualitative methods of assessing research activities.

For an individual, metrics can provide a useful insight into how their research may have been disseminated and used. They can provide a mechanism to benchmark against peers and support future applications for funding and progression. They can also help to identify key researchers and outputs, places to publish and potential collaborators within a particular field.

For a research group, metrics can be used to measure performance against similar groups in other institutions. They can also help evaluate and develop strategies for research and identify potential recruitment and collaboration opportunities.

For an institution, metrics can be used to measure performance against corporate plans and targets. They can also be used to inform research strategies, improve prestige, support staff and student recruitment, and identify potential funding opportunities.

However, metrics can be complex and the misuse of them can be damaging to both individuals and organisations. It is essential that the way in which metrics are used is clear, fair and transparent. Signing up to DORA is part of Coventry University’s commitment to this.

Signatory of Dora Sticker

DORA recognises the need to improve the ways in which the outputs of scholarly research are evaluated. DORA’s vision is to advance practical and robust approaches to research assessment globally and across all scholarly disciplines. It is a worldwide initiative covering all scholarly disciplines and all key stakeholders including funders, publishers, professional societies, institutions, and researchers. For more information and to view current signatories see https://sfdora.org/

The overriding principle of DORA is to break the assumed link between certain metrics and research quality. It argues for qualitative judgments to be exercised, with research judged on its own merits. The key recommendations from DORA include:

• Not using journal level metrics as a surrogate measure of individual article quality

• Being clear about the criteria used in research evaluation

• Considering the value of all output types

• Using a range of metrics and incorporating qualitative indicators.

• Challenging practices which rely on inappropriate metrics

In addition to DORA, following REF2014 a review of metrics in research assessment was carried out and the results published in an independent report, The Metric Tide. This provided the following key principles for research metrics.

• Robustness – basing metrics on the best possible data in terms of accuracy and scope

• Humility – recognising that quantitative evaluation should support, but not supplant, qualitative, expert assessment

• Transparency – ensuring that those being evaluated can test and verify the results

• Diversity – accounting for variation by research field, and using a range of indicators to reflect and support a plurality of research and researcher career paths across the system

• Reflexivity – recognising and anticipating the systemic and potential effects of indicators, and updating them in response

Coventry University is committed to the principles of responsible research assessment outlined above. The following pages gave been developed with these in mind and are based on the SCOPE framework process developed by the INORMS Research Evaluation Working Group for evaluating responsibly. They are intended to support you with interpreting and using metrics at an individual or group level and to appreciate the limitations of different approaches.

Follow the links below to more information on metrics: