Skip to main content

Responsible use of metrics (the DORA declaration)

The DORA declaration - to which the College is a signatory - emerged from the Annual Meeting of the American Society for Cell Biology (ASCB) in San Francisco on December 16, 2012.

The DORA declaration articulates the general principle: 'Do not use journal-based metrics, such as Journal Impact Factors, as a surrogate measure of the quality of individual research articles, to assess an individual scientist’s contributions, or in hiring, promotion, or funding decisions.'

The DORA declaration instructs institutions to instead be explicit about the criteria used to reach hiring, tenure, and promotion decisions, clearly highlighting, especially for early-stage investigators, that the scientific content of a paper is much more important than publication metrics or the identity of the journal in which it was published.

For the purposes of research assessment, institutions should consider the value and impact of all research outputs (including datasets and software) in addition to research publications, and consider a broad range of impact measures including qualitative indicators of research impact, such as influence on policy and practice.

HOW BIRKBECK EMBODIES DORA'S PRINCIPLES

These principles are articulated in the College's guidance as follows:

Our REF 2021 Code of practice articulates how we select research outputs for this external evaluation.

The guidance used in our research hiring, tenure, and promotion decisions can be accessed in our Guidance notes for line managers and our Progression and reward procedures.

The DORA declaration gives the following advice to individual researchers, which has been incorporated into this guidance.

  • When involved in committees making decisions about funding, hiring, tenure, or promotion, make assessments based on scientific content rather than publication metrics.
  • Wherever appropriate, cite primary literature in which observations are first reported rather than reviews in order to give credit where credit is due.
  • Use a range of article metrics and indicators on personal/supporting statements, as evidence of the impact of individual published articles and other research outputs.
  • Challenge research assessment practices that rely inappropriately on Journal Impact Factors and promote and teach best practice that focuses on the value and influence of specific research outputs.

THE LEIDEN MANIFESTO

The Leiden Manifesto was published as a comment piece in Nature in 2015 (Hicks, Wouters, Waltman, de Rijcke, Rafols, Nature, April 23, 2015) and has become a standard reference point for the responsible use of metrics in assessment. It describes ten principles to guide research evaluation.

The College's approach to principles 1 (quantitative evaluation should support qualitative, expert assessment), 6 (account for variation by field in publication and citation practices), 7 (base assessment of individual researchers on a qualitative judgement of their portfolio), 8 (avoid misplaced concreteness and false precision), 9 (recognise the systemic effects of assessment and indicators), and 10 (scrutinise indicators regularly and update them) is also described in our guidance used in our reach hiring, tenure, and promotion decisions as described above.

The College approaches the remaining four principles as follows:

  • Principle 2: Measure performance against the research missions of the institution, group or researcher and Principle 3: Protect excellence in locally relevant research.
  • Principle 4: Keep data collection and analytical processes open, transparent and simple and Principle 5: Allow those evaluated to verify data and analysis.