Skip to Main Content
It looks like you're using Internet Explorer 11 or older. This website works best with modern browsers such as the latest versions of Chrome, Firefox, Safari, and Edge. If you continue with this browser, you may see unexpected results.

Library Research Support: Rankings and REF

Support for Research Staff & Research Students

Citation metrics in the REF and University Rankings

The Guide provides a brief overview as to how citation data is used in some key University Rankings and Assessment Exercises.

  • Research Excellence Framework (REF 2014 and REF 2021)
  • QS World University Rankings
  • Times Higher Education (THE) World University Rankings
  • The Shanghai Academic Ranking of World Universities

For further information on the REF please contact Research & Innovation Services, or for University Rankings, the Strategic Planning Office.

Research Excellence Framework

undefined

For REF 2021, Clarivate Analytics (Web of Science) will provide citation data which will be used by 11 of the 34 sub-panels (sub-panels 1-9,11 and 16).

  • Sub-panel 1: Clinical Medicine
  • Sub-panel 2: Public Health, Health Services and Primary Care
  • Sub-panel 3: Allied Health Professions, Dentistry, Nursing and Pharmacy
  • Sub-panel 4: Psychology, Psychiatry and Neuroscience
  • Sub-panel 5: Biological Sciences
  • Sub-panel 6: Agriculture, Food and Veterinary Sciences
  • Sub panel 7: Earth Systems and Environmental Sciences
  • Sub-panel 8: Chemistry
  • Sub-panel 9: Physics
  • Sub-panel 11: Computer Science and Informatics
  • Sub-panel 16: Economics and Econometrics

Details of how citation data will be used in the assessment are set out in general terms in ‘Guidance on submissions’ at paragraphs 288 to 292. Information on how these sub-panels will make use of citation data to inform their assessment is set out in the ‘Panel criteria and working methods’ at paragraphs 274 to 282. Panel guidance clearly states that "No sub-panel will use journal impact factors or any hierarchy of journals in their assessment of outputs. No output will be privileged or disadvantaged on the basis of the publisher, where it is published or the medium of its publication." [PARA 207]


Stern Review

The Stern Review, published in July 2016, recommended that citation metrics could be used to support REF sub-panels in their assessment of outputs, but that "it is not currently feasible to assess research outputs in the REF using quantitative indicators alone." [PARA 75] It was also recommended that assessment panels should set out explicitly how they intended to use bibliometric data in their working methods.


REF 2014 and citation date usage

For REF2014, citation data from Scopus was used by 11 of the 36 sub-panels, "as additional information about the academic significance of submitted outputs". Journal level metrics such as the Journal Impact Factor, were not supposed to be used to inform the assessment of research outputs by sub-panels.

  • Sub-panel 1: Clinical Medicine
  • Sub-panel 2: Public Health, Health Services and Primary Care
  • Sub-panel 3: Allied Health Professions, Dentistry, Nursing and Pharmacy
  • Sub-panel 4: Psychology, Psychiatry and Neuroscience
  • Sub-panel 5: Biological Sciences
  • Sub-panel 6: Agriculture, Veterinary, and Food Science
  • Sub panel 7: Earth Systems and Environmental Sciences
  • Sub-panel 8: Chemistry
  • Sub-panel 9: Physics
  • Sub-panel 11: Computer Science and Informatics
  • Sub-panel 18: Economics and Econometrics

University Rankings

QS World University Rankings

The QS World University Rankings uses a 6-year citation data window, for publications over a 5-year period. Citation data from Elsevier's Scopus database is used in their ranking methodology. Self-citations are excluded, and citations are normalised to account for different publication and citation cultures across disciplines.

'Citations per faculty' accounts for a 20% weighting in the calculation of the overall scores.

The QS World University Rankings 2021 place Durham 86th globally out of more than 1,000 higher education institutions, and 12th in the UK.


 

THE World University Rankings

The Times Higher Education (THE) World University Rankings use a 6-year citation data window, for publications over a 5-year period. Citation data from Elsevier's Scopus database is used in their ranking methodology. Citations are normalised to account for different publication and citation cultures across disciplines, and a fractional counting approach is used for co-authored papers to ensure all authors institutions receive at least 5% of the 'value', with those that provide the most contributors receiving a proportionally larger contribution.

'Citations (research influence)' accounts for a 30% weighting in the calculation of the overall scores.

The Times Higher Education (THE) World University Rankings by Subject 2020 place Durham University 79th among the world’s leading universities.


 

The Shanghai Academic Ranking of World Universities

The ShanghaiRankings' Academic Ranking of World Universities (ARWU) use publication and citation data from Web of Science in their ranking methodology.

Indicators used include a measure of the number of Highly Cited Researchers as identified in Web of Science, the number of papers published in Nature and Science journals, and a measure of publication output.

Together these indicators accounted for a 60% weighting in the calculation of the overall scores.

[In February 2020, China’s Ministry of Science and Technology and Ministry of Education announced two policy documents directing universities and research institutes not to use SCI-related metrics for university or discipline rankings]

 

undefined  undefined  undefined  undefined  undefined  undefined