Birkbeck team devises new method to measure research impact
Index provides fresh approach to take account of factors such as 'influence' in assessing impact.
![A book to represent the new method of measuring impact of research.](https://www.bbk.ac.uk/news/research-impact-measure/@@images/c5da8eb2-6b66-401d-86d1-871b452947ac.jpeg)
Researchers from Birkbeck’s Department of Computer Science and Information Systems have proposed an innovative new index to measure the impact of research.
The team - Professor Mark Levene, Professor Trevor Fenner and Dr. Martyn Harris from Birkbeck and Professor Judit Bar-Ilan from the department of Information Science at Bar-Ilan University – has identified the χ-index (chi-index) as a more informative way to indicate academic performance.
Debate about quality-versus-quantity in evaluating effectiveness has continued for a number of years. Many rely on the use of the “h-index” which was proposed more than a decade ago and plots the number of publications (quantity) for a researcher against the total number of citations to these publications (quality). For example, if a researcher’s h-index is 10, then he or she has published at least 10 papers each with at least 10 citations.
However, in a paper published by PLOS ONE, the team has put forward its own measure, the χ-index, comparing its effectiveness with that of the h-index.
This new index takes into account other factors, such as the way some researchers may tend to be considered influential even with only a few highly-cited publications, while others may tend to be prolific, with numerous collaborators and publications but fewer citations.
Their paper, entitled A novel bibliometric index with a simple geometric interpretation, gives an example using three researchers: one with a single publication with 100 citations; a second with 10 publications each having 10 citations; and a third with 100 publications each having a single citation. The first (influential) and third (prolific) researchers have an h-index of 1 while the second has an h-index of 10. However, using the new measure, each of these examples would have an index of 10
The team carried out a comparison of the different measures on two data sets, a Google Scholar data set of over 34,000 researchers across all disciplines and a much smaller data set of 99 Nobel Prize winners. For the first, the χ-index was significantly larger than the h-index for around 28% of the researchers, while for the Nobel Prize-winners the χ-index was significantly larger than the h-index for just over 62% of the laureates.
Professor Levene said: “We believe that the ability of the χ-index to distinguish between influential and prolific may lead to a finer ranking of researchers than allowed by the h-index. As metrics such as the h-index are increasingly being used to evaluate researchers, it is important that new alternatives such as the χ-index are put forward.”
Further Information
- A novel bibliometric index with a simple geometric interpretation
- Department of Computer Science and Information Systems
- Professor Mark Levene