Citations and Quality of Research

From the THES (14/12/07): The quality of academics' research cannot be judged on the basis of the number of times their papers are cited by peers, a leading higher education think-tank has said.The Higher Education Policy Institute says in a report this week that "there must be real doubt" over the validity of plans by funding chiefs to allocate about £1.4 billion a year in "quality research" funding on the basis of journal article citations, as is planned after next year's research assessment exercise.
Hepi's report on the Higher Education Funding Council for England's proposed research excellence framework (REF) also implies that Hefce may be ignoring the advice of its own "appointed experts" in pursing the change. "Citation analysis does not measure quality, so there must be real doubt about whether it can be used as a basis for allocating QR (quality research funding)," the report says. "This is awkward in light of the commitment of Hefce to continue to allocate research funds on the basis of quality."
Hepi's director, Bahram Bekhradnia, who as a former Hefce director was the architect of the current RAE, said this conclusion "merely reiterated" what had already been said by a group of academics from the University of Leiden who were commissioned to undertake a scoping study on bibliometrics for Hefce. The Leiden group said that citation analysis measures research impact, not quality, and that it should be used only to allocate funding in conjunction with peer review. In the report, Hepi argues that the results of citation analysis and the current RAE, which most agree does measure quality, should be considered side by side to see how closely they match up.