Measuring research: what are the units of assessment?

Today sees the publication of the report of an independent review of the contentious use of metrics - numerical indicators of performance - in the assessment of UK research and researchers. Can it plot a sensible course in a world increasingly obsessed with numbers?

The 2009 movie Knowing starred Nicolas Cage as a professor of mathematics and was advertised with the strap-line: what happens when the numbers run out? Without giving away too much of the plot of this ludicrous piece of sci-fi shlock, the answer turns out to be A Very Bad Thing. But in the real world of research, where often we want to know who are the top performers - usually in deciding who gets funded, appointed or promoted - we have the opposite problem. There is no shortage of numbers. The question is rather: what happens when the numbers take over?

That question is much discussed in The Metric Tide, the report published today by the UK Independent review of the role of metrics in research assessment. The steering group, of which I was a member, was convened in Spring 2014 by then Minister for Universities and Science, David Willetts. The primary task was to examine the potential of metrics - numerical indicators of performance such as the number of times a research paper is cited by other researchers, or the average citation score embodied in journal 'impact factors' - to inform the Research Excellence Framework (REF). This is a complex and intensive exercise that, every six years or so, evaluates the scholarly prowess of university departments across the UK.