In response to HEFCE's call for responses to the proposition of using metrics in research assessment, many shared their thoughts. Here, we've collected the highlights from both blogs and Twitter. UPDATE: We've added tweets from the October 7, 2014 "In metrics we trust?" event.

byImpactstory775 Views
Favicon for https://storify.com
Favicon for https://twitter.com

  1. The original call for participation:
  2. Here are some of the responses we've seen to date.
  3. Early on, HEFCEMetrics inquiry team member Stephen Curry shared his thoughts on the benefits and drawbacks to the possibility of using any metrics, including altmetrics, in the REF exercises.
  4. Social scientists Dr Meera Sabaratnam & Dr Paul Kirby were among the first researchers to share their thoughts with the public, which started a lively debate on The Disorder of Things blog.
  5. Prof. Steven Fuller responded to many of the concerns they raised in this blog post, and the debate continued into the comments section of the blog:
  6. ...to which mathematician Tim Johnson offered his perspective:
  7. Statistician David Spiegelhalter explained that he thinks gathering metrics is a good thing, so long as those metrics are only used to supplement a more thorough examination of the quality of research.
  8. Research Councils UK submitted their thoughts, as well:  http://www.rcuk.ac.uk/RCUK-prod/assets/documents/submissions/HEFCEMETRICSFINALRCUK.pdf&nbsp ;

    "The use of a standard set of outputs data to track subsequent impacts of research and support evaluation studies is still experimental, and the most efficient route to capture this information, the most effective system to record it, and the best ways to use the data still needs refining."
  9. PLOS offered their perspective, which echoed concerns about the maturity of many bibliometrics measures and their ability to accurately capture the quality of research:
  10. David Colquhoun wrote a passionate statement against the use of any metrics--including citations--in the REF. The most salient of his points was a call for more studies that find correlations (or lack thereof) between various metrics and actual quality of research, as determined by a panel of expert reviewers. (It's worth noting that some studies to date have shown (weak to medium-strength) correlations between F1000 expert peer reviews and bibliometric indicators. 1, 2, 3, 4.)
  11. Researcher Ernesto Priego gave a great rundown of the evidence supporting (and in some cases, not supporting) the use of metrics in assessment:
  12. Research manager Simon Kerridge shared his thoughts on the drawbacks to simply adding bibliometrics to the existing REF review process, rather than optimizing the REF review process to better accommodate them: 
  13. And Martin Smith thinks that metrics aren't the problem; instead, the challenges of measuring research quality run much deeper: