Science of Science

All things science of science including scientometrics, measuring real world impact of scientific work and text mining scientific papers.

Photo by Dom Fou on Unsplash: https://unsplash.com/s/photos/lecture?utm_source=unsplash&utm_medium=referral&utm_content=creditCopyText

Comprehensive Impact
Tools and Metrics for Comprehensive Impact

Comprehensive Impact

Comprehensive Impact, as opposed to Academic Impact is an umbrella term for the impact of scientific work on society, economy and policy. Comprehensive impact covers a broad range of activities and is difficult - if not impossible - to define prescriptively. I coined the term Comprehensive Impact in my 2017 paper Measuring scientific impact beyond academia: An assessment of existing impact metrics and proposed improvements

Why Should We Study Comprehensive Impact?

Why Scientists Care About Comprehensive Impact

Lots of papers write about the effect that the UK's focus on comprehensive impact affects the quality of research and individual researchers

Why Research Funders Care About Comprehensive Impact

For research councils, being able to illustrate how their research impacts the economy and society helps them to compete for and justify their continued funding.

Why The Public Care About Comprehensive Impact

Shortcomings of Academic Impact and Where Comprehensive Impact Helps

Scientific papers are

Related Definitions

REF Definition of Impact

The Research Excellence Framework (REF) - a UK-wide scientometric instrument for research quality defines impact as:

an effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia - REF 2011

The REF definition of impact is ambiguous as it does not specify the type of impact - I would argue that generating lots of citations is still impact but limited to academia. That is why we specify the term "comprehensive impact" when we talk about this concept.

Tools and Metrics for Comprehensive Impact

Research Excellence Framework (UK)

ResearchFish (UK)

ResearchFish is a tool used by many funding bodies in the UK as a way to track outputs and impacts of scientific work. ResearchFish is used almost universally by all public funding bodies in the UK. ResearchFish primarily a data collection tool into which the scientific investigators enter data about their work and corresponding outputs via a fine-grained question set. This information is fed back to the funding bodies. It is based on early work within the RAND group led by Dr Steven Wooding.

In their recent report (mirror), the authors describe their definition of Impact as being the same as that of REF's.

They use the word output to describe both academic and wider (or comprehensive) impacts of research

Figure 1 From "Researchfish: A forward look." Hinrichs, Saba, Erin Montague, and Jonathan Grant (2015).