Making a difference: Impact report 2017

Discussion

Chapter Eight

Discussion

It’s not just about the publication

Researchers are assessed on their success by their publications, and it is only recently that government funders have looked to assess and reward broader forms of output and impact. This report highlights that although publications accounted for the most outputs, sizeable numbers of different outputs were reported by all funders across all categories.

The 5 areas of impact

Five areas of impact

The researchfish® system described 14 areas of impact, which can be hard to navigate for a general user. By collating these into 5 areas aligned to the aims of research funding, it brings more structure and allows for the linking of the outputs of research to the research strategies of funders:

Looking across each impact area, it is clear to see the beginnings of some patterns. As would be expected, the majority of the outcomes were not generated until 3 or more years after the award started, demonstrating the importance of long-term awards. The full impact of these awards may be seen for years to come, so it is vital that all funders have mechanisms to capture impact after the award is completed.

Over half of the awards generated publications, demonstrating that charity funded researchers are integral to creating new knowledge and models that allow research to develop. Given that almost half of the awards in this analysis were looking at the ‘cause’ of conditions, it is vital that this new knowledge is shared widely, to ensure that other researchers can use it to further their understanding, develop new theories, and create new areas for translatable research.

A third of awards reported examples of further funding and partnerships linked to charity-funded research demonstrating that charity funding is having a catalysing effect – leading to more investment, and wider networks that can allow the knowledge created by research to be used.

Engagement activities were also linked to a third of all awards, reflecting the importance of engagement with the general public within the academic community.

Translated research ideas (products, protocols or treatments) had the lowest number of outcomes. Around 10% of the awards generated protected and licensed intellectual properties, spin out companies, and software or technical products. This small percentage may well be because a large proportion of the awards being tracked were in their first 3 years, and so might be ‘too early’ to report. It also reflects the reality of research – that not every idea can be successfully translated, and that a number of awards and areas of work often join together to create the new product, protocol or treatment.

Analysis across cause, cure and care

More than 95% of outputs were linked to awards that could be coded in some way using the Health Research Classification system. By using the ‘research activity’ code, we were able to group awards into ‘cause’, ‘cure’ and ‘care’.

Looking at how impacts are spread across the cause/cure/care continuum, it is clear that more translational outputs would be expected in cure awards than care or cause and that more engagement and influence would be expected in care than cause or cure.

Having said that, all kinds of impact were seen in all areas of research, echoing the findings of the Kings /Digital Science REFlections report that impact is widely distributed: ‘The research underpinning societal impacts is multidisciplinary, and the social benefits arising from research are multi-impactful’. [1]

Challenges for medical research charities in assessing impact

The researchfish® impact measurement system has been designed to capture the outputs and impact of research across large portfolios, where a funder is interested in all aspects of impact. Quantitative analysis across[2] large numbers of awards over many years, can lead to important insights into how researchers are influencing the wider world.

But many medical research charities are supporting research in much smaller numbers, across a narrower field of endeavors. As strategic priorities, they focus funding on developing new researchers, or encouraging early stage translation. For these funders, collecting impact across all 14 fields of researchfish®, or even within all 5 of the thematic areas we identified may be of less value, than focusing in on the specific areas of strategic interest.

The second challenge funders face is the long timescale for impact to be realized. In the ‘average time to impact’ charts, we can see that some impacts (‘partnerships’ and ‘award and recognitions’) are reported within the first year or two of awards, while others take longer. This may be due to the partial nature of our data - many funders only tracked research active in 2013 and 2014, and so most awards were only in their third or fourth year since award. By collecting data over a number of years, this should help these patterns to become clearer, and allow funders of limited portfolios to choose the optimum time to start tracking impact.

So what difference do medical research charities make?

Having assessed more than 64,000 outputs reported to medical research charities using the researchfish® system, it is clear that all charities regardless of size can demonstrate an impact. Not every research award will lead to an impact, and not every funder will be interested in creating impacts in each area, but the system allows the comprehensive collection of this information.

The case studies in this report exemplify these impacts, and show that impact data collected via researchfish® can act as a basis for narratives that bring research funding to life.

Aggregating impacts across funders allows AMRC to start to see how our sector might compare to UK government funders, and allow for international comparisons. At this stage, with partial data (from a subset of AMRC members, across a subset of their awards, over a limited number of years) there is a need to be careful not to over interpret the data. However, this report still demonstrates the scope of impacts that charity funding contributes towards, and reflects the appetite of medical research charities to show the vital role they play in the UK medical funding ecosystem.

References

  1. http://www.kcl.ac.uk/sspp/policy-institute/publications/Analysis-of-REF-impact.pdf page 71 ↩︎

  2. https://www.mrc.ac.uk/publications/browse/economic-impact-report-2015-16/ ↩︎