While the Research and Excellence Framework (REF) has been devised primarily as an instrument to assess the quality of university research and of institutional research environments, the information collected is rapidly proving an invaluable source of qualitative information about the type of research that universities do, and how this generates real-world impact. In particular, the impact case studies submitted to the REF can be used to address the need for more and better evidence about research impact in relation to arts and culture.
In a recent report, we have carried out an exploratory analysis of impact case studies submitted to the Research Excellence Framework (REF) 2014, with the objective to identify the key actors and processes involved in the generation of arts and culture-related impact: the research performers, the sectors that benefit from the research, and the research process that leads to impact generation.
Relying on the REF2014 data has two advantages: first, although the evidence has been submitted in 2014, the actual case studies can span a period of up to 20 years, providing a long term view of how universities generate arts and culture-related impact. Second, as we are expecting the outcomes of the REF2021 to be released in a few months’ time, this investigation of the prior exercise can provide a benchmark against which we can compare the evidence from REF2021.
We adopt a broad definition of arts and culture-related impact, considering both:
- arts and culture-related knowledge and practice produced by universities that generate impact outside academia (including the arts and culture sector as well as on other sectors); and
- knowledge and practice produced by universities in other fields that generate impact on the arts and culture sector.
Our sample consisted of 793 impact case studies where the words ‘art’, ‘culture’ and/or their compounds were mentioned – which made up 11.9% of all cases. According to the above definition, around 62% of cases submitted to Art and Design: History, Practice and Theory and to Music, Drama, Dance and Performing Arts (and 59% of cases submitted to English Language and Literature) have arts and culture-related impact. The knowledge base supporting arts and culture related impact falls mainly within four subject areas: Cultural Studies, Historical Studies, Literary Studies and Film, Television and Digital Media. However, case studies are reported across a very wide range of areas across many other units of assessment, right across the Arts and Humanities as well as the Social Sciences with some, albeit fewer case studies within STEM (science, technology engineering and maths) based subjects.
There are strong differences across Units of Assessment in terms of the types of academic units that perform the underpinning research. Individually-driven research (either by individuals or informal collaborations between individuals) is prevalent in the arts and humanities, while team research (either done in the context of formal organizational units or formal research projects) is prevalent in most of the social sciences, and in STEM (though both approaches are to some extent present in all subjects).
Considering the set of eight impact categories provided in the REF database, which are self-selected by the submitting units, the majority of the impact (76.92%) is cultural in nature, followed by societal (19.04%) and then, as a distant third, technological (2.02%). The other categories of impact only have marginal importance. Overall, 84% of case studies have impacted the arts and culture sector, while 75% of case studies have had some impact on sectors other than the arts and culture.
Through our thematic analysis of the 793 case studies, we identified nine key research processes generating arts and culture related-impact, which are shown in the next figure.
Research processes leading to impact generation
The two key types of process accounting for the majority of the case studies were: ‘Analysis of artistic/cultural research and production by others’ and ‘Artistic or cultural production’.
In terms of impacted sectors, we find that for most research processes, a majority of cases (more than 70%) have impact on the arts and culture sectors, with one exception: artistic and cultural production, where only about 40% of cases have impact on the arts and culture sector, while almost 90% have impact on other sectors, particularly the general public and the education sector.
The implications of our findings are manifold.
First, the analysis demonstrates that the rich evidence base collected through the REF can serve a wider function as a resource for sectors including arts and culture.
Second, our findings confirm that the process of academic research and the process of impact generation are intertwined and complex:
- the processes of academic research generating impact on the arts and culture sector take many forms, with vast differences in elements such as the scale of research endeavour, the duration of the processes, and the scale and number of interactions involved;
- there are cross-cutting impacts between disciplines and sectors: research in the arts and humanities generates impact on the arts and cultural sectors as well as on other sectors (the general public, education, health, tourism, vulnerable groups…) while research in other disciplines such as STEM generates impact on the arts and cultural sectors.
- It is not possible to identify one to one correspondence between research processes and types of impact / impacted categories within the arts and culture sector / impact on other sectors.
Third, whilst the largest focus emerging from the research case studies tends to be ‘on’ the arts and culture sector, it is also very interesting to note how strongly artistic and cultural production itself is also featured in the case studies. This may indicate that practice-based research - as well as emergent ways of identifying practices that are located both in and beyond the academy, such as pracademia - constitutes a major part of the research ecosystem, and that this ecosystem is in turn likely to be one that is supported and nurtured through a variety of mechanisms.
Finally, the identification of a set of research processes leading to arts and culture-related impact can yield some useful insight for impact measurement. In particular, while the narrative approach seems to be the most suitable method for collecting information on research case studies, people should be given a broad range of possible ways to demonstrate impact, with the possibility to pick which metrics best suit the nature of their research. Suitable metrics can differ according to the type of research process, for example, metrics capturing audience engagement could be suitable for artistic/cultural productions; metrics capturing artistic and cultural output and their use could be suitable for databases and development of technology; metrics capturing efficiency gains, improvements in quality, new products/processes generated, could be suitable for service / process design.
We look forward to the outcomes of REF 2021 and to what they will tell us about the arts and culture-related impact of academic research in the UK.
The full report is available here:
How does academic research generate arts and culture-related impact? A thematic analysis of Research Excellence Framework (REF) 2014 impact case studies, NCACE. Rossi, F., Wilson, E., Hopkins, E. 2021.