The Politecnico di Milano School of Management has been promoting a culture of assessment and improvement of research impact on institutions, companies, students, professors, citizens and academic communities since 2016
Federico Caniato, Full Professor of Supply Chain and Procurement Management at School of Management, Politecnico di Milano
Stefano Magistretti, Assistant Professor of Innovation and Design Management at School of Management, Politecnico di Milano
The contribution made by universities to society is being called into question more and more often nowadays. They are therefore increasingly asked to measure and demonstrate this contribution, which is often described as an “impact”. The traditional approach consisted in identifying three major missions: research, training and the so-called “third mission”, a broad term encompassing interactions with society as a whole, such as technology transfer, cultural promotion and external communications. However, there is a limitation to this approach, as it risks seeing the three missions as separate activities, each with its own rules and metrics.
The School of Management has been working on this subject since 2016 with a more integrated approach. Rather than viewing the three missions as separate entities, research is seen as an engine able to generate impact on multiple domains, not only on the academic community and students, but on society in general. We promote a culture of research impact assessment and improvement for this reason, in line with our mission:
“to contribute to the collective good through a critical understanding of the opportunities offered by innovation. We accomplish this mission by creating and sharing knowledge through high quality teaching, exceptional research and active engagement with the community”.
When we embarked on this journey, we aimed, first and foremost, to encourage all our colleagues to reflect on the broader impact of their research projects. In the early years, we focused on stimulating critical thinking and encouraging the creation of an impact measurement culture. In the beginning, we did not assume that all projects, from the simplest to the most complex and the shortest to the longest, would have an impact on different areas of our school’s mission. However, this culture of measurement was, and still is to this day, fundamental in assessing and demonstrating the impact on many domains and not only on the most traditional indicators (e.g. number of academic publications and number of publications in newspapers).
We therefore felt the need to develop our own model to guide impact assessment throughout the SoM, which would enable us to pursue the following objectives:
- Raising awareness throughout our entire community
- Learning to assess the impact of research
- Encouraging all our colleagues to plan, conduct and disseminate research aimed at having a measurable impact
- Improving the capacity to account for the impact generated
- Recognising the results of research conducted by the SoM
- Publishing research results both within the SoM and externally
To meet these objectives, we built a model, inspired by the scientific literature, identifying five domains and three levels of maturity of research impact.
Impact is measured in the following five domains:
- Institutions
- Companies
- Students and professors
- Citizens
- The academic community
The impact on each domain is then measured on a three-level increasing maturity scale:
- Communication of research results
- Adoption of research results
- Benefits obtained through adoption
This model was deliberately designed to be general, so that it could be adapted to the various themes and types of research conducted at the SoM. Precise indicators need to be identified for each domain and maturity level, and they should be quantitative wherever possible (e.g. number of participants at events, number of journal articles published, number of academic conferences organised), thus enabling us to measure and demonstrate impact. The indicators chosen should be consistent with the nature of each individual research project.
The model was tested first of all by a few colleagues who assessed 16 projects according to these dimensions in 2019. This enabled us to evaluate the soundness and validity of the model and identify many useful metrics for the various domains and different maturity levels.
In 2020, we invested in engaging everyone at the SoM in performing this important exercise, thereby broadening involvement and carrying out the Research Impact Assessment for 42 different projects conducted within the SoM, with at least one for each line of research of the SoM.
It was primarily an opportunity for training and reflection on the topic across the school, with sessions organised to give those involved a chance to exchange views and discuss the process.
A booklet was compiled with a summary and the main findings, featuring a wealth and variety of impacts. It will serve at first as a tool for internal communication to raise awareness and gather best practices.
Our work continues. We have already started gathering data for 2021, with a view to updating the information on the 42 projects and expanding participation even further. Our hope is that this exercise will make it increasingly possible not only to measure impact retrospectively, but also to plan the impact of research projects from the very outset. We also hope that this assessment will eventually cover all domains and reach the highest possible level of maturity, in other words, that of real benefits.
