Conversation with:
Federico Caniato, Full Professor of Supply Chain & Procurement Management at School of Management, Politecnico di Milano
Stefano Magistretti, Assistant Professor of Agile Innovation at School of Management, Politecnico di Milano

Universities are increasingly engaged in demonstrating the impact of their research. What is the impact of research? 

The impact of research is crucial not only for the Politecnico di Milano, but for the entire Italian university system, and more in general for universities worldwide. It is not easy to define what research impact is. We can say that the impact of research encompasses all the results, implications, and consequences resulting from scientific research activities aimed at generating knowledge, but they are also expected to provide concrete benefits. In our school, we have defined research impact in three progressive levels of maturity: dissemination, adoption, and benefits. Dissemination is the spread of the results and findings among the relevant stakeholders, adoption is the use of the research results by the stakeholders, and benefits are the consequences of this adoption.

Why is impact so important for research?

Research is often accused of being self-referential, i.e. ‘speaking’ only to members of the academic community without providing a significant contribution to society at large. Instead, research can have a much broader and more significant impact than expected. Therefore, it is crucial to illustrate such impacts to a broader audience, requiring researchers to learn to assess and share the value of their work with multiple stakeholders.

What is the approach to impact assessment in the School of Management?

In 2017, we started a journey in the School of Management to develop a culture of research impact assessment. This journey saw a reflection on the assessment framework, the development of a method, and the collection and analysis of the research impact assessments. We started by combing the literature for impact assessments, interviewing experts, and interacting with our international advisory board to define our framework. The framework comprises the three levels of maturity (i.e. dissemination, adoption, and benefits) and five stakeholder domains (i.e. institutions, enterprises, students and faculty, citizens, and the academic community). The second step was the adoption of the framework. This initially began in 2019 with a set of 16 pilot projects, which then extended to a more extensive set of projects (42 in 2020; 43 in 2021).

The conventional idea of ‘impact’ makes sense in a linear model: changes or discoveries in science and research are expected to cause changes in society, but impact assessment frameworks are usually far more complex, can you tell us why?

The research impact assessment is more complex because the impact is not linear. Some elements impact one stakeholder, causing indirect effects on other stakeholders. For example, research results adopted by public institutions may benefit citizens, or the results disseminated to students may be adopted later, when the students are professionals within companies. Thus, the impact network is intertwined. Seeing the link among the domains and level of maturity, and how an initiative might influence other areas of impact requires a framework that tries to bring everything together. Let’s take an example. If you publish an academic paper, there is diffusion within the academic community, but if you share it in class, there is also an impact on students; if you use it in corporate education, that novel piece of research can become the seed for a potential company project. So from a single action — dissemination of research among the academic community — you might have an impact over multiple stakeholders on different levels.

How much of this impact analysis must be made ex ante, while planning the activity, and how much ex post?

The impact assessment is a helpful tool in every moment of a research project. We saw colleagues adopting it when writing proposals for an EU project or internal research initiative. This is because the impact is both ex-ante and ex-post. The most important thing is to envision potential impact ex-ante, which helps to set the expectations and objective of the project. Ex-post assessment instead aims to measure the results obtained in terms of impact, monitor the results of the planned activities, and demonstrate the actual achievements. Thus, there is not just a single moment for impact analysis; it is always a good to measure it before, during, and after the research initiative.

Is the impact ‘native’ or built over time? Do we need our PhD candidates to be ‘natural-born impacters’ or is it an orientation that can be encouraged and sustained over time?

The impact culture is not native. It is something that PhD candidates and researchers in general should be trained in. Indeed, some impacts are easy to design and achieve, but impacts of a higher level are more challenging and require careful consideration, so it is important to build impact over time. Indeed, it is difficult to gain everything with a single new research programme. As for PhD candidates, it is probably something that we should share with them and encourage them to reflect on. This is something we started at the last AiIG (Associazione Italiana Ingegneria Gestionali) Summer School held by the Politecnico di Bari in September 2021, where we shared the framework with more than 50 Italian PhD candidates and asked them to apply it to their PhD research. The PhD candidates were positively surprised about the unexpected outcomes of this assessment exercise. Disseminating the culture of research impact assessment is something we need to do at every level.