“How we can assess, value and enhance our societal impact?”

“Two years ago, I was asked to write the chapter on societal impact for the strategy of the university, via the seven design team labs. We wrote the chapter about (positive) societal impact in four months. We made proposals about how you can augment this impact. But we also said: you need to be able to assess and value it and make it more visible. What is our impact, and how can you recognise and confirm the impact of our research and education?”

And do you already know what the answer is?

“Well, we have just started a four-year project to explore these questions. So hopefully we will know the answers to these questions by the end of it. To start off with, we are going to compile an inventory of what the faculties are already doing. There are certain differences in that respect. In parallel review how the literature defines impact, and the methods that it offers to evaluate impact. There are some major differences in this area too.”

What types of differences?
“A lot of literature says: impact is about social relevance. Does the research tackle the challenges facing society? We don't think the scope of this definition is broad enough. If your research is relevant, but you aren't able to get it into the spotlight, then sometimes you don't necessarily have impact. Others say: impact is about the way your research and education is used. For example, you can look at the number of citations or contract research projects, but also the number of people who attend your lectures, or the number of articles you write for a wider audience. Nonetheless, this approach doesn't cover everything either: even if you are able to communicate your research successfully, that doesn't always necessarily mean it actually has any positive benefits for society.

We have decided to define impact as the extent to which our research and education helps resolve the challenges facing society, such as inequality, public health, and unemployment.”

What do you mean by ‘helps’? Does research always have to provide a solution?

“In certain cases, it can make a direct contribution to the finding of solutions, yes. You see that happen a lot within, for example, the faculty of medicine. But if you look at sociology, for example, then the purpose of research within that discipline is more about holding up a mirror to society. In that way, research can help to clarify a problem or perhaps show that a certain solution is not working or is actually having a counter-productive effect.

A colleague within ESSB is carrying out research into the factors that explain why some teams perform better than others. It focuses on the case of neighbourhood care teams in the city. He has designed the research in such a way that it actually helps these teams improve their performance. We can adopt this type of approach more often. And afterwards, the neighbourhood teams will be asked: did the research help you, and if so, in what way? Impact is about the complex relationship between the research itself and the effects that it produces.”

Isn't it much more difficult for econometric research to contribute to the resolution of societal challenges, or more difficult to measure that contribution, compared to public administration research into neighbourhood teams in Rotterdam South?
“The contribution of a more fundamental area of science to the resolution of social issues is often more indirect. But even with a fundamental field of science, as a scientist you still have the responsibility to make sure the knowledge that you generate provides useful stepping stones for the next link in the research chain in terms of practical application. The most fundamental econometrician can still reflect on how their models or computations can help a more applied area of science with the resolution of social issues. And that actually happens a lot in practice.”

So the nature and extent of impact that is measured will vary considerably from one faculty or specialist field to the other?
"Absolutely. The article of Paul Scheffer about ’the multicultural drama’ had an enormous impact, but the nature of the impact is different, and is measured in a different way than, for example, the impact that you create if you develop a vaccine. All the variations have value, and we want that to be recognised.

It is also important to point out that as a university, we should be asking our stakeholders more often about the relevance and usefulness of our research. For example, I think we could go to the city of Rotterdam a lot more often and ask them: we did this research last year, did it have any meaningful benefits? Was it relevant and, if so, in what way? We should give our stakeholders or ‘partners’ more of an input. All too often, we do the research first and then try to work out how it can be applied in practice afterwards. If you start by defining your impact ambition, then you turn it the other way round. What is your societal impact ambition, and how will your research or education help you to achieve that ambition? The idea of explicitly formulating your societal impact ambition beforehand is completely new for a lot of research groups.”

At what stage should stakeholders be involved?

Right from the start. During the opening of the academic year, Eveline Crone quite rightly said: ‘I reach out to stakeholders when I’m formulating my research question. That way they can tell me if I'm asking the right question.’”

Is that really such a new concept?

“For some researchers it is, but not for others. It is becoming more commonplace, though. And more and more funding bodies, such as NWO, ZonMw, and ERC, want to know what the expected impact will be, and what parties will be involved, when the initial application is submitted.”

What are the first steps in the four-year project on ‘Evaluating Societal Impact’?

“We are currently holding discussions with the faculties. We are drawing up a list of all their wishes in the field of impact evaluation. We are going to work together with the economists and the psychologists on their self-assessment for the research quality review. With the faculty of medicine, we are going to look at what the impact is of large-scale educational innovation. They are going to modify the bachelor programme, so we will do a baseline measurement now and then another one in a year’s time. With the RSM, amongst other things we will make an analysis of the impact of their part-time PhD programme. In addition, we are working on policy proposals in relation to impact and how to raise the level of awareness. On 26 November, we organise a big impact congress at the EUR. And we are going to develop training courses and events on how to achieve impact.”

Are you going to make any recommendations?

“The ultimate aim is to become an impact-driven university, whereby an impact ambition is formulated at all levels and strategies are put in place to achieve this ambition. The evaluation of impact is a crucial part of this. So we need to have definitions, tools, indicators, and methods. These will be developed as we go along. We are asking the faculties a very simple question: how can we help you? And then we will work on it together. Within Public Administration, we have carried out an impact assessment for two research projects, and that resulted in the development of a method that we can apply for other disciplines as well. In this way we will develop a framework and toolbox for impact evaluation through the practical application of scientific research that will help us to continue improving our performance in this area.”