Understanding our societal impact is important because EUR is undertaking a transition towards an impact-driven university. But it is also a challenge because it requires deep reflection and continuous dialogue at all levels of the organization, for example about the question what ‘positive’ impact consists of.
Many different definitions of societal impact circulate in the academic world and beyond. Ultimately, as a university we create societal impact when our core activities contribute to a better understanding of societal challenges and the capacity to deal with them. The ESI project does not aim to implement one authoritative definition for the entire university, as the types of impact differ significantly between and within disciplines and research projects.
Guiding principles
Based on our expertise and knowledge, the ESI team has formulated ten guiding principles for the evaluation of societal impact of research:
- Societal impact occurs when research helps society deal with the challenges it faces;
- An impact-driven university knows about, supports and promotes the societal impact of its research;
- Clear strategies enable research to increase its impact;
- Impact assessment is part of a learning process that aims to reflect on and improve strategies for achieving impact;
- Impact is assessed in light of the impact ambition and strategy of the project or group concerned;
- Ambitions for impact specify goals, target groups and interaction mechanisms (theory of change);
- Evaluation combines narratives (productive interactions + impact pathways) and context-specific indicators;
- Evaluation also maps how the institutional environment supports and facilitates the creation of impact;
- Stakeholders from society play an active role in creating ambitions for, and assessing, societal impact;
- By 2024, impact ambitions, activities and assessments are embedded in ongoing processes so that evaluating societal impact is part of a learning, responsible and inclusive organization;
Underpinning these guiding principles are core concepts. In the drop down menu below each core concept is described including signposts to relevant practical information in the toolbox section of this website.
Evaluation is a systematic way of reflecting on and assessing the value of what is being done (i.e. a project, a programme, an event). Evaluation is commonly interpreted as an end product or an activity taking place at the end of a project. However evaluation can also be considered as a process, taking place across all phases of a project, used to determine what has happened and whether the initial aims of the project have been carried out and achieved. Evaluation can do more than assess and measure; it helps set the stage for a culture of learning, change and improvement.
We put the development and learning process central (formative evaluation), instead of looking back and assessing completed work (summative evaluation). In accordance with the state of the art in the field and the recently updated Strategy Evaluation Protocol (SEP), impact assessment is then part of a strategic approach in which impact ambitions, activities, evaluation and learning are intertwined. This entails a shift from impact as something that just happens – a welcome, but accidental by-product – towards impact as an explicit goal for which specific actions are taken and a stimulating environment is provided.
Impact ambitions are central to formative evaluation of academic teams and research units. Awareness of the current potential impact and a theory of change are key to the formulation of an impact ambition.
An impact ambition describes:
- The societal challenges that a group will contribute to;
- The ultimate beneficiaries, intermediaries and other societal partners;
- The interaction mechanisms and impact strategies to reach these actors;
- The ultimate change that is aimed for.
A stakeholder consultation can greatly improve the mapping of a group’s potential and can be the start of a process of meaningful interaction with society. A theory of change comprises a deliberate chain of activities and conditions by which this potential impact could be realized. Based upon this, a research group or project can deliberately choose its impact strategies as well as the specific means and activities by which impact is generated.
If you want to know more about developing an impact ambition and how to organise valorisation, take a look at this article.
How can researchers think about and organize their societal impact? Even though societal impact implies a focus on an end-result, what matters are the processes of interactions between various actors.
Many roads lead to Rome, and many pathways can lead to impact. Often it differs per societal sector and scientific discipline what kinds of activities and interactions are suitable, accessible and effective - see table below for examples.
Table 1: Various activities that researchers can employ as part of their pathway to impact.
Professional training | Academic consultancy | Action research |
Commercialization | Evaluation & monitoring | Co-creation |
Policy advice/criticism | Platforming voices | Design methods |
Academic education | Public engagement | Engaged scholarship |
Science communication | Personnel secondment | Living labs |
Tools and services | Stakeholder dialogues | Citizen science |
Based on the defined impact ambition and strategy, specific activities can help to reach those goals. The different types of impact activities define the impact pathways of a project. Further, they influence a projects outputs and eventually outcomes. Therefore, impact activities should always be chosen with the overall aim of the project in mind. Read more about this in the Theory of Change guide.
The capacity of an organisation to achieve impact involves creating the organisational, material, and personal conditions to work on impact. This requires specific knowledge, infrastructure and support for impact. Ranging from free parking for stakeholders to the types of roles that need to be present within a team.
Having an impact strategy and plans in place helps but to create societal impact you also need impact literate staff and a supportive impact literate institution. It starts with fundamental knowledge and understanding about impact. Bayley & Phipps (2017) define impact literacy as the ability “to identify appropriate impact goals and indicators, critically appraise and optimise impact pathways, and reflect on the skills needed to tailor approaches across contexts.”
On an individual level, a researcher can be considered impact literate if they understand the specific benefits being sought, the activities which would achieve these, and how approaches need tailoring in specific contexts:
1.the identification, assessment, evidencing and articulation of impact endpoints (“what”);
2.the practices that create impact (“how”);
3.the successful integration of these by research impact practitioners (“who”).
Although impact is created and captured at the level of the research project, it is delivered within an institutional structure. An individual can build their own competencies, but the extent to which they can implement these depends on their institutional work context. Similarly, institutional drives to improve the non-academic benefit of research are ultimately proportionate to the abilities of those involved to deliver on higher level goals. Therefore, individual capacity building for impact needs to be supported by institutional approaches and institutional capacity building needs to be informed by individual expertise (Bayley & Phipps, 2019).
To find out more about the impact environment building activities that ESI has contributed to in the past, take a look at the Impact Dialogues that we have hosted in the past to foster the dialogue on impact at an EUR wide level.
We distinguish 9 types of impact that can be defined as follows:
- ECONOMIC IMPACT: Driving economic growth, generating new products and services and creating jobs.
- POLICY IMPACT: Informing, influencing and improving decision-making by government and public bodies, NGOs and in the private sector. Increasing the efficiency and/or quality of public services, directing investment to priority areas and raising business productivity.
- SOCIAL IMPACT: Informing public debate, stimulating public interest, improving welfare, equality and inclusion, and improving quality of life and opportunities.
- HEALTH IMPACT: Creating new drugs and treatments and developing new therapies. Improving education and training, public awareness, and access to health care provision, as well as policy, legislation, standards or guidelines.
- CULTURAL IMPACT: Enhancing and preserving our cultural heritage, producing cultural artefacts, creating, inspiring and supporting new forms of expression, and enhancing our understanding of minority groups and communities.
- ENVIRONMENTAL IMPACT: Delivering energy savings and reduced emissions, improving management and conservation of natural resources, stimulating public awareness and influencing policy, improving business and public service operations, and environmental risk management.
- TECHNOLOGICAL IMPACT: Developing new and improving existing technologies.
- LEGAL IMPACT: Improving law enforcement methods, effecting legislative change and improving legal practice and access to justice.
- GLOBAL IMPACT: Delivering positive impacts from our research overseas including collaborating with partners in other countries.
These types of impact are relevant to describe what kind of impact a project aims to create. Specifically, within impact evaluation it is relevant to describe the area that a project aims to or has impacted. As mentioned before, the impact pathway (activities, outputs and outcomes) should match the type of impact you are aiming to create.
Many organizations in the wider academic environment are working on connecting science and society. Here we provide a (non-exhaustive) list of organizations with links to their resources. Some resources are on related concepts and developments in academia, such as open science, sustainable development goals and responsible research & innovation.
National
VSNU
With the Association of Universities in the Netherlands (VSNU), 14 Dutch universities show the outside world how they fulfil their social function, formulate shared ambitions relating to academic education and research and valorisation, and lobby for the preconditions needed to realise these ambitions. The link to society is stated explicitly for several VSNU activities:
- Strategy Evaluation Protocol (SEP) for research
- Recognition and rewards of academics
- Knowledge transfer
- Open science
NWO
NWO defines societal impact as ‘Cultural, economic, industrial, ecological or social changes that are entirely or in part the consequence of knowledge and expertise generated by research.’ The current strategic plan ‘Connecting Science and Society’ includes several themes that relate to societal impact: open science, knowledge and innovation covenant and knowledge utilisation.
The most explicit link NWO makes about creating societal impact is for knowledge utilisation. This is defined as an iterative process where the chance of societal impact increases by stimulating productive interactions with social stakeholders both during the development and in the execution of the research. NWO's three knowledge utilisation approaches vary in type of activity and stage in proportion to the expectation of societal impact.
- Impact Outlook Approach: to facilitate (unforeseen) opportunities for social impact during the project term.
- Impact Plan Approach: for contributing to social issues
- Impact Focus Approach: for applying already developed knowledge and insights
If you are planning to apply for a NWO grant, there are e-learning modules to help you get started on incorporating societal impact into your proposal: https://impact.nwo.nl/en
ZonMW
The Netherlands Organisation for Health Research and Development (ZonMW) has several societal impact related resources. The starting point is the 'Strengthening impact' page, but the 'Demonstrating impact' also has more specific information such as implementation activities.
KNAW
The Koninklijke Nederlandse Akademie van Wetenschappen (KNAW) published a report in 2018 ‘Maatschappelijke impact in kaart’ with several policy recommendations in relation to societal impact evaluation.
International
Sustainable Development Goals (SDGs)
The United Nations developed 17 interlinked global Sustainable Development Goals (SDGs) designed to be a "blueprint to achieve a better and more sustainable future for all". Though broad and interdependent, the 17 SDGs were made more actionable by identifying specific targets with progress indicators for each goal. There is a full list of 169 SDG Targets and 232 unique progress indicators. Mapping how academic activities relate to the SDGs can aid the identification of societal relevance.
Within the EUR, sustainability is one of seven EUR priorities also embodied by the strategic Sustainable Development theme. The Rotterdam School of Management (RSM) uses the SDGs as a reference framework for their mission. In the rough guide section of this website we will add information about a SDG mapper that is being developed by research intelligence advisor Tung Tung Chan.
Responsible Research and Innovation (RRI)
Within the European Union Horizon 2020 program Responsible Research and Innovation (RRI) is the key action of the ‘Science with and for Society’ objective. RRI is an approach that anticipates and assesses potential implications and societal expectations with regard to research and innovation, with the aim to foster the design of inclusive and sustainable research and innovation. Within ESI, research intelligence advisor Tung Tung Chan has RRI in her portfolio.
Open science
Open science is a movement towards more open and collaborative academic practices. This can be about sharing and re-use of academic output at the earliest possible stage (e.g. publications, data, software), but also involving other parties throughout the process (e.g. citizen science). Open Science is a policy priority for the European Commission. For the Dutch context, a recent Rathenau institute report 'Samen verder met open science' described when and how to organize meaningful public involvement as part of open science.
Within the EUR, the Open Science Community Rotterdam (OSCR) (coordinated by Antonio Schettino) is an important driver for open science.