Research Synthesis and Meta-Analysis

Introduction

Meta-analysis is a method that is used to combine and analyse data from multiple empirical studies. In doing so, it provides a more comprehensive, robust and systematic understanding of the overall effect or outcome of interest. The method has become common in diverse disciplines, including the social sciences. 

Course information
 

ECTS: 3
Number of sessions: 5
Hours per session: 3

Course description
 

Meta-analysis is a method that is used to combine and analyse data from multiple empirical studies. In doing so, it provides a more comprehensive, robust and systematic understanding of the overall effect or outcome of interest. The method has become common in diverse disciplines, including the social sciences.

Working method 
 

The course offers a step-by-step guide on how to conduct a meta-analysis, interpret its findings, and effectively communicate the results. Participants will engage in lectures, discussions, and hands-on exercises to master the tools and strategies needed for performing robust meta-analyses. Participants will be encouraged to apply the concepts and tools of meta-analysis to their own research area. By the end of the course, participants will complete an independent meta-analysis project in their research area and submit a short report. The report can ultimately serve as the foundation for a peer-reviewed publishable meta-analysis.

Prior knowledge

No prior knowledge or experience with meta-analyses is required for this course. However, participants are expected to have a basic understanding of systematic literature reviews and statistical concepts, such as correlations, confidence intervals, and regression analysis. Some familiarity with research design and empirical methods would be helpful, but not a prerequisite for the course. Please contact the lecturer of the course in case you have any doubt or questions about whether you have the required prior knowledge for the course.
 

Session descriptions

Session 1: Introduction to research synthesis and meta-analysis

• Definition and importance of research synthesis and meta-analysis
• Historical context and development of meta-analysis
• Types of research synthesis: narrative, systematic review, and meta-analysis
• Steps in conducting a meta-analysis
• Formulating research questions for meta-analysis
• Introduction to software tools

Session 2: Study selection and data extraction

• Organize a step-by-step protocol for carrying out a meta-analysis
• Recap on systematic literature search strategies (this issue is extensively discussed in the ‘doing the systematic literature review’)
• Understanding and handling different types of effect sizes 
• Coding studies and extracting relevant data to generate reproducible data sets
• Assessing study quality and risk of bias
• Dealing with missing data
• Hands-on exercise: conducting a literature search and data extraction for a given research topic

Session 3: Statistical methods in meta-analysis

• Use reproducible datasets to combine and analyses.
• Discuss various techniques to explain differences within and between studies.
• Understanding the issue of publication bias 
• Interpreting funnel plots in the context of the research
• Running basic meta-analyses with retrieved reproducible datasets
• Hands-on exercise: converting between different effect size measures, explain within- and between-studies differences, calculating summary effect sizes, interpret graphical plots, and choosing appropriate models for economic data

Session 4: Dealing with heterogeneity, bias, and sensitivity analyses

• Assessing heterogeneity
• Best practices for visualization and presentation of meta-analysis results
• Dealing with underlying implied effect accounting for biases and heterogeneities
• Interpreting and communicating results in a clear and accessible way
• Sensitivity analysis
• Writing up a meta-analysis for publication 
• Discussing limitations and future directions 
• Hands-on exercise: conducting a multivariate meta-analysis, interpreting and communicate findings

Session 5: Feedback

• Individual/group feedback on final projects

In this session, participants have the opportunity to schedule individual or small group meetings with the instructor to receive personalized feedback on their first draft of a meta-analysis or a specific section of their work. These sessions are designed to offer focused guidance and constructive input, aimed at refining the analysis, improving clarity, and strengthening the overall structure of the meta-analysis. Small groups are encouraged to share feedback, as learning from both peers’ successes and mistakes is an effective way to enhance understanding and build skills. This process aims to support participants in developing a more comprehensive, and well-organized research synthesis and meta-analysis. Before this session, adequate time will be provided to allow participants to develop their final project drafts.
 

Final Assignment

Submit a short report (max. 3,000 words) describing the methodology, results, and interpretation of your meta-analysis. The report should be suitable as a draft of a publishable meta-analysis paper.