Research by Robin Lumsdaine on non-governmental primary schools in rural Guinea Bissau leads to impactful publication

The study enrolled 2,112 children in their pre-primary year who are tracked over a four-year period. The intervention yielded transformative learning gains among children who would otherwise be unlikely to ever achieve literacy and numeracy. The study was funded by Effective Intervention, a UK registered charity.

Background

Guinea Bissau is often considered a “failed state” in which public services, including government-provided schools, are frequently low quality and unreliable. The official education system comprises three levels: nine years of compulsory, basic education, followed by three years of elective secondary school and then higher education. A 2010 nationally representative survey of schools, families, and children across Guinea Bissau found substantial enrolment in school. Unfortunately, these high enrolments did not translate to learning. Fewer than one third of these children could recognise a single digit number or read a single, simple Portuguese word (Boone et al., 2014).

Study Design

The purpose of this cluster-randomised trial was to understand the effectiveness and cost of concerted supply-based interventions in such contexts. In villages randomly selected to receive the intervention, non-governmental primary schools provided four years of schooling (pre-primary through third grade) in lieu of government-provided or other existing schooling in the community. Teachers in the programme schools were assisted at the start by a local adult who spoke the most prominent language in the village and monitored through classroom observations. The intervention focused on teaching Portuguese language acquisition in the pre-primary year, and the national curriculum in grades 1-3.

Findings

Robin Lumsdaine and colleagues registered their statistical pre-analysis plan (also known as an SAP or PAP) at www.socialscienceregistry.com prior to unblinding of the data (RCT ID:  AEARCTR-0003670). During the initial phases of the data analysis, Lumsdaine was assisted by bachelor student Yixun Zeng, who had received a research assistantship from the School’s Econometric Institute. While the study was unblinded to participants – it was impossible to prevent parents from knowing whether or not they were in a village that was receiving materials and teaching support – the research team that conducted the surveys and tested the children were not given information on which villages were in each arm. The team’s analysis found that children receiving the intervention scored 58.1 percentage points better than controls on early grade reading and math tests, demonstrating that the intervention taught children to read and perform basic arithmetic, from a counterfactual condition of very high illiteracy. The dramatic shift in the distribution of outcomes for children in the intervention group versus the control group is shown in Figure 1 on page 21 of the paper.

Robin Lumsdaine: 'Our research provides evidence that particularly needy areas may require more concerted, dramatic interventions in education than those usually considered, but that such interventions hold great potential for increasing education levels among the world’s poorest people.’

‘Next step: a replication RCT in another country’

Already in an early stage the researchers’ working paper attracted the attention, amongst others, of the evidence-based policy team of American philanthropy, Arnold Ventures. The team, which continuously monitors and reviews the evaluation literature in international development, assessed the paper as ‘excellent’, regarding the standards with which the research was conducted. They included a high-level summary on their online repository.

According to Arnold Ventures a next step in the research would be to conduct a replication RCT in another country, where primary education is also low quality and unreliable, in order to (hopefully) confirm the results and establish that they generalise to other settings: ‘Longer-term follow-up of this study would be desirable to see whether students’ learning gains persist once they leave the programme schools at the end of third grade.’

About Robin Lumsdaine

Robin Lumsdaine, who obtained her PhD at Harvard University, is a Professor at American University’s Kogod School of Business where she holds the Crown Prince of Bahrain Chair in International Finance. Since 2016 she is Professor of Applied Econometrics at Erasmus School of Economics through a cooperative arrangement with American University.

Previously she was an Associate Director in the Division of Banking Supervision and Regulation and Head of the Quantitative Risk Management Group at the Board of Governors of the Federal Reserve System. Before joining the Board, Professor Lumsdaine was Director in the Global Markets Research division of Deutsche Bank. She also held positions as Professor of Economics at Brown University, Senior Economist at the President's Council of Economic Advisers under President Bill Clinton, and Assistant Professor at Princeton University.  She is also a Research Associate at the National Bureau of Economic Research, a Tinbergen Institute Fellow, Fellow of the Society for Financial Econometrics, International Finance Fellow at the Center for Financial Stability, and a Netspar Fellow.