Improving Learning Outcomes in Niger

Program Overview

MCC’s $23.1 million Niger Threshold Program signed in March 2008, was suspended in December 2009 due to a pattern of actions by the Government that were inconsistent with MCC’s eligibility criteria. The total Program budget was reduced to $16.9 million. Following the country’s return to democratic rule, MCC made $2 million available from the original Program budget in July 2012 to complete and expand the girls’ education component of the Program. USAID, using some of those funds as well as its own funds, undertook the Niger Education and Community Strengthening (NECS) Project from 2013 to 2016, which continued the work of the IMAGINE (Improve the Education of Girls in Niger) Project of improving education outcomes.

Key Findings

Evaluation Questions

This final impact evaluation was designed to answer the following questions:

1. 1
   Have the activities planned under the NECS project been implemented?
2. 2
   What is the impact of NECS and the combined impact of NECS and IMAGINE on key education outcomes? Were the impacts different for girls and boys and for children from households of different socioeconomic status?
3. 3
   Have the investments made under the IMAGINE Project been sustained?
4. 4
   Were the project investments, NECS Project alone and the combined NECS and IMAGINE Projects, justified from a cost perspective?

Detailed Findings

Enrollment and Attendance

The NECS Project began in 2012 and was designed to complement the IMAGINE Project, which built 62 girl-friendly primary schools in 10 departments in Niger between March 2009 and early 2010. NECS had a twofold objective: (i) to increase access to quality education through a variety of investments including borehole construction and maintenance, community engagement and mentoring programs, and promotion of gender-equitable classrooms and student leadership activities; and (ii) to improve student reading achievement by implementing an ambitious early grade reading curriculum.

By the end of the 2015-2016 school year, almost all schools in the NECS-only villages and the combined NECS and IMAGINE villages had a student government with a school action plan and local-language books available for children (compared to almost none of the schools in the control group). The project schools were about 60 to 65 percentage points more likely than control schools to have a mentoring program. Almost all project schools taught the new reading curriculum in first and second grades compared to almost no control schools.

Children in the NECS-only villages and combined NECS and IMAGINE villages were 9.5 and 10.3 percentage points more likely, respectively, to report school enrollment in the current school year than children in control group villages; and were respectively 11.1 and 13.6 percentage points more likely to report school attendance on the most recent day the school was open than children in control group villages. Differences between boys and girls were not statistically significant.

Impact on Learning

The NECS and IMAGINE Projects significantly improved local-language reading outcomes for girls, boys, and children across a range of socioeconomic backgrounds, but, as expected, did not have a statistically significant impact on French-language test scores. This suggests the NECS Project successfully targeted local-language reading skills, as intended. Differences in education outcomes between girls and boys were not statistically significant, except for local-language test scores in the combined NECS and IMAGINE villages, where the impacts were higher for boys. Similarly, the evaluation found no significant differences in impacts between the socioeconomic groups, except for local-language test scores in the NECS-only group, where impacts were higher for the lowest quintile.

Infrastructure Sustainability

Overall, IMAGINE’s investments in school infrastructure, school resources, and girl-friendly school features appear to have been largely sustained over the seven years since project implementation. The number of overall classrooms in IMAGINE schools and those with finished materials had increased from 6.5 to 7.2 and from 5.0 to 6.9, respectively, between 2013 and 2016. However, the presence in IMAGINE schools of gender-specific latrines and housing for female teachers declined by 24.3 and 19.4 percentage points, respectively, between 2013 and 2016, whereas the general presence (i.e. not assigned to a specific gender) of latrines and housing did not change during that timeframe.

The likelihood of having a functioning potable water source at IMAGINE schools improved from 50 percent in 2013 to 75.4 percent in 2016, which is consistent with the borehole construction and rehabilitation activities implemented under the NECS Project.

Economic Rate of Return

The evaluator calculated the economic rate of returns of the projects as part of the final report. MCC does not produce ERR point estimates for Threshold Programs.

• -4%
  NECS + IMAGINE
• 2%
  NECS-only

The combined NECS and IMAGINE Projects produced a negative ERR of -4 percent, as estimated from increased lifetime earnings for children exposed to the projects, largely because of the relatively high costs of infrastructure improvements and the low economic returns to education in Niger. The NECS Project alone was also a costly way to achieve the targeted improvements in education outcomes; the ERR was 2 percent. The combination of the NECS and IMAGINE Projects is less cost-effective than the NECS Project alone because the projects achieved highly similar impacts but the IMAGINE Project involved costly school construction.

MCC Learning

• 
  Impacts of IMAGINE and NECS Projects do not appear to be additive. The results suggest that both the IMAGINE Project and the NECS Project had positive impacts on most education outcomes. However, it is uncertain whether the combination of the two projects had additional benefits on enrollment, attendance, or early grade reading skills in local languages on children of primary school age beyond the benefits of each project alone.
• 
  The cost of education investments should account for the labor market and economic realities that project beneficiaries will face. The demand for education and the economic and employment prospects of those targeted by the investment should be explicitly accounted for when designing education interventions.
• 
  The evaluation observed a positive impact on math test scores, which was not directly targeted by the projects and may have resulted from improvements in school attendance, teaching quality, or other aspects of the learning environment.

Evaluation Methods

The NECS final impact evaluation used two rounds of clustered random assignment, the first of which was conducted for the IMAGINE evaluation. Data were collected for IMAGINE’s evaluation in 2011 and subsequently used to inform the NECS evaluation. In 2012, all IMAGINE villages were selected to receive NECS, and a group of villages that were eligible for IMAGINE but did not receive it were selected through random assignment to receive NECS. The final evaluation sample consisted of 60 villages for group A (IMAGINE and NECS combined), 82 villages for group B (NECS only), and 50 villages for group C (control, receiving no interventions).

NECS activities were first introduced in schools in 2013, and data were collected in two rounds, one in October-November 2013 and another in May-June 2016 after most NECS activities had been fully implemented. Data used for estimating the impact of NECS included a child assessment, a school survey and a household survey.

2021-002-2615