Improving Education Quality in El Salvador

Program Overview

MCC’s $271 million El Salvador Investment Compact (2015-2020) funded the $100 million Education Quality Activity to better align students’ skills with those demanded by employers. The activity intended to improve teaching and create school systems comprised of nearby schools that shared teaching resources and had at least one secondary school, and was based on the theory that these interventions would lead to fewer students dropping out and more students transitioning from middle to high school. By the end of 2021, 47 schools were constructed or rehabilitated and 77,578 students were enrolled in MCC-supported schools.

Key Findings

Evaluation Questions

This final performance and impact evaluation was designed to answer the following questions.

1. 1
   How were the components implemented, were they implemented according to plan, and which program targets were and were not met?
2. 2
   What were the key facilitators and challenges in implementing each component?
3. 3
   What plans or actions for the sustainability of the program are in place or in progress?
4. 4
   What is the impact on dropout, progression and graduation rates from secondary school?
5. 5
   What is the impact on students’ academic performance?

Detailed Findings

Program Implementation

Despite delays, the project implemented all its components and met most of its output targets. The components included: teacher training, improving school governance and management, strengthening learning evaluation, designing and implementing technical programs in secondary schools, improving English language instruction, offering after-school workshops, establishing reading communities, improving the education information management system, supporting the design and implementation of the gender-equity policy and improving school infrastructure. In 44 of the 45 school systems, the secondary school was equipped with modern classrooms, science labs, libraries, sports courts and cafeterias.

To improve the alignment between the private sector’s needs and the technical programs offered in high schools, new technical and certificate programs were supposed to be designed with input from the private-sector committees from the TVET System Reform Activity, implemented as part of the compact. Although the new technical-certificate programs were designed and implemented, they were not developed in conjunction with sectoral committees from the TVET activity.

Several factors contributed to implementation delays. The original design was very ambitious, exceeded the budget, and required characterization studies to assess the needs of the 45 integrated systems. These studies took two years to complete, which reduced the time available to implement the activities. Because of this shortened period, teachers were overloaded with too many training sessions, especially in 2019. Students were also affected because teachers were not present in the classroom while they attended training sessions. While the school infrastructure was rehabilitated, it was difficult to find spaces for student instruction.

The collaboration and flexibility of FOMINELIO II (the accountable entity), the Ministry of Education, and the implementers facilitated implementation. Collaboration between the implementers and the Ministry was essential for clarifying issues related to work streams and deliverables. The flexibility of all stakeholders in finding solutions to challenges was key in reaching agreements and attaining implementation targets.

Program Impact

The transition from middle school to secondary schools was a key outcome for the activity and the evaluation found statistically significant and positive impacts on student transition from middle school to high school. Transition was calculated as the percentage of 9th grade students who enrolled in 10th grade in any secondary school in the country in the following year. Students receiving the intervention had higher transition rates to any high school than students who were not part of the program, and the difference was significant for all years except 2021-2022. For student transition to a high school within the same school system as their middle school, the evaluation also found significant and substantive impacts on the treatment group for this indicator for all years except 2018-2019.

Students attend either general or technical high schools in El Salvador. No consistent impacts were found on high-school drop-out or repetition rates for students in either type of high school. The project had a positive and significant impact on reducing dropout rates for 10th grade. From 2020 to 2021, fewer technical high-school students in grades 10 and 11 repeated a grade. The evaluators did not find impacts on repetition or dropout rates for general baccalaureates.

The project had no consistent impacts on high-school academic achievement. The evaluators found a significant impact on test scores for technical high-school students in 2018 and for general high-school students in 2020. However, these impacts were not consistent in other years or across subject areas.

In addition, for middle-school students, no impacts were found on grade-progression, repetition and dropout rates.

MCC Learning

• 
  The lack of clear problem diagnosis and prioritization resulted in a complicated activity and in delays while key information was collected.
• 
  The risk of “training fatigue” should be considered in the design of projects, as well as incentives for teachers and staff to participate in these trainings.
• 
  To avoid disruptions to student instruction, plans should be made for teacher coverage during trainings, and alternative venues for classrooms should be identified prior to construction.
• 
  Effective communication, qualified key staff, and a shared vision among stakeholders were key to the success of implementation.
• 
  If data-quality issues are diagnosed early, improving data systems can be part of program implementation.

Evaluation Methods

The evaluation comprised both a performance evaluation of the entire activity and an impact evaluation of one sub-activity, the Full-Time Inclusive Schools Model (SI-EITP in Spanish).

The performance evaluation sought to (1) provide a comprehensive description of the activity’s implementation and (2) evaluate the extent to which the activity met its performance goals and contributed to the desired outcomes.

The impact evaluation sought to estimate the impacts of the SI-EITP model on student outcomes. From 147 school systems under consideration, the project randomly selected 45 school systems to participate in the project as the treatment group and 55 school systems to track as the control group. Outcomes for all middle-school and high-school students in the treatment schools were compared to the same outcomes for all middle-school and high-school students in the control group.

The exposure period was two non-consecutive years, because of COVID-related school disruptions. In 2021, fewer than 20% of students were attending school in person with the majority hybrid (60-75%). By the 2022-school year, approximately 40-45% of students were attending school in person and an additional 50-57% were hybrid. Therefore the exposure period was approximately two years and included the 2019 and 2022 school years, but not the 2020 and 2021 school years. In other words, by the second semester of 2022, more than 95% of students were in person or hybrid.

The Performance evaluation included both quantitative and qualitative data sources. Staff from the Ministry of Education and Fomilenio II were interviewed in 2019, 2020 and 2022. The evaluators also interviewed implementing entity staff in 2019 and 2020. Lastly, school leaders were interviewed in 2022 to understand how schools were functioning post-COVID.

The impact evaluation relied on administrative, student-level data from the Ministry of Education. Student-level data included repetition, progression and dropout rates and test-score data. The project funded a new education management information system for the Ministry of Education. In 2017-2018, before the new system was functional, data came from the old systems. For the 2019-2022 school years, the evaluators relied on data from the new, MCC-funded information system, which improved student tracking.

2024-002-2994