Addressing Barriers to Education in Namibia

Program Overview

MCC’s $304.5 million Namibia Compact (2009-2014) funded the $145 million Education Project which included the Improving the Quality of General Education Activity (Education Activity) and the Improving Access to and Management of Textbooks Activity (Textbook Activity), which aimed to improve the quality of the Namibian workforce by enhancing the equity and effectiveness of basic education. The Education Activity rehabilitated nearly 50 schools and funded training for school personnel. The Textbook Activity distributed textbooks to all Namibian schools and supported textbook procurement and management reforms.

Key Findings

Evaluation Questions

This final performance evaluation was designed to answer eight evaluation questions, a subset of which are listed below:

1. 1
   Do improvements in school infrastructure (including teacher housing) attract more qualified teachers?
2. 2
   Does improved infrastructure have an impact on learner flow (such as enrollment)?
3. 3
   Did the improved infrastructure or provision of additional textbooks improve learner outcomes in English, science, and mathematics?
4. 4
   Have investments in reforms to systems and protocols for textbook procurement and distribution resulted in improved efficiency?

Detailed Findings

School Infrastructure Improvement

The compact constructed, renovated, and equipped 49 schools in Namibia. While school personnel appreciated the improved facilities, they often did not meet their needs. For example, junior and senior secondary schools received the same equipment and materials despite differences in curricula. Additionally, maintenance of MCA-supplied computers has been unaffordable post-compact, which has negatively impacted the use of the computers in the classroom.

Teacher housing was constructed to attract more qualified teachers in remote areas. The average proportion of qualified staff at compact-supported schools increased from 22 percent in 2009 to 93 percent in 2016, compared to an increase in the average across the regions of 79 percent to 82 percent in the same period. However, the most significant increase at the compact-supported schools took place between 2009 and 2012, before many of the compact activities were implemented. As a result, the evaluator found no evidence that the improved school facilities or the teacher housing caused the increase in the proportion of qualified teachers at the refurbished schools.

Improved facilities also aimed to increase student enrollment. While total enrollment increased at compact-supported schools, actual enrollment did not meet the projections in the corresponding cost-benefit analysis (CBA). The closeout CBA projected a 26.5 percent increase in enrollment at the junior and senior secondary levels, and no change at the primary level. Actual enrollment between 2009 and 2016 decreased by 5 percent at the primary level, increased by 6.4 percent at the junior secondary level, and increased by 67.7 percent at the senior secondary level. Senior secondary has the fewest students enrolled so despite the large percentage increase at that level, total enrollment across all levels underperformed the closeout CBA projections by over 2,500 students (i.e., 26,651 students were enrolled in 2016 compared to a CBA projection of 29,210 for 2016).

Improved Access to Learning Resources

It is likely the textbooks procured by the compact improved the textbook-to-pupil ratio (TPR) in Namibia. However, the Ministry of Education, Arts and Culture (MEAC) did not have records of the number of textbooks at each school, so the evaluation could not verify the actual TPR. Stakeholders believe the post-compact TPR likely suffered due to the introduction of a revised curriculum in 2015 and government austerity measures in 2016, which restricted funding to purchase new textbooks aligned with the new curriculum.

Textbook training was structured around the “train the trainer” method of training, also known as the “cascade” approach. While the evaluator considered the training of master trainers to be “largely successful,” the textbook usage trainings seemed to have been replicated within some regions, while two other types of training either experienced limited replication or limited effectiveness of the replication.

Finally, the Supply Chain Management Unit, which was established to plan, coordinate, supervise, and monitor textbook procurement, is restricted in the amount of time they can spend monitoring the effectiveness and efficiencies of the textbook ordering processes for which they are ultimately responsible. Furthermore, MEAC never implemented the Learning Support Materials Management Information System that was part of a textbook planning and data management framework intended to improve textbook procurement and distribution.

Student Learning Outcomes

Compact-supported schools exceeded year five targets for the Grade 5 standardized tests but did not meet or exceed targets in any of the subjects in the Grade 10 and 12 leaving exams. Compared to the national average in 2015/16, compact-supported schools performed better in four exams, essentially on par in three, and worse in one. The year-on-year trends are also mixed, with a few exams showing consistently-decreasing scores, while the remaining scores increased between 2009 and 2016, though some trends are more consistently positive than others.

According to the evaluator, the evidence does not suggest that the compact interventions, on their own, positively impacted learner performance on the 10th or 12th grade exams, and cannot determine with certainty whether the compact-supported facilities influenced the positive performance on the 5th grade exam. Furthermore, given that not all test scores improved at the national level, the evaluator questioned whether or how the textbooks could have helped improve performance on some exams but not others.

Finally, sex-disaggregation of average test scores shows that girls at compact-supported schools consistently outperformed boys in English at every level of testing. In Grade 5, girls also performed better in math. However, beginning in Grade 7, girls’ and boys’ performance was essentially even in both math and science. And by junior secondary school, boys performed better than girls in math and science, and this trend continued into senior secondary school.

MCC Learning

• 
  Without the proper momentum and capacity, a “cascade approach” to training educators/trainers risks becoming a “trickle approach.” Considering resource limitations, it is likely that MCC programs might propose similar models in the future. It is therefore recommended to cautiously explore such implementation models, and where relevant, ensure the training of trainers happens early enough in the implementation period to support early replication and help establish enough momentum.
• 
  Unreliable financing of school operations and maintenance can affect the use of compact-funded resources and results. Budget uncertainty can promote sub-optimal hoarding and/or rationing behaviors. MCC cannot fully influence resource allocation after the conclusion of the compact period. However, MCC can mitigate this risk during implementation by clarifying the incremental costs of educating students and supporting government prioritization and resource allocation in a way that reduces uncertainty, such that compact-funded resources can be used and provide benefits as intended.
• 
  Evaluations beginning in the post-compact period require more complete documentation. For future evaluations commissioned post-compact, it will be important for M&E leads to work with government partners to gather relevant documents well in advance of compact closure. MCC Country Teams should also provide more rigorous guidance about how MCAs store documents in their archives, document who will manage those archives, and how one might gain access. Without that information, the requirement to maintain such an archive risks being much less meaningful than intended.

Evaluation Methods

This final performance evaluation was primarily qualitative, drawing on the review of written reports, observations, key informant interviews, focus group discussions, and was complemented by descriptive analysis of the relationships between project investments and quantitative learner outcomes. The evaluation used administrative data from 2012 to 2016 and primary data that were collected in April 2018. The exposure period is approximately 2-3.5 years. The following data sources were used:

• Data from the Education Management Information Systems and the Directorate of National Examinations and Assessment, including exam results, administrative data, and standardized test results for the 49 compact-supported schools, along with regional and national results.
• Document review of MCC, MCA, and consultant reports.
• Primary data collected at 12 schools (six compact-supported schools and six schools that did not receive support), which included consultant observations, structured interviews with head teachers, focus groups with select school staff, focus groups with senior students, interviews with textbook relevant respondents, and end users.
• Interviews with Regional Directors in four regions; focus group discussions with key stakeholders from “circuits” (i.e., groups of schools within a region) including the circuit inspector, principals, and advisory teachers; and interviews with other stakeholders.