Modernizing Vocational Education In Mongolia

Program Overview

MCC's $285 million Mongolia Compact (2008-2013) included the $49 million Vocational Education Project (VEP), which aimed to increase employment and income among unemployed and marginally employed Mongolians through improving the quality and relevance of vocational education. Its five activities worked together to reform TVET policies, create system-wide skills standards, design a competency-based training system, develop career guidance and labor market information systems, and improve learning environments by upgrading and modernizing infrastructure and equipment for 28 schools.

Download Mongolian translated Mongolia evaluation brief.

Evaluator Description

MCC commissioned Social Impact to conduct an independent final performance evaluation of the Vocational Education Project. Full report results and learning: https://data.mcc.gov/evaluations/index.php/catalog/223.

Key Findings

Quality of TVET Education

Nearly all stakeholders agree that the quality of Technical and Vocational Education and Training (TVET) has improved significantly as a result of the VEP. However, instructors, students, and graduates are more positive than members of the private sector.
Competency-based training (CBT) curriculums have been formally adopted, though practical application varies significantly from school to school.
Facilities and equipment have generally been well used, though funding for maintenance was an issue.

Relevance of TVET Education

Policy changes aimed to increase private sector engagement in TVET at the national and local levels. However, after the political transition of 2012, entities set up by the Compact to encourage such engagement were weakened.
Locally, TVET schools and private sector partners (PSPs) have increased the depth and breadth of their partnerships. However, collaboration is still far below where it needs to be.

Employment and Income

Results on employment rates and incomes of graduates are unclear, with differing data from national-level statistics and interviewees. Potential effects from the VEP are further obscured by macro-level factors such as the overall economy, differences in school quality, and national wage policies.

Evaluation Questions

The VEP performance evaluation was designed to assess the project's four key outcomes: improving TVET quality and relevance, and increasing graduate employment and income. Of the 30 evaluation questions covered in the final report, this brief highlights a sub-set of key questions, summarized as:

1 How has TVET quality changed? What is the status of, and how have CBT curriculums and new training equipment and facilities affected quality?
2 How has the relevancy of TVET education to employer needs changed? How is the private sector engaging in TVET and why? What effect did the Agency for TVET (ATVET), a new government agency created to support the sector, have on engagement?
3 How has the employment rate and income of TVET graduates changed, and why?

Detailed Findings

Quality of TVET Education

There is near unanimous agreement among stakeholders that the quality of TVET training has improved substantially and many attribute much of this change to the VEP. However, instructors, students, and graduates are more positive than members of the private sector. Nonetheless, not all the quality-focused sub-activities were sustainable; the National Learning Resource Center and Centers of Excellence, for example, were not successful in establishing a process to extend learning and collaboration across TVET schools.

Chart of perceptions of training quality — Perceptions of Training Quality

Activities such as establishing CBT curriculums and providing equipment and improved facilities were widely praised (despite some challenges). Based on the initial work of the VEP, and additional efforts of other donors, CBTs are becoming commonplace. Though more work is needed to ensure CBTs are used by all schools and to further unify curriculums across the sector, the sector now has a common language focused on competencies, and is united in its efforts to increase private sector involvement and practical training.

training classroom with equipment simulators in Mongolia. — Training Classroom with Equipment Simulators

Some equipment is no longer operational due to license problems and difficulties in obtaining spare parts. Where the VEP installed equipment and facilities were still operational, they appear well-used and maintained to the extent possible. However, concerns exist regarding long-term maintenance and replacement (including funding for maintenance) and the increasing age/obsolescence of the equipment. And, despite increases in the amount of equipment, classrooms still did not have enough equipment for every student.

Relevance of TVET Education

As with TVET quality, stakeholders share strong agreement that engagement between the private sector and the TVET sector has improved substantially, with schools and private sector partners interacting through multiple forms. However, many respondents also report that collaboration was almost nonexistent before the MCA project, so its current level is still far below where it needs to be.

Chart of perceptions about the level of private sector engagement — Perceptions about the Level of Private Sector Engagement

Larger companies have their own internal training programs and don't need to rely on TVET. Policies put in place to support more private sector-TVET partnerships, particularly those ensuring schools are able to retain funds for their own use, have not come to fruition, deterring such relationships. Additionally, after the political transition of 2012, two entities set up by the compact were weakened, further lessening the TVET sector's ability to engage with the private sector: the ATVET and the National Council for Vocational Education and Training (NCVET) lost substantial authority to carry out their mandates. A shift in understanding of the importance of partnership—for both TVET schools and potential business partners—has been the primary enabler for increased partnerships. Major challenges include a limited ability to contribute from smaller, local companies and a lack of engagement by larger businesses.

Employment and Income

Long-term effects of the VEP on graduate employment and incomes are unclear. They are obscured by the overall economic situation in Mongolia and a variety of other factors. National graduate data show employment is on the decline while interviewees report employment rates are rising. National graduate data on income is not available, but the impact evaluation of the VEP's equipment provision activity found students that selected into trades with improved equipment did not have significantly higher incomes than those that did not.

Chart of national employment rate — National Employment Rate (World Bank, 2018a)

MCC Learning

Understanding the linkage between inputs and expected outcomes: This evaluation reiterates the lesson learned in the impact evaluation of the equipment provision activity of this project: Training inputs are important, but more is required to ensure results are achieved (in this case, employment and income).
Despite good design and implementation, national policy reform may still fail: This project included relevant national policy reforms – particularly the establishment of the ATVET and NCVET. Political change reversed the policy in a way that was not predicted.
Incremental reform is a solid building block: Other donors have taken forward both CBT and the National Vocational Qualification Framework (NVQF) – both important tools for flexible, demand-driven TVET.
Private sector partnerships need to be deeply rooted and institutionalized: Partnerships generated through the compact didn't last. In order to be sustainable, they need to be more structurally integrated into the governance of the TVET system, especially at local training centers.
Private sector partnerships should confer tangible, sustained advantages on private partners: In addition, to be sustainable, partnerships must feature positive financial returns for a prospective partners as well as a manageable risk profile.

Evaluation Methods

This ex-post performance evaluation used a mixed-methods approach, combining 110 qualitative interviews with secondary quantitative data. Interviews were conducted with government representatives, former project staff, school administrators and instructors, students, and graduates. Secondary data included TVET school enrollment, graduation, and employment data, as well as data collected as part of another evaluation of this project. Qualitative data was coded and analyzed to identify thematic trends. Quantitative data was analyzed using descriptive statistics. The exposure period was 5 years from the end of the Compact, which is believed to be sufficient for the expected outcomes.

electricity circuit practice board — Electricity Circuit Practice Board

engineer repair practice equipment — Engineer Repair Practice Equipment

2019-002-2249