Improving Technical and Vocational Education in Guatemala

Program Overview

MCC’s $27.3 million Guatemala Threshold Program (2016-2021) funded the $20.6 million Education project which included the $4.2 million Technical and Vocational Education and Training (TVET) Activity. The activity aimed to provide quality youth educational opportunities that have relevance to the labor market through two institutional partners, the Guatemalan Ministry of Education and the National School of Agriculture. More broadly, the project focused on improving teacher professional development and mobilizing additional government resources that are needed to address binding constraints to economic growth.

Key Findings

Evaluation Questions

This final performance evaluation was designed to assess the implementation process and how TVET activities contributed to participants’ preparation to enter the labor market. Of the 13 evaluation questions covered in the final report, this brief highlights a sub-set of key questions, summarized as:

1. 1
   Did new TVET programs recruit enough participants? What gender differences exist in enrollment? To what extent to students demonstrate the skills required to receive certification?
2. 2
   To what extent do the program curricula align with labor market demands? How did the sectoral studies, round tables, and meetings influence the design and implementation of the TVET programs?
3. 3
   To what extent do teachers in the TVET system demonstrate new skill sets covered by the MCC-supported TVET training? To what extent do the Ministry of Education and the National School of Agriculture have the organizational capacity (e.g., human, institutional) to continue implementing and improving the TVET system?

Detailed Findings

TVET Recruitment, Enrollment, and Retention

High interest in Ministry of Education TVET programs facilitated support of and enrollment in the new careers. To meet student demand for the new careers, 11 of the 13 pilot centers opted to offer the new career options in addition to their current options. Many centers also opened additional sections to accommodate demand. However, approximately 30 percent of students dropped out after the first year. In focus groups, students in five of the six schools noted the program failed to meet their expectations due to insufficient equipment and infrastructure.

There were few changes in enrollment and completion by gender and no shift of girls into male-dominated sectors. Girls tended to enroll primarily in the food and beverage sector and boys in the software programming, transportation, and mechanics sectors. This trend continued throughout the program.

Relevance of TVET Education

The TVET activities were grounded in extensive research on labor supply and demand, the educational offer, and the interests of young people. The research included 56 roundtables with public and private sector stakeholders, which helped harmonize the needed competencies, course specific qualifications, and training modules. Six new TVET career paths (along with teacher guides and student manuals) were developed for the Ministry of Education’s programs in hospitality, transportation, and informational technology.

During the design phase of the National School of Agriculture (ENCA) TVET program, implementers engaged closely with stakeholders in the productive sector. They held several roundtables with representatives of each sector, including agriculture, forestry, and agro-industrial. 174 upper secondary teachers were trained in these occupational families through 200 hours of training. However, implementors failed to create a productive relationship with ENCA’s Board of Directors. When the Board was finally engaged to review the curricula, they did not agree with the proposed changes. While many key informants felt that the new curricula were appropriate to the Guatemalan context, the Board felt that it was too European-based and did not adopt or implement the curricula.

Barriers to Implementation

More than half of the Ministry of Education’s TVET pilot centers selected to participate in the programs lacked the infrastructure and equipment needed to implement them. Despite financial support from government entities, only four centers had a parent organization that could receive and manage the donations. Students in three of the six schools in the sample reported their centers did not have sufficient equipment or instructors and are therefore not receiving training.

Students who had labor training teachers noted that the teachers were less prepared than previous teachers and often taught more than one cohort at the same time, limiting their ability to deliver the planned number of hours. The shortage of teachers is related to poor teacher retention due to lack of stability in teaching positions. In 2019, schools did not have the administrative or budgetary structure to hire teachers with permanent contracts, so the Ministry used consultant contracts to meet immediate classroom needs. Because those contracts expired halfway through the training, only 20 of the 58 teachers who received the training update continued teaching during the 2020 school year.

Students also noted that they did not have enough opportunities for labor training practice, including internships. Barriers to practice are related to the lack of sufficient equipment, the low number of hours spent in the laboratory, and the lack of private-sector engagement with TVET institutions. Implementers supported the Ministry in presenting a proposal for strategic partnership with the productive sector, but they were not signed due to the start of the COVID-19 pandemic.

MCC Learning

• 
  Having institutional-level commitments rather than individual-level commitments to implement desired reforms may help secure more sustainable buy-in from stakeholders and partner institutions.
• 
  Programs should consider sustainability of funding for recurrent expenses and supplies.
• 
  Align level of training with the skill level of those who will be enrolling in TVET programs and have realistic expectations for the level of improvement that is possible.
• 
  Clearly define and develop strategy for measuring technical competencies to be acquired by students in a TVET program.

Evaluation Methods

Mathematica conducted a mixed-methods performance evaluation with two components. First, they conducted a fidelity of implementation study that (1) explored how implementation was conducted by implementers; (2) examined whether implementers adhered to the original program design, and how and why changes to the design were made by implementers; (3) helped contextualize short-term outcome results by describing the geographic, social, and policy environment in each of the departments as well as each TVET center; and (4) provided a deeper understanding of how the project interventions may contribute (or not) to short- and medium-term outcomes. The second component was an implementation study that used administrative data, interviews with key stakeholders, focus groups with students, and document reviews. The evaluation team also conducted an outcomes analysis that aimed to shed light on potential benefits from the investments in the TVET centers.

This evaluation faced several limitations:

1. The study is mostly qualitative in nature, and the results are not generalizable.
2. The study used a purposeful sample which is subject to sample and recall bias.
3. The research team was only able to interview ENCA administrators not any teachers or students.
4. The COVID pandemic affected the study in the following ways: (1) it delayed the interventions; (2) it delayed student focus groups, originally intended to be a cohort analysis; and (3) it required virtual data collection from teachers and school directors.

2024-002-2943