Program Overview
MCC’s $140 million Georgia Compact II (2014-2019) funded the $16 million Industry-Led Skills and Workforce Development (ISWD) Project, which aimed to improve the alignment between the skills of Technical and Vocational Education and Training (TVET) graduates and the skills demanded by the labor market. It provided grants to establish new or improved courses reflecting industry demand; awarded smaller grants to document and disseminate best practices; provided technical assistance to the government related to sector policy; and held annual TVET conferences.Download Georgian translated evaluation brief.
Evaluator Description
MCC commissioned Mathematica to conduct an independent interim performance evaluation of the ISWD Project. Full report results and learning: https://data.mcc.gov/evaluations/index.php/catalog/195.
Key Findings
New Courses Successfully Established
- Through the Program Improvement Competitive Grants (PICG) Activity, the ISWD Project established 51 new or improved TVET courses in Georgia.
- These courses were established through close cooperation between the TVET provider and private sector partners, who provided valuable knowledge and material support.
Course Participation and Early Perceptions
- Total enrollment in supported courses exceeded the compact’s target.
- Trainees and teachers had positive first impressions of the new courses.
- Trainees and employers were optimistic about trainees’ labor market prospects.
Other Project Activities
- The small grants supported dissemination of best practices in ways that could be replicated by others.
- The technical assistance component delivered support for a wide range of policy reforms, but more work will have to be done after the compact.
- The Annual TVET Conference, which aimed to strengthen ties between TVET providers and private sector firms, was well-attended and well-received by stakeholders.
Evaluation Questions
This interim performance evaluation focused on assessing program implementation, early results, and expectations for the future. It will contribute to answering the following questions:- 1 How did the implemented Program Improvement Competitive Grants (PICG) courses compare with the original grant proposals, and what were the reasons for any deviations?
- 2 Did trainees enroll in PICG courses and graduate from them at targeted levels?
- 3 What were the labor market outcomes (employment and wages) for graduates from PICG courses?
- 4 What are TVET providers’ perceptions of the best practices identified and disseminated by the program, to what extent have they adopted them, and what are the main barriers to doing so?
- 5 To what extent have the Ministry of Education and Science (MES) and its agencies adopted the policy reforms supported by the program, and what have been the main challenges in doing so?
- 6 How and to what extent has the annual TVET conference influenced providers, employers, the MES, and other TVET sector stakeholders?
Detailed Findings
New Courses Successfully Established
The PICG Activity successfully established 38 new or improved degree courses and 13 new certificate courses in areas such as information technology, agriculture and veterinary services, aquaculture, maritime operations, tourism, railways, and aviation. These courses were established through close cooperation between the grantees and private sector partners, who provided valuable knowledge and material support. Other notable features of implementation included a rigorous proposal development and selection process, strong grant management systems, and solid working relationships and open communication between key players.Course Participation and Early Perceptions
Other Project Activities
The Strengthening TVET Provider Practice (STPP) Component of the Strengthening Sector Policy and Provider Practice Activity provided small grants on a competitive basis to identify, document, and disseminate innovative best practices in the TVET sector, such as the use of television and social media to conduct outreach. These best-practice grants supported a wide range of practices. The potential for replicability is highest for practices that are aligned with the policy reform efforts of the MES.The Strengthening TVET Sector Policy (STSP) Component provided technical assistance related to TVET sector policy. STSP aimed to promote increased business engagement in TVET, improve and promote the quality and attractiveness of TVET, and support the enhancement of learning and qualifications opportunities for adults. The Component was flexible, responsive to MES needs, and policy-relevant. However, these reforms are long-term efforts and more work will be necessary to ensure countrywide adoption of those policies post-compact.
Information booth for Georgian Technical University at the 2017 TVET conference
MCC Learning
- The grant application process was rigorous and well-managed, resulting in high quality grant recipients. MCC should consider the importance of the grant application process and grant management capacity when designing and implementing grant facility projects in the future.
- Although the program sought to increase female participation in STEM fields, only 14 percent of all trainees in the PICG-supported courses were female. In order to recruit more women into traditionally male fields, future programing should aim to better understand the underlying causes of low female participation and target specific interventions to address those causes.
- Overall, initial impressions of the PICG-supported courses were positive, however both instructors and trainees noted a lack of instructional materials available in Georgian in technical subjects. Future TVET projects should strive to provide resources for written materials in the local language if not already available.
Training equipment for the courses conducted by Georgian Railways
Evaluation Methods
This mixed-methods interim performance evaluation was conducted towards the end of the compact, when program implementation was almost complete. It drew largely on a quantitative baseline survey of trainees, as well as qualitative data collected through interviews and focus groups.Trainee Survey Data
Between mid-2018 and early-2019, a local data collection firm conducted a baseline survey with trainees enrolled in 41 PICG courses (all degree courses and three certificate courses). The baseline survey included a sample of 1,148 enrollees who provided information about their demographic characteristics, employment history, experience and satisfaction with PICG courses, and expectations for future employment and earnings. For the final study, Mathematica will administer an endline survey to these trainees one year after they graduate.
Qualitative Data
In late 2018, a local data collection firm conducted semi-structured interviews or focus groups with grantees, PICG-affiliated employers, PICG course teachers, and PICG trainees. The evaluation team also conducted high-level stakeholder interviews with organizations that were involved in program implementation and/or in the TVET sector more broadly to learn about challenges and successes during implementation, and the prospects for sustainability.
Next Steps
A final study covering the full set of evaluation questions is underway and results will be available in 2022.2019-002-2247