Improving Non-Formal Skills in Northern El Salvador

Program Overview

MCC’s $449.5 million El Salvador Compact (2007-2012) funded training in skills such as cooking, tailoring, and electric installation, as well as job placement services through the $5 million Non-Formal Skills Development Sub-Activity. It was based on the theory that an increase in skilled workers plus connection to the market would increase employment of participants and therefore increase overall household income.

Key Findings

Evaluation Questions

This final performance evaluation was designed to answer the following questions:

1. 1
   What were course participants’ demographic and socioeconomic characteristics?
2. 2
   How was the sub-activity planned and implemented? What were implementation challenges and facilitators?
3. 3
   How did participants’ labor market outcomes and income change from before beginning non-formal skills courses to approximately one year after completing the courses?

Detailed Findings

Surveys were conducted one year after each training group completed their courses, with eight survey rounds overall. The findings presented here are for the last groups who completed training (survey rounds seven and eight), with some discussion of all rounds pooled together, due to the limitations of the survey instrument used in early survey rounds.

Participants

True to its target of vulnerable populations, the median trainee of the program was 30 years old, female, rural, with only eight years of formal education.

Implementation

Overall, implementation was successful and exceeded MCC’s targets for enrollment and completion. From 2009 to 2012, a total of 11,310 individuals completed the intervention’s short-term courses, exceeding MCC’s target of 6,888 course graduates. Furthermore, the courses had a 95 percent completion rate, also exceeding MCC’s target rate of 82 percent.

Throughout the sub-activity’s implementation period, notable challenges included: (1) a lack of suitable organizations to provide new training courses, (2) stringent contracting requirements that required bids from at least three potential implementing firms, (3) a lack of implementer experience in the Northern Zone, and (4) a lack of implementer capacity regarding job market placement and self-employment. To some extent, these challenges were mitigated by the following implementation facilitators: (1) close supervision by sub-activity managers, (2) strong interest and commitment from participants and local authorities, and (3) the use of starter kits distributed at the outset of courses.

Courses were offered at participants’ request. Participant demand did not necessarily relate to labor demand with the exception of tailoring school uniforms, for which the government began to issue more contracts during the implementation period.

Employment

While there were documented self-reported improvements to employment and income after participating in training, there was no evidence that these short courses systematically improved the labor market outcomes of participants compared to non-participants.

Participants’ self-reported employment rates grew by 14 percentage points in survey rounds seven and eight. When all eight rounds of data were pooled together, employment rates increased by 26 percentage points. However, these estimates are likely biased upward due to the limitations of the survey instrument used in early survey rounds, as noted above.

Although most participants still typically only worked part-time, the average number of hours worked per week increased significantly one year after the training compared to baseline in all survey rounds. Additionally, self-employment increased more among women, and salaried employment increased more among men one year after participation in the non-formal skills training.

Income

Total annual self-reported income did not change significantly when data was pooled for rounds seven and eight. Round seven participants experienced an increase in total self-reported income of $254, whereas round eight participants experienced no statistically significant change in total self-reported income. When all rounds with income data are pooled, on average, total net annual self-reported income increased by $298 one year after completing a course as compared to baseline. Here, too, due to the limitations of the survey instrument used in early survey rounds, it is likely that these estimates are biased upward. As a reference, according to the National Household Survey, annual household income in the Northern Zone was $4,572, $4,644, and $4,848 for 2010, 2011, and 2012, respectively. Therefore, despite the possible upward bias for both employment and income, it seems that the sub-activity did contribute to higher employment and income independent of regional trends in the Northern Zone.

The evidence presented here is consistent with the possibility that the sub-activity’s training activities contributed to improved labor market outcomes for participants. These changes were of a great enough magnitude that they may also have translated into improvements in participants’ well-being.

Job Placement

A small group of non-formal skills training participants also participated in the PILAS program, which provided job search and placement assistance for some trainees and business development for others (between 10 and 26 percent in each round, with only one participant in round 8). Trainees participating in PILAS were as likely as those not participating in PILAS to find any type of employment, but PILAS trainees were more likely to find salaried employment. Both PILAS participants and non-participants self-reported increases in total income when all rounds were pooled, but PILAS participants reported larger income increases than non-participants in secondary forms of income.

A major caveat of this comparison is that PILAS participants may be more or less motivated and skilled than non-participants, on average, and these differences in pre-existing characteristics may explain some differences in outcomes.

MCC Learning

• 
  Project design determines feasibility of evaluation type and methodology. The design of the non-formal training program was demand-driven with no excess demand (meaning that no community that requested training would be denied). Since the program was able to cover every community, it was impossible to identify a strong comparison group up-front.
• 
  Participant feedback may be useful for implementers in identifying strategies to improve course effectiveness. For instance, the majority of survey respondents in the last four rounds indicated that they would have benefited from more time for training and more opportunities to practice what they were learning. Some of these participants may have satisfied their desire for more time by taking a second related course. But in other cases, it may have been beneficial to extend the courses by more days or hours, or to introduce an internship or on-the-job component.

Evaluation Methods

The evaluation used a pre-post methodology. Participants were surveyed approximately one year after their first course ended and asked retrospective questions about their situation before the training course. Eight rounds of surveys were conducted between 2009 and 2013.

The full evaluation report focuses on the results for the last four rounds of training participant surveys. The sample consists of 1,160 individuals who completed their first non-formal skills training course between January 2011 and March 2012 and who completed a follow-up interview in rounds five, six, seven, or eight.

The instrument was modified before round seven to make questions about pre-training employment and income identical to questions about post-training employment and income, given difficulties interpreting data from prior survey rounds. As a result of these changes, employment and income estimates for rounds seven and eight are likely more accurate than estimates for previous rounds.