For independent evaluations, as soon as an Evaluation Design Report is cleared by MCC staff, MCC staff and the contractor will begin documentation sharing. As per the Standard SOW Section F.3 Deliverables, a sub-set of study documentation will be posted on the MCC Evaluation Catalog and must be Section 508[[In 1998, Congress amended the Rehabilitation Act of 1973 to require Federal agencies to make their electronic and information technology (EIT) accessible to people with disabilities. The law (29 U.S.C. § 794 (d)) applies to all Federal agencies when they develop, procure, maintain, or use electronic and information technology. Under Section 508 of the Act, agencies must give disabled employees and members of the public access to information that is comparable to access available to others. The United States Access Board discusses the Section 508 law and its responsibility for developing accessibility standards for EIT to incorporate into regulations that govern Federal procurement practices. More information is available online at https://www.section508.gov/.]] compliant (https://www.section508.gov/). Table 3 summarizes the required documentation and format for study documentation that must be made publicly available.
| Document | Requested Format | Description |
|---|---|---|
| Metadata File(Annex 2) | Nesstar file;PDF for viewing purposes | The contractor should prepare the metadata file for the public evaluation catalog entry. The metadata can be updated/revised as necessary over the course of the evaluation. Contractors should not attach any data sets or related documents under the “other materials” or “external resources” sections. Data is reviewed, cleared, and posted separately as per the DRB review process detailed in later sections.Please note, MCC reviews the PDF export of the Nesstar file and recommends contractors review this PDF export prior to submitting to MCC. |
| Evaluation Design Report, Baseline Report (as applicable), Interim Report(s) (as applicable), Final Report, any relevant presentation materials | Word or searchable PDF | These documents (deliverables required under MCC contracts) provide necessary design and analytical information. Contractors should ensure that all public use documents/reports have been reviewed and edited to remove any references, such as geographic locations, that may threaten or undo data de-identification efforts. MCC requires contractors to update Evaluation Design Reports (EDRs) as needed over the life of the evaluation. Any revisions should be documented in the EDR so that course corrections/revisions are clearly documented. In the event that one contractor inherits an evaluation from another, the original contractor’s EDR will be posted on the Evaluation Catalog along with the new contractor’s EDR. |
| Cost-Benefit Analysis (CBA) Model | Excel or other relevant format | |
| Informed Consent Statement | Word, searchable PDF | The IRB approved informed consent statement should be published, either independently or as part of the questionnaire(s). |
| Questionnaires (English and local language) and related documentation | Original editable source and searchable PDF | All survey questionnaires – baseline, interim, final - should be shared as the original editable source file. Contractors may also submit a searchable PDF. Related documentation may also include sampling strategy, field operations and interviewer manuals when needed for complete documentation of survey protocols. Any translation requirements should follow the contractor scope of work. For qualitative data, this documentation should include the interview guide(s) and any other study materials necessary for understanding how the data was generated and analyzed (as feasible). |
Contractors may be required to submit an ‘internal only’ version of a document, as well as a ‘public-use’ version of the document in order to mitigate the public release of linkage documentation that could support future re-identification of the publicly available data. For example, if the Design Report contains geographic identifiers that may enable future re-identification of the data provider(s), that information may be included in an internal-only version but must be removed from the public document.
Registries
In addition to the requirement to publish documentation and data on the MCC Evaluation Catalog, contractors may also choose to register the data activity on other study registries:
| Registry | Notes |
|---|---|
| AEA Registry (socialscienceregistry.org) |
Registry for randomized control trials (RCTs) in economics |
| Clinical Trials or ICTRP (clinicaltrial.gov) |
Registry for randomized control trials (RCTs) in health-related fields |
| 3ie (ridie.org) |
Registry for randomized control trials (RCTs) and quasi-experimental designs in development economics/program evaluation |
| EGAP (egap.org) |
Registry for experiments and observational studies in governance and politics |
| OSF (osf.io) |
Registry with multiple formats: short, long, structured, and open ended for any method, across social sciences |
As feasible, any additional registries for independent evaluations should be directed back to the MCC Evaluation Catalog as the centralized source for all documentation and data associated with the evaluation.
Contractor Reporting Guidelines
Reporting guidelines are a standardized procedure to report on study design, implementation, analysis, and interpretation of findings. For MCC, the evidence generated by any single independent evaluation is intended to contribute to a body of knowledge – such as MCC’s Principles into Practice (https://www.mcc.gov/our-impact/principles-into-practice) papers, systematic reviews, and other knowledge products. To facilitate this, MCC provides contractors with templates for reporting requirements for Design, Baseline, and Interim/Final Reports (Annex 3). Any deviations from the standard report templates should be discussed and agreed between MCC staff and the contractor prior to developing the report. The goal is for MCC-funded evaluations – and other data activities - to be accessible for broader learning and systematic reviews, and to serve as inputs to future cost-benefit analysis models.