Purpose
The purpose of these guidelines is to (i) set forth the principles and procedures for implementing transparent, reproducible, and ethical data and documentation (TREDD) for data activities funded by the Millennium Challenge Corporation (MCC), and (ii) facilitate MCC’s observance of the general principles of the Federal Policy for the Protection of Human Subjects or “Common Rule[[The Common Rule was published in 1991 and revised in January 2017; it is codified in separate regulations by 15 Federal departments and agencies (published here - https://www.hhs.gov/ohrp/regulations-and-policy/regulations/finalized-revisions-common-rule/index.html.) The Common Rule’s focus is safeguarding the rights and welfare of human subjects involved in federally funded research. Although MCC is not a signatory of the Common Rule, MCC recognizes and acknowledges the ethical and privacy implications involved in research involving human subjects.]]”.For MCC:
- Transparent data and documentation refers to the set of practices and tools used to disclose all methods, findings, and data for a data activity. For independent evaluations, this is needed to achieve objectives of evaluator independence, computational reproducibility of analysis, and maximizing broad usability of data for learning.
- Reproducible data and documentation refers to facilitating access to code and data required for independent analysts (including MCC staff, staff of the accountable entity designated by a country partner to oversee an MCC-funded program (MCA), and other researchers) to reproduce analysis from shared data with minimal effort. For independent evaluations, this is needed to allow external assessment and understanding of the analysis.
- Ethical data and documentation refers to practices that follow the ethical principles of beneficence, respect for persons, and justice, with particular emphasis on informed consent, independent review (institutional or by other review board), and proper data de-identification and management.
Scope
These guidelines apply to all data and documentation produced by independent evaluators contracted by MCC’s Department of Policy and Evaluation (DPE) (MCC staff) and done in coordination with the country government partners receiving MCC assistance in the form of compacts or threshold program grant agreements (country partners). As needed, these guidelines may be referenced for other data activities performed by other contractors (evaluators, data collection firms, economic analysis firms, due diligence firms hired by MCC or MCA to conduct MCC-related data collection activities).TREDD Summary
Staff and contractors need to consider TREDD sharing issues throughout the life cycle of data activities: (i) Design, (ii) Collection and/or Extraction, (iii) Documentation Sharing, (iv) Storage and Transfer, (v) Analysis, and (vi) Data Sharing. Therefore, these guidelines are organized by life cycle stages. These guidelines conclude with a section on managing disclosure risks. A glossary of key terms is provided in Section 11. Although discussed in more detail throughout these guidelines, Table 1 summarizes the TREDD practices MCC requires for each life cycle stage[[These tasks are also built into the standard scope of work and personnel qualifications for MCC-funded contractors.]]:| Design | Primary data handlers must complete training on protection of human subjects. |
| Contractors must have the study documentation (research protocol, informed consent, etc.) reviewed by a US Department of Health and Human Services (HHS)-registered Institutional Review Board (IRB) AND in accordance with any local requirements. | |
| The informed consent must inform data providers who will have access to what data (identifiable, de-identified) and for what purpose. | |
| United States (US) and European Union (EU) citizens must be excluded from MCC-funded surveys. | |
| Collection and/or Extraction | Contractors must facilitate and document informed consent with data providers. |
| If secondary data owned by another entity is required for analysis, MCC staff and contractors should work toward obtaining Data Sharing Agreements with the data owners to enable public and/or restricted access to the data on the MCC Evaluation Catalog or similar platform. | |
| Documentation Sharing | Contractors must use standard reporting templates unless otherwise agreed with MCC. |
| MCC staff must publish relevant documentation of all independent evaluations on the MCC Evaluation Catalog (https://data.mcc.gov/evaluations/index.php/catalog).[[Other public platforms may be identified for data activities outside the principal scope of these guidelines.]] | |
| Storage and Transfer | MCC staff and contractors must have specific practices in place to protect data confidentiality and data integrity during storage. This includes: encrypting data files; employing password protection on data systems and data encryption; and requiring relevant stakeholders to sign non-disclosure agreements. |
| Data handlers must use secure file transfer to transmit digital data files with personally identifiable information (PII). | |
| Analysis | Contractors should establish a reproducible workflow to facilitate computational reproducibility of analysis to the extent feasible. Contractors may consider de-identifying the data prior to analysis to ensure a closer link between the data that produces the analysis and the data that can be shared. |
| Data Sharing | As feasible and appropriate, Contractors will prepare data and code for public-use and/or restricted-access in a way that adheres to promises of confidentiality made during the informed consent process. |
| MCC staff must publish data and code from independent evaluations on the MCC Evaluation Catalog (as feasible). | |
| Contractors must produce a Transparency Statement in which they confirm (i) public-use and/or restricted-access data and code will reproduce analysis in Interim/Final Results Report, or (ii) justifications for why public-use and/or restricted-access data and code do not reproduce analysis in Interim/Final Results Report. |