Making IFAD Impact Analysis Reproducible

To evaluate the effectiveness of its development projects, the Research and Impact Assessment Division (RIA) of IFAD has been conducting quasi-experimental and experimental impact assessments for years. RIA sought to ensure that the results of these assessments were computationally reproducible—so that results could be generated directly from the underlying data and code. rowsquared was hired to assess and improve the reproducibility of both older and newly completed IFAD impact evaluations.

For a set of older IFAD11 evaluations, rowsquared reviewed the available Stata code and project folders, re-ran the analyses, and compared outputs against official reports and policy briefs. We reverse-engineered undocumented workflows. Documentation was often incomplete, and original analysts were no longer available for clarification. Managing the de-identification of sensitive data added a layer of care and technical handling. Each analysis received a structured reproducibility review and a performance score.

For the more recent IFAD12 assessments, rowsquared conducted full reproducibility checks, restructured analysis folders, de-identified data, and ensured push-button replication functionality while documenting everything for future users. We delivered fully cleaned, documented, and standardized analysis folders for three IFAD12 evaluations, push-button replication (PBR) packages, and feedback and documentation to guide reproducibility improvements.

The project established a foundation for improved analytical workflows and documentation standards by ensuring IFAD evaluation results could be independently reproduced.