Automating Risk Assessment for Evaluation Planning

UNICEF needed a systematic way to assess risks across its work and steer evaluations toward the highest-risk areas. Existing approaches relied too heavily on manual review and fragmented expert input. To address this, UNICEF asked rowsquared to design a replicable, data-driven methodology to assess risk across its 2021–2025 Strategic Plan Goal Areas and guide the next Plan for Global Evaluation (PGE).

rowsquared worked with UNICEF’s headquarters in New York to develop this methodology, starting with the design of data pipelines and a structured data fabric to organise raw data into formats suitable for analysis. We used Natural Language Processing (NLP) techniques and large language models (LLMs) to extract, classify, and summarise information from a range of text-based sources, supporting both content analysis and topic mapping.

We also applied document parsing methods to extract tables, specific chapters and headings, and key metadata from UNICEF reports and strategy documents. To assess risk, we employed the Alkire-Foster method to construct a multidimensional index that aggregated risk across operational, financial, and delivery dimensions.

We designed a tool that provides a risk score for different areas of UNICEF’s work, which guides the Evaluation Office in planning global evaluations. Built to be reusable, UNICEF intends to apply the tool at regular intervals, ensuring an ongoing, systematic assessment of risk. It will help ensure prioritization of the most strategically relevant evaluation topics and rationalize resource allocations.