Evaluation goes digital: KfW presents 16th Evaluation Report

The 16th edition of KfW's biennial Evaluation Report is now available in tandem - in print and, for the first time, also in an extended online version. Under the motto: "Evaluate – Measure - Learn", Professor Jochen Kluve's team focuses on the effects of Financial Cooperation projects coupled with increasing progress in digitalisation. The KfW Development Bank projects evaluated in this way over the past two years achieved a predominantly successful rating. At the same time, lessons can be learned from the results for the fight against the Corona pandemic.

The view from above reveals it: Where are forests being cut down, where are protected areas halting deforestation? Which measures against illegal logging have been successful? High-resolution satellite images are now routinely used for evaluation. Together with the French development bank Agence Française de Développement (AFD), for example, the evaluation department has developed a new initiative to use freely available satellite data for planning, monitoring and evaluation. "We are seeing a significant digitalisation push that is increasingly changing evaluation work and enabling additional perspectives and analyses as well as even more reliable learning," says Christiane Laibach, who is responsible for Financial Cooperation on KfW Group's Executive Board. The increasing spread of mobile phones worldwide is also contributing to the digitalisation of development cooperation: It is now easier to collect data directly from the target groups. Furthermore, automation processes make it easier and faster to analyse the data.

"In addition to strong digitalisation, we are increasingly applying methods of rigorous impact evaluation in our work. The initiators of these methods were awarded the Nobel Prize in Economics in 2019, and their methods allow precise measurement of the effects of a project. We implement them in selected FC projects from the start of the project, thus complementing the comprehensive ex-post evaluations," says Professor Kluve, Head of the Evaluation Department at KfW.

An example illustrates this: In Burkina Faso, cash transfers are being provided to mothers to ensure food security for their small or as yet unborn children. This support is very important for the development of the children - the first 1,000 days can decide their entire lives. Mothers of children over the age of two do not receive these payments. The evaluation deals with the question: What level of transfer achieves the highest impact here, and what nutritional level can be reached by the children within three years? These findings help to improve the current project and also future projects.

The evaluation team led by Professor Jochen Kluve examined a total of 171 projects, 86 percent of which were rated as successful. This rating corresponds to a rating of "3 or better" in a system similar to school grades used by the OECD-DAC, the Development Cooperation Committee of the Organisation for Economic Cooperation and Development.

KfW projects in the financial sector scored best. This is because financial system development projects presuppose a stable environment. Conversely, the favourable framework conditions contribute to the high success of the projects, which received an average rating of "good" (2.3). On average, water supply projects were rated the worst (3.1). In some of these projects, the involvement of the local population was not successful, which is an important factor.

There are also clear differences in success rates by region: More than 90 percent of the projects in Asia/Oceania were rated "successful". This region thus scored best (average 2.5), while Sub-Saharan Africa did less well (3.1). One of the main reasons for the comparatively poor performance of the Sub-Saharan Africa region is the context of extreme poverty and structurally weak partners: implementing projects successfully is more challenging here.

Lessons from the evaluation

The evaluation department, which is independent of KfW's project work, formulates several lessons from its evaluations: Programmes in the health sector show that combating epidemics is successful when they pursue cross-border approaches. After all, diseases do not stop at borders, and people are becoming more and more mobile. A regional approach to HIV/AIDS prevention in Central African countries has been particularly successful.

As to the so-called reform-based approaches (policy-based lending), these are increasingly being integrated into projects.

Here, the agreed funding is only disbursed if the government of the partner country implements policy reforms step by step. The individual steps are also jointly agreed and serve as "triggers" for the disbursement of funding tranches. In their report, the evaluators emphasise that this type of project requires a high level of staff commitment and continuous presence on the ground in order to maintain a constant dialogue with the partner government. If this is taken into account, reform-oriented approaches are very successful.

Fragile contexts do not affect success

In previous years, it was feared that the increase in fragile contexts would impair the effectiveness of projects. However, the evaluation could not confirm this. The increasing number of projects in fragile contexts was reflected in the fact that they made up a higher proportion of the evaluation sample than in the past. But they did not perform worse than the average. 68 projects in fragile contexts were examined. Examples from Yemen and Afghanistan show that the success of education measures increases when communities are intensively involved. In Afghanistan, for example, people decided locally on the reopening of schools that had been closed due to conflicts: in fact, they were best able to assess the security situation.