Every year a representative random sample of projects is selected for evaluation, which the FC Evaluation Unit then assesses on the basis of OECD-DAC criteria. The results of these evaluations form the basis for calculating the success rate, in other words, statistically estimating the percentage of successful FC projects.
Every year, KfW Development Bank’s Evaluation Unit takes a random sample (stratified by sector) from the basic population of completed measures. An ex post evaluation is conducted for all projects in the random sample. This method ensures that the evaluations are statistically representative and informative for the FC portfolio and maintains independence in the selection of projects to be evaluated. In 2019/2020, a total of 149 projects were randomly selected from the basic population of 292 projects.
Further projects can be evaluated, even if they are not in the random sample, for example because they are similar in terms of location or time to projects in the random sample or if requested by operational departments. In the calendar years 2019 and 2020, there were 42 such “pooled” projects. Together with 129 evaluations from previous random samples, 171 projects were thus evaluated in the years 2019 and 2020.
Only projects that are part of the random sample are used to estimate the success rate (unbiased estimate). The success rate is defined as the percentage of projects that received an overall rating of 3 or above. Overall, 63 projects have been included in the success rate calculation so far for the basic population for 2019/2020.
Note that success rates are estimates and therefore always subject to statistical uncertainty. The confidence interval provides information about the accuracy of the estimate.
For example, the success rates from past two-year periods since the introduction of the random-sample approach in 2007/2008 are not statistically different from one another. Despite the average success rates fluctuating between 77% and 87%, the confidence intervals for the success rates overlap in the region of 81% – it is therefore possible and seems likely that the underlying success rates of all FC projects are within this overlapping range.
To take into account the fact that individual evaluation findings differ in their significance, the overall rating of a project is not a simple average of the ratings for the individual OECD-DAC criteria (1–6 or 1–4 for sustainability). Particularly serious deficiencies in individual sub-criteria should not be offset by positive results in others. Hence, poor results in the OECD-DAC criteria of effectiveness, impact and sustainability generally prevent a project from being classified as successful overall. The underlying idea is that projects which do not achieve their objectives, or whose impacts are not sustainable (in the sense of being long-lasting), do not deserve to be evaluated as successful.
