Sample size and grade distribution – Results 2021-2022

1. Graph: Development of evaluation portfolio

This graph shows the number of the projects within the entirety (i.e, all projects ready for evaluation) for each of the two-year intervals since 2007. The size of this entirety varies over the years, with a low point of 206 projects for the years 2011-2012 and a maximum of 328 for the years 2015 and 2016. The graph also shows the size of the representative sample, which is usually approximates half of the entirety. The third data row shows the number of projects within each sample that have been evaluated to this date. The values for each two-year period are:

2007-2008: 224 projects, thereof 117 projects included in the representative sample and thereof 116 projects evaluated to this date. 

2009-2010: 212 projects, thereof 111 projects included in the representative sample and thereof 110 projects evaluated to this date.

2011-2012: 206 projects, thereof 112 projects included in the representative sample and thereofr 111 projects evaluated to this date.

2013-2014: 256 projects, thereof 135 projects included in the representative sample and thereof 133 projects evaluated to this date.

2015-2016: 328 projects, thereof 166 projects included in the representative sample and thereof 164 projects evaluated to this date.

2017-2018: 304 projects, thereof 159 projects included in the representative sample and thereof 154 projects evaluated to this date.

2019-2020: 285 projects, thereof 136 projects included in the representative sample and thereof 111 projects evaluated to this date.

2021-2022: 236 projects, thereof 124 projects included in the representative sample and thereof 55 projects evaluated to this date.

2. Graph: Distribution of ratings over time (2007-2022)

This graph shows the distribution of the total number of evaluated projects to the total ratings (from 1 to 6) achieved. All projects with a rating from 1 to 3 are considered successful. The graph looks at the period between 2007 and 2022 and shows that the distribution has remained relatively stable during this time. For the 2021-2022 period the cumulated distribution is as follows:

2% of all projects have achieved a rating of 1
53% of all projects have achieved a rating of 2 or better
84% of all projects have achieved a rating of 3 or better
95% of all projects have achieved a rating of 4 or better and
100% of all projects have achieved a rating of 5 or better

3. Graph: Distribution of ratings according to project volume, federal budget funds (2007-2022)

This graph shows how the total federal budget funds of evaluated projects are distributed among the achieved results. It looks at the period between 2007 and 2022 and reveals that this distribution has remained relatively stable during this time. For the 2021-2022 period the cumulated distribution is as follows:

2% of the budget fund volume went to projects that have achieved a rating of 1
46% of the budget fund volume went to projects that have achieved a rating of 2 or better
88% of the budget fund volume went to projects that have achieved a rating of 3 or better
98% of the budget fund volume went to projects that have achieved a rating of 4 or better and
100% of the budget fund volume went to projects that have achieved a rating of 5 or better

4. Graph: Estimated success rate and 95% confidence interval

This graph describes the estimated success rate including the respective confidence interval (95%) for the two-year intervals since 2007. For all two-year intervals between 2007 and 2020 the two-sided confidence interval was constantly near plus/minus 5% Only for the most recent interval 2021-2022 the confidence interval is significantly higher at plus/minus 9%, which is due to the fact that this estimate - for now - is based on fewer observations i.e. completed evaluations. The average success rate estimates for the respective two-year-intervals are:

2007-2008: 81%
2009-2010: 78%
2011-2012: 79%
2013-2014: 83%
2015-2016: 82%
2017-2018: 79%
2019-2020: 85% and
2021-2022: 84%.

1. Graph: OECD-DAC Criteria

The graph shows the 6 evaluation criteria (relevance, coherence, effectiveness, efficiency, impact and sustainability) as well as the question of how to approach each criterion. Issues include

Relevance - are we doing the right thing?

Coherence - Is the intervention compatible with other interventions and political goals?

Effectiveness - do the project achieve its objectives?

Efficiency - Are the results achieved in a cost effective manner?

Impact - Does the project help to achieve overarching goals?

Sustainability - Are outcomes long-lasting?

2. Graph: Rating scale

The graph on the score scale shows a 6-step rating of very successful (rating scale 1), successful (rating scale 2), limited success (rating scale 3), rather unsuccessful (rating scale 4), mostly unsuccessful (rating scale 5) and completely unsuccessful (rating scale 6).

Rating scale 1: very successful, result that clearly exceeds expectations

Rating scale 2: successful: fully in line with expectations and without any  shortcoming

Rating scale 3: moderately successful: project falls short of expectations, but the positive results dominate

Rating scale 4: moderately unsuccessful, significantly below expectations, with negative results dominating despite discernible positive results

Rating scale 5: unsuccessful: despite some positive partial results, the negative results clearly dominate

Rating scale 6: highly unsuccessful: the project has no impact or the situation has actually deteriorated