A project manager who has to deliver the evaluation report for a completed project on a Friday afternoon quickly writes: "The objectives were achieved and the collaboration went well." That one sentence doesn't answer any of the six questions the OECD DAC criteria ask, and that's exactly where most evaluation reports get stuck.
Six criteria, six questions
Relevance. Did the project align with a genuine need at the time it was carried out? A project can be executed technically perfectly and still have become irrelevant, because the original rationale has since changed.
Coherence. This criterion was added in the 2019 revision and asks: does this project fit with other policy, other programmes or other projects within the same organisation? Does the project reinforce them, or does it get in the way of another initiative?
Effectiveness. Were the intended goals actually achieved? This is the most obvious question, and at the same time the question that is answered most vaguely in many evaluation reports: with a narrative summary instead of a testable statement.
Efficiency. Were the goals achieved with a reasonable input of time, money and people? Effectiveness and efficiency are often used interchangeably, but they are two separate questions: a project can be effective (the goal was achieved) and still inefficient (at a much higher price than necessary).
Impact. What are the broader, longer-term effects of the project, including effects that were not foreseen? This criterion looks beyond the project's direct results to its wider ripple effects.
Sustainability. Do the project's benefits persist after the project has closed and the project team has been disbanded? This criterion explicitly asks about the financial, social and institutional capacity to sustain results over time. It is directly related to the benefits realisation question that keeps recurring from evaluation to evaluation.
Two usage rules that matter as much as the six criteria
The OECD guidelines contain two warnings that are at least as important as the criteria themselves. First: the criteria must not be applied mechanically. They are meant to generate evaluation questions that fit the purpose and context of the specific evaluation, not to serve as a checklist to be ticked off without thought. Second: not every criterion is equally relevant in every evaluation. An evaluation of a short internal IT project has little use for an extensive coherence analysis; an evaluation of a multi-year transformation programme has plenty.
Why "the standard" is too big a claim
It is tempting to present the OECD DAC criteria as the global framework for project evaluation. That is not accurate. The criteria were developed for the context of development cooperation, where they have been used since the 1990s to assess programmes and interventions. That they are now widely applied outside that domain, in government policy, in non-profit evaluations, and increasingly in evaluations of IT and infrastructure projects, is a real fact, but the assumption that this makes them the standard for, say, IT or construction project evaluation does not hold. What is true: these are six criteria that, in combination with frameworks such as the British Gate 5 review, PRINCE2 practice and the CIPP model, show a striking degree of agreement on what they expect from a good evaluation. Testing benefits against the original promise, using pre-set criteria, and looking beyond the project itself.
How this lands in an evaluation report
An evaluation that applies the OECD criteria seriously does not automatically pick all six, but argues which three or four are most relevant for this particular evaluation, and then works those out substantively, with an explicit evaluation question per criterion, not a catch-all paragraph that summarises everything in a single sentence.
EvaluatieScore uses the OECD DAC criteria as one of four normative anchors behind its 7×3 rubric, alongside the British Gate 5 review, PRINCE2 practice and the CIPP model. Category 1 of the rubric (scope, purpose & evaluation questions) directly tests whether an evaluation applies pre-set, argued criteria, rather than generic headings.
Upload the evaluation report for €15 at dutchmind.com/producten/evaluatiescore and see how it scores on these six questions.
Sources
- OECD DAC Network on Development Evaluation (EvalNet) (2019), Evaluation Criteria (herzien, DCD/DAC(2019)58 FINAL) — one.oecd.org.
- HM Government / Cabinet Office–IPA (nu NISTA) (2021), Gate 5: Operations Review and Benefits Realisation (Assurance Portfolio Standard, V1.0) — assets.publishing.service.gov.uk.
- PRINCE2-praktijk — End Project Report en Lessons Report Template (prince2.wiki, secundaire bron, geen officiële AXELOS-manual).
- Stufflebeam, D.L. (2015), CIPP Model checklist (2e ed.) — rszarf.ips.uw.edu.pl.