In the archive of a housing association sits the evaluation of the 2019 IT migration. Conclusion: stakeholder communication fell short, and residents and neighbourhood teams heard about the changes too late. In 2022, a comparable migration takes place in another part of the organisation. The evaluation of that project names, almost word for word, the same cause. In 2025 it happens a third time.
Three evaluations, the same conclusion three times. Nobody had put the logs side by side.
Documenting is not the problem
Every project had a lessons log. Every project had someone who took the evaluation seriously and named the cause. The problem is not the documentation; it is what happens with it after it is written. Williams (2007) describes exactly this mechanism: without a deliberate effort and a designated process, lessons are rarely incorporated into broader organisational policy. A single evaluation report is a snapshot of one project. Whether it is part of a pattern only becomes visible when you lay several reports side by side, and that comparison almost never happens by itself.
The reason is organisational, not personal. A project manager evaluates their own project, delivers the report and moves on to the next one. Nobody in that role has the mandate, or the time, to read back through three years of reports and ask: is this the third time we have run into this?
A rubric that makes comparison possible
Comparison is only possible if evaluations are built the same way. A lessons log written as free text, what went well and what went less well, mostly produces noise when placed side by side. An evaluation that tests against criteria set in advance, with a fixed set of questions per project, produces comparable outcomes. Predetermined criteria are one of the building blocks that OECD DAC, Gate 5, PRINCE2 practice and CIPP arrive at independently of one another, not only to judge one project but to make outcomes repeatable and comparable.
Without that fixed structure, comparing remains guesswork. With it, it becomes a quarter of an hour's work: open three reports, look up the same question in each, and set the answers side by side.
Who puts the logs side by side?
The question that stays open in practice is not whether you can compare, but who does it. In the association's example, the answer did not lie with the project managers, who were rightly busy with their own project. It lay with nobody, until a portfolio manager decided to lay the last five evaluations from the IT domain side by side for a quarterly overview. Within an hour, the recurring cause was on paper.
That is not a capacity problem of a few extra hours per evaluation. It is a process choice: who is structurally given the task of reading evaluations across projects, rather than only their own? As long as that role is not assigned, every evaluation remains an island.
Start with comparing
You do not need to wait for a new process to begin. Take the last three evaluations of comparable projects in your organisation, look in each report for the stated root cause, and set those three answers side by side. Does the same thing appear twice? Then you have a pattern that a single isolated evaluation report could never have shown.