Post-Implementation Review

Was it worth it?

A few months after a kitchen renovation, you know whether it was worth it. Do the drawers still stick? Do you actually cook more? Did it cost what the builder quoted? And what would you do differently next time?

A post-implementation review (PIR) asks the same questions about a new system. The yardstick is what was promised: the business case and the requirements.

A post-implementation review checks objectives, benefits, costs, controls, users and lessons.

Does the system meet its objectives and requirements?
Are the benefits in the business case showing up?
How did actual costs compare with the budget, and what does it cost to run?
Are controls working as designed, including audit trails and security?
Are users satisfied, or are they relying on workarounds?
How well was the project run, and what should future projects do differently?

A PIR isn’t only about benefits. Controls and user satisfaction are just as much part of it.

When to do it

Week one is too early, three to six months is about right, two years is too late.

Usually some months after go-live, once the system has settled down and there’s enough data to measure benefits. Too early, and all you see is teething problems. Too late, and the team has moved on and the lessons are lost.

Who does it

The project office, an independent team, or internal audit. Whoever does it must be independent of the project. An auditor who helped design the system’s controls, or approved it, shouldn’t review it: that would be marking their own homework.

What comes out

an assessment against the business case and requirements
issues and corrective actions, each with an owner
lessons learned, fed back into the organisation’s SDLC and project methodology (they’re useless otherwise)

Example. Eight months after a procurement system went live, a PIR finds that purchase order cycle time improved as planned. But 30% of purchases still bypass the system through urgent purchase cards, which have weaker approval controls. The review recommends tighter card limits and monitoring, and its lessons note that process change should have been in the project scope.

How the exam asks about it

If the question asks…	The answer is usually…
Primary purpose of a PIR	Determine whether objectives and benefits were achieved and controls are adequate
When a PIR should be performed	After the system has stabilised, often several months after go-live
Document used as the baseline for a PIR	The business case and requirements
Auditor who designed controls for the project	Shouldn’t perform the PIR

In short

A post-implementation review (PIR) evaluates whether the system meets its objectives, delivers expected benefits, and has adequate controls.
It's typically done some months after go-live, once the system has stabilised and benefits can be measured.
It compares actual costs, benefits and performance with the business case.
It also evaluates the project process itself, capturing lessons learned.
The IS auditor may perform or contribute to a PIR, but shouldn't review a project they helped design or approve.
Findings lead to corrective actions and improvements to future projects.

Exam traps

Doing the PIR immediately after go-live is too early to measure benefits.
A PIR isn't only about benefits; control adequacy and user satisfaction are also assessed.
An auditor who was part of the project team shouldn't perform its PIR.
Lessons learned only help if they're fed into the methodology for future projects.

Slides

Click the slide and use the arrow keys. F toggles full screen.

If the question asks…	Answer
Primary purpose of a PIR	Were objectives and benefits achieved; are controls adequate?
When to perform a PIR	Once stable, often several months after go-live
Baseline for a PIR	Business case and requirements
Auditor who designed the project's controls	Shouldn't perform the PIR