Project Post-mortems

Running better projects by learning from the past.

The unfortunate reality with Project Post-mortems (also called Project Debriefing or Lessons Learnt) is that everyone thinks they are a great idea, but they rarely ever get done. I would say the main reason for this is because upper management generally doesn’t think its worth the commitment of resources. I have no sure-fire way for remedying this, but a good way of getting support from management is to stress that it wont take much time (e.g. 2 hours worth of debriefing meetings with technical staff, 2 hours to analyse the data, 2 hours to write the report).

What’s the point of a post-mortem you may ask? The obvious answer to this question is so you can learn from your mistakes (e.g. “I know for next time to start putting in the request for the SSL certificate much earlier”). But there are some subtler benefits to be had from creating this report. For one, it lets you analyse the effectiveness of your QA process (e.g. how many bugs in total were logged? how many bugs did you find vs. the client?, how many bugs seem to be the result of coders not following the spec?).

Probably the most important reason for undertaking a project post-mortem is the least obvious; its to engage technical staff, to ask them how they felt about the project. Far too often programmers feel they have been rail-roaded into doing things they didn’t want to do during a project, for example; cutting features, following technology choices made by management, having to work over-time because of poor time planning, etc. You must listen to people – that doesn’t necessarily mean you enact the ‘nerf guns for everyone’ policy your zaniest programmer requests, it just means sincerely listening to what they have to say.

If possible, I would suggest implementing at least one change based on your development teams’ feedback since this demonstrates you are actually listening. For example, if programmers are saying “we worked a lot of over-time to deliver this project, we should get time in lieu”, I wouldn’t propose getting them that time in lieu, I would instead suggest getting each programmer some kind of token gift to recognise their efforts (time in lieu isn’t the point, being appreciated is). Also, I’m against presenting gifts or bonuses to staff in meetings or in public settings. When management types do this, I think there is a certain aspect of ‘hey everyone, look what a good manager I am’ to it – instead, you do it in private, you just say “thanks Tony for the extra time you put into the project. I got you a Borders gift voucher for $30. I know its not much, but I just wanted to let you know we did spot the extra effort”.

The structure of the post-mortem report I use is very streamlined, being only two pages long. If your report comes out to be longer then that, consider stripping information until it gets down to two pages (remember, no one likes to read long documents). The process itself is broken up into three stages; debriefing the development team - where you ask programmers to give their thoughts on how the project went, data analysis - where you produce some simple statistics based on the bugs logged during the project, and lastly, writing the report itself – where you summarise both the feedback from programmers and make statements about the bug statistics. You may also want to include a conclusion at the end of the report where you ‘bring it all together’ and perhaps make some statements about what went well in the project and what could be improved.

When it comes to the debriefing session with the programmers, try find an independent party to run it. Their role is just to ask the questions you’ve prepared, note down the answers, and watch the time (two hours maximum, if it needs to go beyond that – bad luck). Having someone outside of the project asking the questions allows programmers to answer questions in an open manner; the project post-mortem isn’t a witch hunt. What sort of questions should you ask? Some examples would be; “are you proud of the work you did on the project – if yes; what in particular? If no; what went wrong?”, “what was the single most frustrating part of the project?”, “how would you do things differently next time?”, “what was the most satisfying part of the project?”, “which of our processes worked well?”, “which of our processes were hard to use?”, “if you could wave a magic want and change something about the project – what would it be?”, “did the customer participate effectively? If not, how could we improve this?”.

If you have been tracking bugs over the course of the project, you will be able to analyse these to produce some basic statistics. The sorts of stats I like to go for include; how many issues were logged during the project (and how many of these were bugs vs. feature requests), how many bugs were critical ‘show stoppers’ vs. low priority (like UI issues), how many of the bugs did you find compared to how many the client found (this is a good indicator of the effectiveness of your QA process).

Lets talk about the structure of the report itself. As is often my habit, I begin with a Document Purpose paragraph (e.g. “this document is a deconstruction of how the project went. The hope of this is to learn from our mistakes and identify successful processes which should be continued.”). You will most likely need to edit the answers given by the development team during their debriefing session; for instance, if one of the programmers has said “the client was a complete ass, he kept changing his mind every 10 minutes”, that’s probably not the best of things to be recorded in a document which upper management may see. This kind of statement would be converted to something like “members of the development team felt that the client was indecisive with regard to their requirements”.



For the Bug Analysis section, go for bullet point statements like “22.5% of all bugs logged were found by the client, the remaining 77.5% were found by us”.

Lastly, the conclusions paragraph is what its all about. Where you have a chance to make some grand statements which are backed by anecdotal statistics. You may want to say things like “the bug statistics indicate that as a team, we are not following specifications closely enough” or “based on feedback from the development team, there seems to be a consensus that a more detailed project schedule is needed”.

Will upper management really care about the project post-mortem report; probably not, at least not directly. But the post-mortem report is like a project schedule or spec, upper management generally wont be interested in the details, but they will appreciate that you have done it and it will be a reflection of your professionalism. Just be sure that it doesn’t become a mini-project in itself.