How to Conduct Post-Mortem Project Evaluations

This is an extensive rewrite of a previous ChangeOrder post for my next book Design Business from A to Z—so much so I'm reposting it!

The website went live last week, and the entire staff is throwing a party to celebrate! The developers are huddled in the corner with some microbrews, plotting how they'll splice into the agency intranet to add a virtual dartboard. Designers are mingling with the copywriters and account people, clinking wineglasses and bonding over the ads they saw during The Office.

Yes, the job went way over budget—and the last thing your team wants to think about is who needs to take responsibility for it. Not the best time to mention that tomorrow, you're scheduling a post-mortem meeting (a.k.a. lessons learned, post future, etc.) to talk about how the project really went.

Was the estimate wrong to begin with? Did the designer spend too long tweaking those page comps? How come the developer pulled so many late nights wrangling with the content management system, when he said he knew .NET?

Discovering how a creative agency fails to make profit on a project usually boils down to a series of in-project decisions that, while intended to contribute to project success, lead to cost overruns and errors. Isolating and clarifying those agency decisions, role by role, can be punishing if conducted incorrectly. But if carried out in the right manner and in a safe group setting, a post-mortem meeting can galvanize a team and bring them closer together. By being aware of everyone's perspectives, your team members can see repeated problems in patterns of behavior and discover ways to change them. Plus, the ongoing learning that comes from open communication and active collaboration is what makes businesses more sustainable—especially on large, multi-phase projects that continue over months, if not years.

 

The flow of a well-structured post-mortem meeting

Here’s a draft agenda for an hour-long post-mortem meeting. Make sure you are meeting in a space that has a large whiteboard, so you can capture what everyone says as the meeting unfolds. And when scheduling the meeting, think about what you can attach to the meeting request that help ground people in what you’ll accomplish during the meeting.

1. Set the tone for the meeting (3-5 minutes). The post-mortem process requires the same deliberate care as the facilitation approach you would take for any form of client collaboration. Post-mortems should always be constructive, and be conducted in a manner that is professional and respectful. It often helps to have the meeting leader be someone who wasn’t on the project. They will be responsible for taking notes on the whiteboard.

Kick the meeting off by letting everyone know the goal for the meeting: understanding what went well and what could be improved about your recently completed project that you can apply to future projects. There must be a clear balance between the two buckets, or the room may only linger on complaints about everything that went wrong.

Everyone in the room should be aware that nothing said by any team member will go down on their permanent record. You are all taking part in the post-mortem to learn from your colleagues. Nothing will be taken personally against anyone who offers deep feedback. If mistakes were made along the way, more respect will be given to those who own up to causing them. No finger-pointing!

Let everyone know that if an issue is large enough to warrant at least five minutes of group discussion, promise to follow up with the team members individually to talk further and outline next steps to address similar situations in the future.

2. Describe the business problem and the proposed agency approach (5-10 minutes). Clearly describe what the client wanted to achieve from the project. Move at a very high level through the proposal the client signed, if anyone in the room wasn't aware of the agreed-upon terms. Ask your team to consider how the proposal did (or didn’t) guide the team towards the final delivered project. Common questions during this phase of the post-mortem may include:

• Were there challenges that could have been foreseen from previous agency projects that weren’t factored into the proposal?
• Were there any issues with how the proposal was structured or written?
• Were there deviations that occurred during the project that could be traced back to poorly defined scope?
• Were promises made in early meetings, but not in writing, that influenced client expectations?

3. Walk through how the agency executed the project (30 minutes). This portion of the post-mortem is a deep dive into what happened, and why, over the life of a project. Discuss the major milestones of the project process and key points where there may have been rework beyond your usual creative process. The staff can jump around in time as appropriate; this doesn’t have to be linear.

If necessary, have on hand the artifacts that supported project completion: creative brief, technical and functional specifications, client communications, vendor input, and the paper/digital trail of design work occurring over the project. You shouldn’t be using the time to re-proof or critique the minutiae of each deliverable. They exist in the room to provoke your team’s memory banks.

Here’s a sampling of common issues that crop up during this type of discussion:

• Did you fulfill what you'd promised in the proposal?
• Did the deliverables line up with what the agency actually created for the client? If not, why?
• Was the creative brief an accurate reflection of what is in the proposal and the strategic direction from your agency?
• Were there client requests that changed the project strategy from the approved brief while the project was in process? If not, did the agency absorb into the project cost getting the client to the right strategy? Were you paid for that extra time investment? What were the repercussions?
• Were there technical challenges that occurred during the life of the projects that were unexpected? Should they have been expected and factored into the schedule?
• Did the team have a complete knowledge of what the project required?
• Were you penalized for the team verging outside their area of core expertise? Did you factor that time into the project budget to support the time they needed to gain the knowledge necessary to succeed? (Another way of phrasing this is: Did you bid the project for your best case scenario, while knowing full well that best case scenarios rarely occur if they are contingent on technologies or unique deliverables that are new to your team?)
• Did the team spent time scrambling to address client concerns without proper triage?
• Was the chain of command followed through each phase of the project? (Is there one?)
• Did your vendors fall in line with your agency process/timeline? Did they contribute to your success or provide further hurdles to surmount?
• Did lack of communication or personality friction dictate staff behavior? (This is something that may not be discussed in the post-mortem, but will need to be acknowledged to said parties.)

4. Solve for perceived problems (10 minutes). At the end of the previous phase, you should have a whiteboard list of your team’s thoughts and impressions regarding what went well and what could have been improved on their projects. Now is your chance to be problem solvers and suggest changes to your agency process! Circle pain points and ask your team to brainstorm ways to keep them from happening in the future. When the brainstorming is over, copy down everyone’s ideas and send them to the team after the meeting. Seek out owners and action items for making sure the changes stick.

5. Show the team, in a pleasing visual format, where the time and money went (5 minutes). In any agency, your staff should have no illusions: time equals money. There must be a balance between quality of work delivered and profit for your company. So, if you can, show your staff what your financial targets were and how the agency performed against them. With this visualization in hand, you can ask your team (and yourself): Based on what we know now, how would we approach a project like this in the future? Was this project an investment in a new discipline for your agency, a great piece for your portfolio that came at extra expense, or something you're never going to attempt again?

This kind of transparency and candor is rare for most agencies. But when you hear this kind of feedback from everyone across your organization, it can be worth millions to your staff morale. I’ve even worked at agencies where the final post-mortem has included detail regarding how each team member billed their hours. (This means your entire staff needs to keep accurate timesheets, and not lie if they’re going to go over their allocated hours on a project.)

Some of your staffers may not run the business, so why should they know how profitable their project is for the owners or parent company of the agency?

The answer is simple: if designers see how their behavior influences stability (and profit) for their employer, they can think more holistically about how their actions in the future. Did mismanaged client expectations burn up staff time and profit? Did the team suggest placing a few “bonus concepts” in front of the client, causing extra rounds of changes before down-selection to the final direction? All of this comes out of the company's pocket, and can be gauged in this closing discussion. Your staff will likely appreciate seeing real big-picture thinking around exactly what this project meant to the agency—just as long as it's always delivered in a constructive manner. You can then fold this into the solutions you discussed earlier.

6. Highlight the most important thing each team member learned (5 minutes). To close your post-mortem meeting, ask each person in the room to put two stars on the whiteboard. One star is drawn beside the most important positive thing to remember from the meeting. The second star is drawn next to the one thing that should be changed going forward. When everyone is done, the facilitator should ask any last clarifying questions regarding everyone’s selections, then thank everyone for coming. Notes from the meeting should go out to everyone in a day or so.

 

Don’t wait until the end to reflect on project success (and failure)

The post-mortem gives staff a chance to fully comprehend what challenges hit each member of the staff and create a dialogue about how everyone can be supported by their agency peers. Think of it as an anthropological journey into exactly how your agency functions.

But you don’t wait until the end to reflect on what’s working for your project. You can hold a Pre-Mortem, a Mid-Mortem, whatever you want to call it! See how your team is doing, and start to solve for open issues before the project comes to a close.

Got any tips for designers looking to conduct post-project evaluations? Share them in the comments!

Photo 3202963823 by Sudhamshu on Flickr, shared via a Creative Commons Attribution 2.0 Generic License.

Comments

[Sylvie]

This is an EXCELLENT post! Thank you so much for writing it, I noticed no comments and just wanted to leave one so you know it was not for naught! I had just completed an end-of-project evaluation for a different project before I came across this, but I have yet another to do so I will absolutely put some of these items into action. Not only that, I really think a "pre-mortem" assessment, like you suggested, should be done for future projects, particularly when starting a big scale project. I know I am...

[Mpfolio.wordpress.com]

I love this site! When is the book due out? I want.

I've worked for several small companies, where I learned a lot through helping to grow them, and this site helps me to better organize my experiences with new and well-defined information and approaches.

I've had mostly horrible (and un-paid) freelancing experiences (I'm new to it), but everything I read here helps me react to or avoid the problems I've had, and conduct myself as a business rather than just a part of one.

[K.S.]

Really enjoy your design project posts. I learned the ADDIE model: Analyse, Develop, Design, Implement, Evaluate which you incorporate in your posts. Well done.

[J.G.H.]

My agency is just beginning to conduct post-mortems. This post is the most helpful guide I've found on the Internet. Thanks for sharing such valuable (and hard earned) knowledge!