How to Conduct Post-Future Project Evaluations

The website finally went live last week, and the entire staff is throwing a party to celebrate! The developers are huddled in the corner with some microbrews, plotting how they'll splice into the agency intranet to add in a virtual dartboard. Designers are mingling with the copywriters and account people, clinking wineglasses and bonding over the ads they saw during The Office--did you see that amazing CG work, where the car dissolved into a field of flowers?

Not the best time to mention that tomorrow, you're scheduling a post-future meeting (nee post-mortem) to talk about how the project really went.

Yes, the job went way over budget — and the last thing they want to think about is who needs to take responsibility for it. Was the estimate wrong to begin with? Did the designer spend too long tweaking those page comps? How come the developer pulled so many late nights, when he said he knew .NET? Besides, they'll still be nursing their hangovers. If you're the one in charge, you get to take responsibility for the result.

Or do you?

Discovering how a creative agency fails to make profit on a project usually boils down to a series of in-process decisions that, while intended to be good, lead to cost overruns and errors. Isolating and clarifying those agency decisions role by role can be painful and time-consuming if done incorrectly -- but if carried out in the right manner and in a group setting, a post-future meeting can galvanize a team and bring them closer together. By being aware of how they've behaved in the past, they'll be able to see repeated patterns and anticipate ways to stop them from happening again.

A post-future meeting affords more than just saying, "Well, next time we'll do it better." It can give your staff real visibility into how everyone's jobs are interrelated. Those dependencies are what cause true profit on a creative task. Agency professionals get budgets and hourly estimates of what it'll take to do a creative task, but those estimates can be severely influenced by staff interaction. Day-to-day miscommunications may seem like they're out of managerial control, but they aren't. Fostering dialogue and collaboration is what truly makes profit possible, especially on large, multi-phase projects that continue over a series of months.

Even if you're a solo-flight designer, answering the following questions after completing a big project will help you take a dispassionate look at how you're doing business.

 

Structuring the Post-Future Meeting

The post-future process requires the same planning, deliberation, and care as the approach you took to fulfilling the project in the first place. Think of it as an anthropological journey into exactly how your agency functions. Everyone on the team may not be aware of the true challenges they were up against. Agency staff are often shielded from the brunt of client feedback or the sheer complexity of technical tasks because it would burn up agency time to describe them in detail. The post-future gives staff a chance to really unfold what challenges hit each member of the staff and create a dialogue about how everyone can be supported by their agency peers.

This critique could be structured as follows within a single hour, or in the case of really big and/or fraught projects, even longer. Even very small shops or solo-flight designers should consider walking themselves into this process simply to get out of their daily workflow and explore, separate of their everyday client concerns, the real impact of their design process on their business performance.

 

1. Set the Tone for the Meeting (3-5 Minutes)

Everyone in the room should be aware that nothing said by any team member will "go down on their permanent record." You are taking part in the post-future to learn from your colleagues. Nothing will be taken personally against anyone who offers deep feedback. The only behavior that isn't allowed is finger-pointing. If mistakes were made along the way, more respect will be given to those who own up to causing them. If an issue is large enough to warrant at least five minutes of group discussion, promise to follow up with the team members individually to talk further and outline next steps to address similar situations in the future.

 

2. Describe the Business Problem and the Proposed Agency Approach (5-10 Minutes)

Clearly describe what the client wanted to achieve from the project. Walk at a very high level through the proposal the client signed, if anyone in the room wasn't aware of the agreed-upon terms. Ask your team to shoot holes in the proposal, versus the final delivered project. Take the shots as they come and don't be defensive. Common questions during this phase include:

Were there challenges that could have been foreseen from learnings via previous agency projects? Was key information not shared with this team or assumed to be available? It's incredible how important discoveries are provided anecdotally and never recorded anywhere for posterity at creative agencies.

Were there any issues with how the proposal was structured or written that caused hiccups in the project? A good follow-up to this question flows into the next section of the post-future:

Were there deviations that occurred during the project that could be traced back to a poorly defined scope? Did you deliver at the end what you'd promised in the proposal, or was it weakly worded? Perhaps promises that were made in meetings but not in writing that created undocumented client expectations? On heavy projects such as Flash website or application design, features that come up in casual client conversation can create holes in proposals large enough to drive a truck through.

 

3. Work Through How the Agency Executed the Project (35-40 Minutes)

This portion of the post-future is the deep dive into what happened, and why, over the life of a project. Talk through the major milestones of the project process and key points where there may have been rework beyond your usual creative process.

If necessary, have on hand the artifacts that carried the project to completion: creative brief, technical and functional specifications, client communications, vendor input, and the paper/digital trail of design work occurring over the project. They aren't in the room to be examined in-depth; they exist in the room to prompt people's memory about warning bells that may have foreshadowed project challenges.

Also, be mindful of agency process and where the project conformed or deviated from those standards. (Note: If you can't articulate your agency process or your standards of how a project should behave, then schedule a series of meetings with your staff to document the ideal process a project should conform to in your agency. Otherwise, you never have a standard to measure what path a profitable project should take through your company -- or what the term "profitable project" even means.)

Key questions that come up during this phase include:

Was the creative brief an accurate reflection of what is in the proposal and the strategic direction from your agency? A creative brief is not a proposal reformatted for your creative team. Key elements missing from a brief such as audience demographics or a clearly articulated value proposition are a major time imposition for designers. The designers would then second-guess the brief and form their own opinions to validate their creative thinking -- using valuable agency time usually spent concepting against a clearly delineated business problem. Plus, this also assumes that you aren't going to burn even more time working out the strategy through the actual design executions.

Did the deliverables line up with what the agency actually created for the client? If you showed 4 concepts instead of 2 concepts, what kind of impact did that have on your creative budget?

Were there client requests that changed the project strategy from the approved brief while the project was in-process? If not, did the agency absorb into the project cost getting the client to the right strategy? And were you paid for that extra time investment? What were the repercussions?

Were there technical challenges that occurred during the life of the projects that were unexpected? Should they have been expected and factored into the schedule?

Did the team have a complete knowledge of what the project required? Were you penalized for the team verging outside their area of core expertise? Did you factor that time into the project budget to support the time they needed to gain the knowledge necessary to succeed? Another way of phrasing this issue is: Did you bid the project for your best case scenario, while knowing full well that best case scenarios never occur if they're contingent on technologies or unique deliverables that are new to your team?

Did lack of communication or personality friction dictate staff behavior? This is something that may not be discussed in the post-future, but will need to be acknowledged out loud to said parties.

Did the team spent time scrambling to address client concerns without proper triage? Did key decision makers make client feedback actionable and mitigate what was feasible before it hit the agency?

Was the proper chain of command followed through each phase of the project? This one should speak for itself. If rework occurred because of avoiding a key approval internally, you asked for it.

Did your vendors fall in line with your agency process/timeline? Did they contribute to your success or provide further hurdles to surmount?

This is only a sampling of questions you can use to start meaningful conversation.

 

4. Show the Staff, in a Pleasing Visual Format, Where the Money Went (5-10 Minutes)

In any creative agency, your staff should have no illusions: time equals money. You can create amazing design work and put yourself out of business. There must be a balance between quality of work delivered and overhead and profit for your company. So show them what your financial targets were and how the agency performed against them, in aggregate and by project stakeholder.

It's easy to recount the overall cost of a project to an agency (including overhead), versus the revenue from the client. What's more difficult, especially in a group setting, is showing each stakeholder where and when they billed their hours. In this context, it's hard for a staff member to argue when the principal notes the following:

How much time was used by your designers for creative concepting versus execution of the project? Designers can often misjudge their budgeted time when chasing potential design solutions. Show them a well-designed graph generated from their billed hours each day and that indicates where their hours were bunching up over the course of a design project.

Designers don't run the business, so why should they know how profitable their project is for the owners or parent company of the agency?

The answer to this is simple: If designers see how their behavior has real influence on the quality of life (and profit) for their company, they will think more holistically about the result of their behavior and ask for help if they see they are about to breach the budget for their work. Otherwise, what incentive will they have to be mindful?

Designers yearn against boundaries and sometimes want to act in a manner that works outside the box of a particular project. If their behavior didn't have an impact on the overall project profitability independent of addressing other project issues -- such as a botched creative brief or multiple rounds of unclear client feedback -- they get a pass. If you've given them a realistic time estimate and they've missed the mark, offer them the carrot of working with their fellow creatives to learn how to work more efficiently.

Be aware that the surest way to piss off a designer is to give them no control of their time estimate on a projects, then ding them for not meeting the estimate. Always solicit their feedback on an estimate before the client signs it.

Where and when did mismanaged expectations burn up profit? If the account manager caves on a client request when it's clearly not in the project scope, then everyone on staff pays for it. The same goes for an excited designer who had a few bonus concepts make it in front of the client, causing an extra round or two of extra edits until they were retired. It comes out of the company's pocket. Often we argue against this by talking about the lifetime value of a client, we'll get more work in th future, and so on. This kind of talk is only merited if you have a very steady flow of work with said client, as well as a strong long-term relationship. In the short term, providing gimme after gimme can be paralyzing in terms of cashflow.

There are always hidden costs in any "easy" fix or change to a client deliverable. I have nightmares about making free "minor" changes to a web page (15 minutes) and then burning through 8 hours of agency time to test, debug, and resubmit the URL to the client. It's very important to educate the client when necessary on what their changes will require, just to ensure that you address their concerns in a way that accomodates their needs and won't break your budget. (Change order?)

Based on what we know now, should we have taken the project? Finally, if necessary, the agency principal or manager gets to fall on their sword. Design agencies are often happy to chase the bucks, even if it's in an area of expertise that is unproven, just to keep their staff. The learning curve can be costly in the short term, and everyone needs to be prepared for it. Was this project an investment in a new discipline for your agency, a great piece for your portfolio that came at extra expense, or a big waste that you're never going to attempt again? Your staff will likely appreciate seeing real big-picture thinking around exactly what this project meant to the agency -- just as long as it's always delivered in a positive manner.

This kind of transparency and candor is rare for most agencies. But when you hear this kind of feedback from the top, it's worth millions to your staff morale.

 

Create a Space for Reflection -- Not Blame

Don't feel like all of this activity has to happen in the post-future meeting. And when scheduling the meeting, think about what you can attach to the meeting request that can help set the tone. But be sure not to start a blame game! Post-future meetings should always be constructive, and be conducted in a manner that is professional and respectful.

I have fond memories of sitting in tense meetings to discuss major agency errors. The Powers-That-Be would pull out a mighty tome of previous mistakes, and duly record in tiny print the names of those who were found to be guilty as charged of misspelling the word "the" as "teh" or mailing the final brochures to Mississippi instead of Missouri. When the names of the guilty parties had been recorded in the book, it would return to the shelf, where it slept, waiting for the next victim. Is that the most constructive and supportive way to grow the knowledge of your staff?

In parting, I leave you with this: Mistakes happen. It's a fact of doing creative work, which is founded on exploiting mistakes to create well-considered artifacts. And everyone within an agency needs to learn from mistakes -- not just the one who may have caused them in the first place. So listen, learn, improve, and maintain a level of humanity and respect that will leave everyone smiling instead of smarting when they leave the conference room.